web-infra-dev /midscene
See interfaces like a human. E2E testing in natural language.
★ 15,616⑂ 1,223👁 69⑃ 2,697● 689▤ 228
+6
日增长
15,616
stars
1,223
forks
69
watchers
2,697
prs
689
issues
228
releases
README
See interfaces like a human. E2E testing in natural language.
Bring AI vision to E2E testing with a newly designed test framework for web, mobile, and desktop apps.
Website · English / 简体中文
Write a test in natural language
Midscene.js brings AI vision to E2E testing. Describe UI steps and expected results in natural language, then use the visual GUI Agent to carry them out and verify what users see. Organize tests with Midscene Test, or integrate the Agent APIs into your existing testing framework.
YAML test cases with Midscene Test
After creating a Midscene Test project and configuring a model and the Web runtime, write a case like this. Replace the example URL with your app's address:
yaml
beforeEach:
- gotoUrl: https://your-shop.example
cases:
- name: Find headphones under $100
steps:
- aiAct: Search for headphones and filter the results to prices under $100
- aiWaitFor: The filtered search results are visible
- aiAssert: There are products in the search results, and each costs less than $100
JavaScript in your existing tests
You can also use the Agent directly in your existing tests. Configure a model and open your app in a Playwright page, then describe the same steps in code:
javascript
import { PlaywrightAgent } from '@midscene/web/playwright';
const agent = new PlaywrightAgent(page);
await agent.aiAct('Search for headphones and filter the results to prices under $100');
await agent.aiWaitFor('The filtered search results are visible');
await agent.aiAssert('There are products in the search results, and each costs less than $100');
For mobile and desktop apps, use the platform Agent with Rstest or Vitest. Open the generated HTML report to inspect screenshots, element locations, the AI decision process, and action and assertion results.
👁️ See, act, and verify like a human
E2E testing verifies complete workflows from the user's perspective, so Midscene tests the way a user does: observe the screen, act on what you see, and verify the visible result.
Locate and act from screenshots
Midscene locates elements by their appearance and position, then clicks, types, or scrolls to carry out your instructions. You can target icon-only buttons, custom controls, , and elements inside cross-origin iframes without maintaining selectors or adding semantic annotations.
Verify what users see
Describe the expected appearance in natural language to check colors, selection highlights, layout, and visual feedback — including content drawn on or displayed in native apps.
javascript
await agent.aiAssert('The selected plan has a blue border and a checkmark');
await agent.aiAssert('The error message is visible below the email field');
🌐 One API, across platforms
Visual localization does not depend on the UI rendering stack, so the same Agent APIs work on every platform:
| Platform | Integration |
| --- | --- |
| Web | Playwright, Puppeteer, or desktop Chrome |
| Android | Devices aTop issues
#426[Feature request] Support RAG6 comments · 2025-02-25#3213Can only scroll up and down, not left and right on the webpage.4 comments · 2026-10-08#2757Is there more detailed documentation for model selection?4 comments · 2026-07-06#2240[Feature]: Export AI execution steps as reusable Playwright scripts4 comments · 2026-03-30#3205(feat) Load model config from a user-level config file or OS keyring, not just env vars1 comments · 2026-09-30#3040[Bug]: The last level cannot be selected in cascader's multi-level multi-selection mode1 comments · 2026-08-26#3020[Feature]: Add bounded-latency flow control to the Android Studio scrcpy preview1 comments · 2026-08-21#2935[Studio Recorder] Long recording sessions have delayed updates, screenshot degradation, and slow finalization1 comments · 2026-08-05
Latest release
v1.17.8 v1.17.8 2026-11-11English v1.17.8 Changes Features 🎉 feat(core): add Tencent HY model adapter by @EAGzzyCSL in https://github.com/web-infra-dev/midscene/pull/3362 feat(test): remove automatic credential redaction from run artifacts by @xiong35 in https://github.com/web-infra-dev/midscene/pull/3371 feat(badge): support skipped test counts by @yuyutaotao in https://github.com/web-infra-dev/midscene/pull/3380 Fixes 🐞 fix(webdriver): enforce request timeout through response body reads by @quanru in https://github.com/web-infra-dev/midscene/pull/3232 fix(web-integration): honor explicit slow swipe durat