e2e: Natural-language end-to-end testing for web and mobile apps
An end-to-end framework for web and mobile apps that drives them with natural language and replays recorded agent actions.
GitHub tester-army/e2e Updated 2026-10-06 Branch main Stars 4.8K Forks 198
TypeScript End-to-end testing Playwright agent-device Web and mobile

🧭 Decision Guide

Try it if you

  • You test Vite, Next.js, Expo, or SwiftUI apps and want to describe business actions with agent.act.
    The README's Quick start and example code list Vite, Next.js, Expo, SwiftUI, and agent.act.
  • You need Chromium, Firefox, or WebKit coverage while continuing to use Playwright.
    The README's Packages section says @e2e-dev/web provides Chromium, Firefox, and WebKit through Playwright.
  • You already have a subscription, API key, or local model and accept the model dependency of agent steps.
    The README explicitly says, Bring your own subscription, API key, or local model.

Skip it if you

  • Your team cannot accept API or configuration changes before 1.0.
    The README's Status section says APIs and config can still change between minor releases.
  • Your test environment cannot use a subscription, API key, or local model.
    The README says agent steps require your own subscription, API key, or local model.
  • Your compliance requirements prohibit the CLI from sending anonymous data about commands, engines, and failure locations.
    The README's Telemetry section says the CLI sends anonymous usage data and provides an opt-out method.

Requirements

  • When using npx e2e init, select an engine (web or mobile) and a model provider.
  • Tests containing agent steps require your own subscription, API key, or local model.
  • The mobile engine targets iOS simulators and Android emulators.
  • The web engine uses Chromium, Firefox, and WebKit through Playwright.

First step (verbatim from README)

npx e2e init

Watch out

  • Agent steps replay recorded actions on later runs and call the model again only after the app changes.
    The README says later runs have no model calls when the app has not changed and replay the actions.
  • Tests without agent steps do not need a model; model dependency is limited to agent flows.
    The README says Tests without agent steps need no model.
  • The CLI sends anonymous telemetry by default; disable it with npx e2e telemetry disable.
    The README's Telemetry section gives the disable command and lists the telemetry scope.

Not stated in the README

  • The README does not specify the supported Node.js versions.
  • The README does not name the supported model providers or describe their pricing and compatibility.
  • The README provides no latency, stability, or accuracy data for agent.act and agent.assert.
  • The README does not specify minimum system versions for iOS, Android, Chromium, Firefox, or WebKit.
  • The README does not describe CI concurrency, resource usage, or large-suite performance.
  • The README does not provide the complete changes across the 5 releases.

💡 Deep Analysis

6
Yes I own web tests involving billing, plan upgrades, and amount verification. I want the agent to handle complex navigation but cannot let the model decide the final result alone. Does e2e support this hybrid style?
For: A full-stack test engineer who needs both natural-language actions and deterministic assertions for payment and plan-change flows

Yes, it is suitable: e2e is explicitly structured to place natural-language actions together with locators, expect, and deterministic assertions.

  • The README example uses agent.act for the plan upgrade, agent.assert for the prorated invoice preview, and expect(screen.getByRole('status')) for the final Pro status.
  • The documentation says tests describe goals in natural language and check results with locators and assertions, so the model is not the only verification layer.
  • Agent actions verified by later assertions are recorded and replayed, which helps repeated execution of an already validated flow.
  • For amounts, permissions, and data changes, the project insight requires explicit UI- or API-level verification; an agent judgment should not be treated as audit evidence.

Use the agent for exploratory interaction, but keep deterministic checks for billing amounts and final state.

  • README introduction: Describe a goal in natural language and an agent drives the app to reach it. Check the result with locators and assertions in the same test.
  • README example: `agent.act`, `agent.assert`, and `expect(screen.getByRole('status'))` appear in one checkout test.
  • Project insight usage_limitations: amounts, permissions, order status, and data consistency require explicit UI- or API-level verification.
  • README introduction: An agent step that a later assertion verifies records its actions.
npx e2e init
Not stated in the README:The README does not say whether there is a direct assertion API for backend billing or order data.;The README does not specify whether `agent.assert` supports exact numeric comparisons, currencies, or rounding rules.
Yes I maintain TypeScript web tests, and CI cannot depend on an external model service or upload test content. Can I use a local model with e2e while disabling CLI telemetry?
For: A TypeScript test engineer who wants natural-language tests in CI but must use a local model and cannot upload test content

Yes, it is suitable: the README explicitly supports local models and provides ways to disable CLI telemetry, but your team remains responsible for model deployment and accuracy.

  • The README says the model can come from a user subscription, API key, or local model, so an external service is not mandatory.
  • The CLI sends anonymous usage data such as commands, engines, and failure locations, while explicitly not sending test content, app content, or credentials.
  • You can run npx e2e telemetry disable or set E2E_TELEMETRY_DISABLED=1, satisfying an explicit CI opt-out requirement.
  • Documentation ships inside node_modules/e2e/docs, allowing coding agents or offline environments to read it locally.

This fits a privacy-constrained TypeScript CI setup, but the README gives no guarantees about local-model APIs, resource requirements, latency, or decision quality.

  • README introduction: Bring your own subscription, API key, or local model.
  • Telemetry: CLI sends anonymous usage data ... but no test content, app content, or credentials.
  • Telemetry: Opt out with `npx e2e telemetry disable` or `E2E_TELEMETRY_DISABLED=1`.
  • Documentation: The `e2e` package ships every page in `node_modules/e2e/docs`.
E2E_TELEMETRY_DISABLED=1
Not stated in the README:The README does not list supported local model names, inference protocols, hardware requirements, or offline installation steps.;The README does not say whether disabling telemetry affects the CLI, reporters, or model execution.
Yes I write TypeScript tests for Vite or Next.js and use a coding agent to generate and repair e2e tests in an offline environment. Does the project ship enough documentation inside the npm package?
For: A TypeScript developer using a coding agent to maintain tests and wanting the agent to read e2e documentation offline from node_modules

Yes, it is suitable: the README explicitly says the e2e npm package includes the documentation, so a coding agent can read it from the local dependency; the model and agent workflow still need to be configured separately.

  • The Documentation section says the e2e package ships every page under node_modules/e2e/docs, so online access is not required.
  • Quick start provides npx e2e init, which asks for a web or mobile engine and a model provider, then generates configuration and an example test—useful initial context for an agent.
  • The examples include standalone Vite, Next.js, Expo, and SwiftUI projects that can serve as stack-specific references.
  • The project is primarily TypeScript, and the e2e package provides the SDK, runner, and CLI, matching your TypeScript maintenance workflow.

It is suitable for local documentation retrieval and code generation, but the README does not establish that an agent can automatically judge whether business assertions are correct.

  • Documentation: The `e2e` package ships every page, so coding agents can read them offline in `node_modules/e2e/docs`.
  • Quick start: `npx e2e init` asks for an engine, web or mobile, and a model provider, then writes a config and an example test.
  • Quick start: examples include Vite, Next.js, Expo, and SwiftUI.
  • Packages: `e2e` is the SDK, runner, and CLI; project data lists TypeScript as the main language.
npx e2e init
Not stated in the README:The README does not say whether the docs cover every installed-version API or how documentation/package version mismatches are handled.;The README does not specify the index format, permissions, or recommended prompts for a coding agent reading the docs.
It depends I maintain a Next.js app and need to verify business flows such as upgrading a workspace plan across Chromium, Firefox, and WebKit while retaining explicit locators and assertions. Is e2e suitable for replacing part of my Playwright test suite?
For: A frontend test engineer maintaining a Next.js web app who must cover business flows on Chromium, Firefox, and WebKit

It depends: e2e is suitable for complex business intent, but it should not replace deterministic Playwright tests wholesale.

  • Its web engine uses Playwright and supports Chromium, Firefox, and WebKit, matching your browser matrix.
  • A single test can mix agent.act, agent.assert, locators, and expect; the README demonstrates this with a workspace-plan upgrade flow.
  • Agent actions verified by a later assertion are recorded and replayed when the app has not changed, reducing later model calls.
  • The project is still in active development before 1.0, and APIs and configuration may change between minor releases, so stable Playwright coverage should not be migrated all at once.

A sensible boundary is to use the agent for semantic navigation while keeping explicit assertions for critical state, permissions, and amounts.

  • Packages: Browser engine: Chromium, Firefox, and WebKit through Playwright.
  • README example: `await agent.act('upgrade the workspace to the Pro plan')` and `expect(screen.getByRole('status'))`.
  • README quote: An agent step that a later assertion verifies records its actions, and the next run replays them with no model calls until the app changes.
  • Status: e2e is in active development on the way to 1.0. APIs and config can still change between minor releases.
npx e2e init
Not stated in the README:The README does not describe behavioral differences or coverage maturity for agents across Chromium, Firefox, and WebKit.;The README does not explain where replay records are stored, how invalidation is detected, or how teams share them.
It depends I have an Expo app and must verify login, navigation, and form flows on both iOS simulators and Android emulators. Can e2e's mobile engine let me reuse a test style similar to the web?
For: A mobile engineer maintaining an Expo app who needs to test both iOS simulators and Android emulators

It depends: e2e can unify the mobile test entry point and natural-language expression, but the README does not justify assuming that iOS and Android interactions are fully reusable.

  • @e2e-dev/mobile connects to iOS simulators and Android emulators through agent-device, matching your target environments.
  • npx e2e init asks whether to use the web or mobile engine and which model provider to use, then generates configuration and an example test.
  • The README examples explicitly include a standalone Expo project with a passing suite, showing that Expo is a demonstrated stack.
  • Mobile testing still depends on simulators, app build artifacts, permissions, and initial state; the README does not say that init provisions these automatically.

It is therefore suitable for a shared business-flow style, while platform-specific permissions, navigation, and controls still need separate validation.

  • Packages: `@e2e-dev/mobile`: iOS and Android engine: simulators and emulators through agent-device.
  • Quick start: `init` asks for an engine, web or mobile, and a model provider.
  • Quick start: examples include Vite, Next.js, Expo, and SwiftUI, each a standalone project with a passing suite.
  • Project insight common_pitfalls: mobile testing requires the correct simulators, app build artifacts, permissions, and initial state.
npx e2e init
Not stated in the README:The README does not specify the Expo version, iOS/Android system versions, or required build artifact formats.;The README does not specify which APIs or locators can be reused directly between iOS and Android.
It depends I maintain a SwiftUI app, and my team wants iOS mobile regression tests on every pull request with results posted back to the PR. Does e2e already provide enough execution and reporting components?
For: An iOS engineer maintaining a SwiftUI app whose team needs mobile regression runs on pull requests

It depends: e2e provides a SwiftUI example, an iOS simulator engine, and a GitHub PR reporter, but the README does not prove that a complete PR mobile execution environment is configured automatically.

  • The examples include a standalone SwiftUI project with a passing suite, which provides a useful integration baseline.
  • @e2e-dev/mobile supports iOS simulators, while @e2e-dev/eas provides hosted iOS simulators and Android emulators.
  • @e2e-dev/github posts results as a pull request comment, matching your feedback channel.
  • The project has only five releases, the latest is [email protected], and it remains in active development before 1.0; CI compatibility and upgrades therefore need careful verification.

If the team already has a launchable SwiftUI build and simulator pipeline, the components can be combined. If you expect one command to provision all CI infrastructure, the available evidence is insufficient.

  • Quick start: examples include SwiftUI, each a standalone project with a passing suite.
  • Packages: `@e2e-dev/mobile` provides the iOS and Android engine; `@e2e-dev/eas` provides hosted iOS simulators and Android emulators.
  • Packages: `@e2e-dev/github` is a reporter that posts results as a pull request comment.
  • Project data: release_count is 5 and latest_release is `[email protected]`; Status says it is still in active development before 1.0.
npx e2e init
Not stated in the README:The README does not provide the complete GitHub Actions configuration, required macOS runner conditions, or SwiftUI build commands.;The README does not say whether the PR reporter supports failure screenshots, videos, or agent action logs.

✨ Highlights

  • agent.act drives application actions with natural language
  • Agent steps are recorded and replayed on later runs
  • Playwright covers Chromium, Firefox, and WebKit
  • Supports iOS simulators and Android emulators
  • [email protected] is still in active development before 1.0

🔧 Engineering

  • npx e2e init generates engine, model configuration, and an example test
  • agent.assert verifies the result of natural-language steps
  • The e2e SDK provides both a runner and CLI
  • examples provides projects for Vite, Next.js, Expo, and SwiftUI

⚠️ Risks

  • The README states that APIs and configuration may change before 1.0
  • Agent steps require your own subscription, API key, or local model
  • The CLI sends anonymous telemetry about commands, engines, and failure locations
  • Model calls occur for agent steps and are avoided on later replays

👥 For who?

  • Web testing teams using Vite or Next.js
  • Mobile teams using Expo or SwiftUI
  • Teams needing Chromium, Firefox, or WebKit coverage
  • TypeScript teams wanting to describe business flows in natural language