Test Runs

A test run is a named, rerunnable collection of test cases. Where Test Execution runs one script at a time, a test run groups many test cases together, executes them as a set, and keeps the result so you can compare one attempt against the next.

Beta feature

Test Runs is a Beta feature and may not be visible on every account. Contact your workspace admin or [email protected] if you'd like access.

Runs, Sessions and Cases

Three words that mean different things. Getting them straight makes the rest of the page — and the UI — much easier to read.

Term	What it is
Run	The container you create and name. It holds a set of test cases, an engine, a language and an environment. It is rerunnable — the same run can be started many times.
Session	One press of Start or Rerun. Each session records its own pass/fail counts and progress.
Case	One test case's membership in the run, plus the status it ended up with in the latest session.

A run never fails. It goes NOT_STARTED → RUNNING → COMPLETED even when every case inside it failed — the run describes whether the work finished, not whether the application passed. Failure is reported per case.

Creating a Test Run

1. Open your project and go to the Test Runs tab
2. Click Create Run
3. Give the run a name, and optionally a description
4. Choose the engine — Selenium or Playwright
5. The language is Java (Python is shown in the dialog but is not available yet)
6. Optionally set an environment label to record where the run was pointed
7. Adjust the execution preferences (window size, timeouts, video recording — the same settings described in Test Execution)
8. Pick the test cases to include
9. Click Create

info

A run is single-engine. Every case in it executes the same way, so a project that has both Selenium and Playwright scripts needs a run for each. The engine is chosen at creation; cases whose script was built for the other engine are skipped as Blocked.

tip

A run stores its own copy of the execution settings rather than pointing at a saved preference. Editing a saved preference later does not change a run that was already created — which is what makes an old run reproducible.

Starting and Rerunning

• Start launches the first session.
• Rerun launches another session over the same membership. Previous sessions are kept.

Progress updates live while the run is going — you do not need to refresh.

Run Status

Status	Meaning
Not Started	Created, never run
Running	A session is in flight
Completed	The session finished — regardless of how many cases passed

Session status is separate and mostly of interest when something goes wrong at the infrastructure level:

Session status	Meaning
Queued	Waiting for an execution slot
Running	Executing
Completed	Finished normally
Failed	The session could not run at all. This does not mean "some cases failed" — that still completes.

Case Results

Each case in a run ends up with one of these:

Result	Meaning
Not Started	Not yet reached
Queued	Waiting to execute
Running	Executing now
Passed	The test passed
Failed	The application failed the test's assertions
Blocked	The case could not be executed at all — see below
Flaky	Passed in the end, but failed at least once during the run

Flaky is worth paying attention to. A case that only passes on retry is usually a timing problem in the test or genuine instability in the application, and it will bite you later if you treat it as a pass.

Why a case is Blocked

Blocked cases are dimmed and cannot be opened. There are three reasons, and each has a different fix:

Reason	What to do
No script	The test case has no generated test script. Generate one — see Test Scripts.
Script not ready	The script is still generating. Wait for it to finish, then rerun.
Engine mismatch	The script was built for the other engine. Either regenerate it for this run's engine, or move the case to a run that matches.

info

Progress is measured over every member case, blocked ones included. A run of four cases with two blocked tops out at 50% — completion is signalled by the run's status, not by progress reaching 100.

Editing a Run

Open the run and use Edit to change its name, description, environment, execution preferences or membership.

warning

Changing the case selection replaces the run's membership rather than adding to it. The edit dialog is pre-filled with the current cases so this is safe when you use the UI — just don't clear the picker expecting the existing cases to be kept.

The Run Detail View

A run opens on three tabs:

Tab	What it shows
Cases	Every case in the run with its current result, and its blocked reason where relevant
Details	The run's properties — engine, language, environment, execution settings, who created it and who last edited it
Analytics	Pass/fail breakdown for the run, with KPI tiles and a chart whose legend filters the view

Comments are available on a run, so you can discuss a result with your team in place. See Collaboration.

Filtering and Finding Runs

The Test Runs list supports search and a filter drawer, plus an All / Active / Completed toggle. Active means runs that are currently executing.

KPI cards above the list show total runs, active runs, completed runs, and average progress across the project.

Troubleshooting

All my cases show as Blocked

• The most common cause is engine mismatch. Check the run's engine on the Details tab against the engine the scripts were generated with.

The run completed but nothing passed

• That is a normal completed run reporting real failures. Open a failed case to see its logs and screenshot, and create a bug from it — see Bugs.

Progress stopped below 100%

• Check for blocked cases. They count toward the total but never settle, so a run with blocked cases cannot reach 100%.

A case keeps coming back Flaky

• The test passes only on retry. Look for timing assumptions in the script, or genuine instability in the application under test.

Next Steps

• Test Execution — Run a single script and watch the browser live
• Test Scripts — Generate the scripts a run needs
• Bugs — Turn a failed case into a tracked bug
• Collaboration — Comment on a run with your team