How to author autotest steps without false failures ​

Recommendation for authors of Playwright steps in ComplexQA: how to set import, COMMON, URLs, and Target so a run fails only when the screen truly disagrees with the step.

Why ​

The worker runs steps literally. A relative goto always uses the autotest run starting domain, not the current tab. waitForURL and toHaveURL parse Value differently. JSON import accepts a narrow field set. The report then shows FAILED on a step while the right screen is already open, or toHaveURL stays green on a 404 page.

How to work ​

1. Put repeated sign-in in an autotest of kind COMMON. The screen under test is a separate CASE. On the CASE card list the predecessor and bind a test account. Kind COMMON has no account: TEST_ACCOUNT steps read that CASE account. Flow: When to use COMMON and CASE.
2. After JSON import, change the kind to COMMON yourself. Import does not accept pre_autotest_case_ids or kind. Do not put COMMON in the run queue.
3. Store screens on the same host as the starting domain in the object repository as a relative path (/web/ru/user/auth/login). Store screens on a team subdomain as a full https://{slug}.host/… URL. A verified root covers its subdomains. On a goto step pick the catalog page; leave Value empty.
4. For waitForURL use a Playwright glob (**/web/ru). For toHaveURL use /pattern/flags (the worker compiles a regular expression) or a glob without a trailing ** glued to the last path segment.
5. Target must resolve to one node. If ROLE with one name matches two headings, switch kind to CSS.
6. Assert the screen that the previous step actually opened, not a screen from an earlier session. A screenshot is a run option, not an action_name.

Action how-tos: goto, waitForURL, toHaveURL, toBeVisible.

What to avoid ​

Each subsection is one false failure: what the report shows, why, what to put on the step.

Import rejects value_source_type ​

In the import review. Unknown field value_source_type / value_source_field. Create keeps only valid array items.

Why. The autotest import contract allows on a case autotest_case_title, autotest_case_description, autotest_case_status, autotest_steps. On a step: sort_order, action_name, target, value_text, timeout_ms. Card fields (value_source_type, value_source_field, object_repository_page_id, test_account_id, pre_autotest_case_ids) are not in import JSON.

What to do. Import fill with value_text (for syntax). On the step card set Value source TEST_ACCOUNT and field login / password. Do not put a password in JSON. Bind the test account on the CASE.

Sign-in copied into every CASE ​

In the report. The same goto + fill + click on sign-in in ten results. A sign-in form change breaks every case.

Why. A CASE with no predecessor opens the session itself. Import cannot set kind COMMON or the predecessor list.

What to do. One COMMON for sign-in. After import change the kind to COMMON, status READY. On each CASE, in Pre autotest cases, list that COMMON. Do not attach it to a CASE that captures the sign-in form: the shot would be after sign-in.

waitForURL with /pattern/flags ​

In the report. page.waitForURL: Timeout … waiting for navigation to "/\/web\/ru\/(?!user\/auth)/" while https://host/web/ru is already open.

Why. The worker passes waitForURL Value to Playwright as a string. A /…/ string is not compiled to a regular expression. In a glob, ? is one character; a lookahead does nothing. The log shows navigation; there is no match.

What to do. Use a glob, as in the waitForURL how-to: **/web/ru. Use /pattern/flags on toHaveURL.

waitForURL glob with a slash the URL does not have ​

In the report. Waiting for **/web/ru/ or a pattern with \/web\/ru\/, received https://host/web/ru with no slash after ru.

Why. Playwright compares against the real href. An extra slash in the pattern does not match a path without a trailing slash.

What to do. Write the glob as the address appears in the browser: **/web/ru, not **/web/ru/, when there is no trailing slash.

toHaveURL glob with a trailing ** ​

In the report. expect(page).toHaveURL Expected: "**/web/ru/project/listing**" Received: https://{slug}.host/web/ru/project/listing. The screen is correct; the step is red.

Why. For toHaveURL the worker runs Value through /pattern/flags parsing. Otherwise the string goes to Playwright as a glob. A trailing ** glued to the last segment (listing**) does not match a URL that ends on that segment.

What to do. For toHaveURL set a regular expression:

/\/web\/ru\/project\/listing/

Or a glob without the tail: **/web/ru/project/listing. Do not copy the **…** wrap that worked on waitForURL onto toHaveURL.

toHaveURL checks /web/ru/ while the catalog page opens /web/en/ ​

In the report. Expected: **/web/ru**. Received: https://{slug}.host/web/en/project/listing. The menu is in English.

Why. goto with a repository page substitutes the catalog record path. The URL assert on the CASE must match that path, including the locale segment.

What to do. Keep the locale segment you need for the scenario in the catalog page path (/web/ru/… or /web/en/…). Repeat the same segment in toHaveURL Value.

ROLE heading: strict mode, two nodes ​

In the report. strict mode violation: getByRole('heading', { name: '…' }) resolved to 2 elements - for example an h5 in the app bar and an h2 on the card with the same text.

Why. Playwright strict mode forbids the action when the locator matches more than one node. Kind ROLE plus one name does not tell two headings apart.

What to do. Kind CSS with a selector for one node (for example h2.team-listing__title). If the step only confirms the screen and the next click is unique, drop the extra toBeVisible on a non-unique heading.

Relative goto after COMMON changed host ​

In the report. goto is green; the shot is a 404 not-found page. The address looks like https://webapp.example/web/ru/project/listing without the team subdomain.

Why. A relative goto path joins the run starting domain, not the host where COMMON finished. A click on a team may open {slug}.host; the next goto /web/ru/project/listing returns to the root. That screen does not exist on the root.

What to do. Do not add a second starting domain. Create a catalog page with the full URL https://{slug}.host/web/ru/project/listing and select it on the goto step. A verified root covers the subdomain. Details: object-repository Page and goto.

toHaveURL green on a 404 ​

In the report. toHaveURL for path /web/ru/project/listing is PASSED. The next toBeVisible on a menu item is FAILED. The shot is 404.

Why. A path-only assert does not see the host. The same path on the root and on the team subdomain is a different screen.

What to do. Put the host or subdomain in the toHaveURL pattern (/\/{slug}\..*\/web\/ru\/project\/listing/ or glob **://{slug}.host/web/ru/project/listing). Keep a toBeVisible on unique workspace text: 404 will fail that check.

Asserting a screen that sign-in no longer opens ​

In the report. toBeVisible on h2.team-listing__title («Choose a team») while the shot is the team project listing.

Why. After sign-in on the team subdomain the start page is the workspace (project listing), not the team picker. A step written for the picker looks for a node from another screen.

What to do. Build COMMON from the shot after the sign-in click. If the picker is gone, remove the team click and the picker heading assert. Assert what opened: a project card, the Projects item.

Screenshot as action_name ​

In the report. Import or syntax check rejects action screenshot, or the action is missing from the step catalog.

Why. A screen capture is an autotest run option (Screenshot on failure, Result screenshot), not a step action. Catalog: Autotest step.

What to do. Turn on the option in Autotest run creation. Keep navigation and expects in the steps.

Locale from the html lang attribute ​

In the report. The URL contains /web/ru/; the Playwright log shows <html lang="en">. The author rewrites steps as if the route locale were English.

Why. Path segment /web/ru/ and the lang attribute on the root html are different fields. The URL assert looks at href. Read menu labels from the step shot.

What to do. Keep the locale segment in the catalog page path and in toHaveURL the same. For TEXT, use the label visible on the shot (a menu item or a demo project name), not the html lang value.