tests(e2e): Refactor various tests

Goals:

- speedup
- less flakiness
- best practices and more use
- documentation

config:
- sync ports in Makefile and playwright config
  (otherwise, some tests fail locally because they assert the full URL including the (wrong) port)
- even more generous timeouts
- limit workers to one again (because I finally understand how
  Playwright works)
- allow nested functions to group them together with the related test

all:

- deprecate waitForLoadState('networkidle')
  - it is discouraged as per https://playwright.dev/docs/api/class-page#page-wait-for-load-state
  - I could not find a usage that seems to require it actually (see
    added documentation in README)
  - adding an exception should be made explicitly
  - it does not do what you might expect anyway in most cases
- only log in when necessary

webauthn:

- verify that login is possible after disabling key
- otherwise, the cleanup was not necessary after the previous refactor to create a fresh user each

issue-sidebar / WIP toggle:

- split into smaller chunks
- restore original state first
- add missed assertion to fix race condition (not waiting
  before state was reached)
- explicitly toggle the state to detect mismatch earlier

issue-sidebar / labels:

- restore original state first
- better waiting for background request
This commit is contained in:
Otto Richter 2024-11-12 21:07:09 +01:00
parent a39f726643
commit 40551de313
15 changed files with 230 additions and 117 deletions

View file

@ -1,6 +1,6 @@
import {devices, type PlaywrightTestConfig} from '@playwright/test';
const BASE_URL = process.env.GITEA_URL?.replace?.(/\/$/g, '') || 'http://localhost:3000';
const BASE_URL = process.env.GITEA_URL?.replace?.(/\/$/g, '') || 'http://localhost:3003';
/**
* @see https://playwright.dev/docs/test-configuration
@ -12,7 +12,7 @@ export default {
// you can adjust this value locally to match your machine's power,
// or pass `--workers x` to playwright
workers: process.env.CI ? 1 : 2,
workers: 1,
/* Maximum time one test can run for. */
timeout: 30 * 1000,
@ -22,7 +22,7 @@ export default {
* Maximum time expect() should wait for the condition to be met.
* For example in `await expect(locator).toHaveText();`
*/
timeout: 2000,
timeout: 3000,
},
/* Fail the build on CI if you accidentally left test.only in the source code. */
@ -30,6 +30,8 @@ export default {
/* Retry on CI only */
retries: process.env.CI ? 1 : 0,
/* fail fast */
maxFailures: process.env.CI ? 1 : 0,
/* Reporter to use. See https://playwright.dev/docs/test-reporters */
reporter: process.env.CI ? 'list' : [['list'], ['html', {outputFolder: 'tests/e2e/reports/', open: 'never'}]],
@ -41,7 +43,7 @@ export default {
locale: 'en-US',
/* Maximum time each action such as `click()` can take. Defaults to 0 (no limit). */
actionTimeout: 2000,
actionTimeout: 3000,
/* Maximum time allowed for navigation, such as `page.goto()`. */
navigationTimeout: 10 * 1000,
@ -95,8 +97,8 @@ export default {
},
],
/* Folder for test artifacts such as screenshots, videos, traces, etc. */
/* Folder for test artifacts created during test execution such as screenshots, traces, etc. */
outputDir: 'tests/e2e/test-artifacts/',
/* Folder for test artifacts such as screenshots, videos, traces, etc. */
/* Folder for explicit snapshots for visual testing */
snapshotDir: 'tests/e2e/test-snapshots/',
} satisfies PlaywrightTestConfig;