Run Playwright tests against Dendrite & Pinecone periodically (#28888)

* Switch to TestContainers for manging services in Playwright

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Flip fixture dependency order

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Remove mas dep

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Update matrix-authentication-service in Playwright tests

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* delint

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix SMTP port

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Comments

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Strip ansi from playwright logs to make them more readable

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Actually do the update

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Remove access to homeserver.config.baseUrl field in favour of homeserver.baseUrl

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Use sane default_server_config and specify server.invalid in the specific tests which demand it

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix mas run

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* break cycle

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* typo

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* prettier

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Wire up basics of dendriteHomeserver

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Run Playwright tests against Dendrite & Pinecone periodically

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix playwright-image-updates.yaml workflow

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Add `X-Run-All-Tests` label for running all tests

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Ignore failing tests in stale-screenshot-reporter.ts to avoid confusing errors

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Specify Synapse ui_auth.session_timeout only on tests which require it

As Dendrite lacks this configuration option

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* React to MatrixEvent sender/target sentinels being updated for rendering state events

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Remove test code

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* React to sentinel changes in EventListSummary

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Docs

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Avoid reusing user1234

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix stale-screenshot-reporter.ts

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Clean up public rooms between tests on reused homeserver

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Deflake spotlight when homeserver is reused

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Deflake more tests using existing username

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Clean mailhog between tests

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix more flakes

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix missing _request

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix playwright flaky tests

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Wipe mailhog between test runs

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Delint

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* delint

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Deflake more tests

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix flaky tests

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix flaky tests

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix mas config

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix another flaky test

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix playwright flakes due to floating promises

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix flaky playwright tests

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Update services.ts

---------

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
This commit is contained in:
Michael Telatynski
2025-01-14 18:00:30 +00:00
committed by GitHub
parent bb41616d5f
commit dbce48b23d
55 changed files with 259 additions and 95 deletions
@@ -10,6 +10,7 @@ import type { Preset, Visibility } from "matrix-js-sdk/src/matrix";
import { test, expect } from "../../element-web-test";
test.describe("Room Directory", () => {
test.skip(({ homeserverType }) => homeserverType === "pinecone", "Pinecone's /publicRooms API takes forever");
test.use({
displayName: "Ray",
botCreateOpts: { displayName: "Paul" },
@@ -32,14 +33,14 @@ test.describe("Room Directory", () => {
await localAddresses.getByRole("textbox").fill("gaming");
await expect(page.getByText("This address is available to use")).toBeVisible();
await localAddresses.getByRole("button", { name: "Add" }).click();
await expect(localAddresses.getByText("#gaming:localhost")).toHaveClass("mx_EditableItem_item");
await expect(localAddresses.getByText(`#gaming:${user.homeServer}`)).toHaveClass("mx_EditableItem_item");
// Publish into the public rooms directory
const publishedAddresses = page.locator(".mx_SettingsFieldset", { hasText: "Published Addresses" });
await expect(publishedAddresses.locator("#canonicalAlias")).toHaveValue("#gaming:localhost");
await expect(publishedAddresses.locator("#canonicalAlias")).toHaveValue(`#gaming:${user.homeServer}`);
const checkbox = publishedAddresses
.locator(".mx_SettingsFlag", {
hasText: "Publish this room to the public in localhost's room directory?",
hasText: `Publish this room to the public in ${user.homeServer}'s room directory?`,
})
.getByRole("switch");
await checkbox.check();
@@ -87,7 +88,7 @@ test.describe("Room Directory", () => {
.getByRole("button", { name: "Join" })
.click();
await expect(page).toHaveURL("/#/room/#test1234:localhost");
await expect(page).toHaveURL(`/#/room/#test1234:${user.homeServer}`);
},
);
});