Run Playwright tests against Dendrite & Pinecone periodically (#28888)

* Switch to TestContainers for manging services in Playwright

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Flip fixture dependency order

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Remove mas dep

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Update matrix-authentication-service in Playwright tests

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* delint

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix SMTP port

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Comments

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Strip ansi from playwright logs to make them more readable

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Actually do the update

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Remove access to homeserver.config.baseUrl field in favour of homeserver.baseUrl

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Use sane default_server_config and specify server.invalid in the specific tests which demand it

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix mas run

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* break cycle

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* typo

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* prettier

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Wire up basics of dendriteHomeserver

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Run Playwright tests against Dendrite & Pinecone periodically

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix playwright-image-updates.yaml workflow

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Add `X-Run-All-Tests` label for running all tests

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Ignore failing tests in stale-screenshot-reporter.ts to avoid confusing errors

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Specify Synapse ui_auth.session_timeout only on tests which require it

As Dendrite lacks this configuration option

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* React to MatrixEvent sender/target sentinels being updated for rendering state events

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Remove test code

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* React to sentinel changes in EventListSummary

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Docs

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Avoid reusing user1234

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix stale-screenshot-reporter.ts

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Clean up public rooms between tests on reused homeserver

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Deflake spotlight when homeserver is reused

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Deflake more tests using existing username

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Clean mailhog between tests

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix more flakes

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix missing _request

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix playwright flaky tests

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Wipe mailhog between test runs

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Delint

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* delint

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Deflake more tests

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix flaky tests

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix flaky tests

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix mas config

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix another flaky test

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix playwright flakes due to floating promises

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix flaky playwright tests

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Update services.ts

---------

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
This commit is contained in:
Michael Telatynski
2025-01-14 18:00:30 +00:00
committed by GitHub
parent bb41616d5f
commit dbce48b23d
55 changed files with 259 additions and 95 deletions
@@ -9,8 +9,11 @@ Please see LICENSE files in the repository root for full details.
/* See readme.md for tips on writing these tests. */
import { test } from ".";
import { isDendrite } from "../../plugins/homeserver/dendrite";
test.describe("Read receipts", { tag: "@mergequeue" }, () => {
test.skip(isDendrite, "due to Dendrite bug https://github.com/element-hq/dendrite/issues/2970");
test.describe("editing messages", () => {
test.describe("in threads", () => {
test("An edit of a threaded message makes the room unread", async ({
@@ -9,8 +9,11 @@ Please see LICENSE files in the repository root for full details.
/* See readme.md for tips on writing these tests. */
import { test } from ".";
import { isDendrite } from "../../plugins/homeserver/dendrite";
test.describe("Read receipts", { tag: "@mergequeue" }, () => {
test.skip(isDendrite, "due to Dendrite bug https://github.com/element-hq/dendrite/issues/2970");
test.describe("editing messages", () => {
test.describe("in the main timeline", () => {
test("Editing a message leaves a room read", async ({ roomAlpha: room1, roomBeta: room2, util, msg }) => {
@@ -9,8 +9,11 @@ Please see LICENSE files in the repository root for full details.
/* See readme.md for tips on writing these tests. */
import { test } from ".";
import { isDendrite } from "../../plugins/homeserver/dendrite";
test.describe("Read receipts", { tag: "@mergequeue" }, () => {
test.skip(isDendrite, "due to Dendrite bug https://github.com/element-hq/dendrite/issues/2970");
test.describe("editing messages", () => {
test.describe("thread roots", () => {
test("An edit of a thread root leaves the room read", async ({
@@ -9,8 +9,10 @@ Please see LICENSE files in the repository root for full details.
/* See readme.md for tips on writing these tests. */
import { customEvent, many, test } from ".";
import { isDendrite } from "../../plugins/homeserver/dendrite";
test.describe("Read receipts", { tag: "@mergequeue" }, () => {
test.skip(isDendrite, "due to Dendrite bug https://github.com/element-hq/dendrite/issues/2970");
test.slow();
test.describe("Ignored events", () => {
@@ -9,8 +9,11 @@ Please see LICENSE files in the repository root for full details.
/* See readme.md for tips on writing these tests. */
import { many, test } from ".";
import { isDendrite } from "../../plugins/homeserver/dendrite";
test.describe("Read receipts", { tag: "@mergequeue" }, () => {
test.skip(isDendrite, "due to Dendrite bug https://github.com/element-hq/dendrite/issues/2970");
test.describe("new messages", () => {
test.describe("in threads", () => {
test("Receiving a message makes a room unread", async ({
@@ -9,8 +9,11 @@ Please see LICENSE files in the repository root for full details.
/* See readme.md for tips on writing these tests. */
import { many, test } from ".";
import { isDendrite } from "../../plugins/homeserver/dendrite";
test.describe("Read receipts", { tag: "@mergequeue" }, () => {
test.skip(isDendrite, "due to Dendrite bug https://github.com/element-hq/dendrite/issues/2970");
test.describe("new messages", () => {
test.describe("in the main timeline", () => {
test("Receiving a message makes a room unread", async ({
@@ -9,8 +9,11 @@ Please see LICENSE files in the repository root for full details.
/* See readme.md for tips on writing these tests. */
import { many, test } from ".";
import { isDendrite } from "../../plugins/homeserver/dendrite";
test.describe("Read receipts", { tag: "@mergequeue" }, () => {
test.skip(isDendrite, "due to Dendrite bug https://github.com/element-hq/dendrite/issues/2970");
test.describe("new messages", () => {
test.describe("thread roots", () => {
test("Reading a thread root does not mark the thread as read", async ({
@@ -9,8 +9,11 @@ Please see LICENSE files in the repository root for full details.
/* See readme.md for tips on writing these tests. */
import { test, expect } from ".";
import { isDendrite } from "../../plugins/homeserver/dendrite";
test.describe("Read receipts", { tag: "@mergequeue" }, () => {
test.skip(isDendrite, "due to Dendrite bug https://github.com/element-hq/dendrite/issues/2970");
test.describe("reactions", () => {
test.describe("in threads", () => {
test("A reaction to a threaded message does not make the room unread", async ({
@@ -9,8 +9,11 @@ Please see LICENSE files in the repository root for full details.
/* See readme.md for tips on writing these tests. */
import { test } from ".";
import { isDendrite } from "../../plugins/homeserver/dendrite";
test.describe("Read receipts", { tag: "@mergequeue" }, () => {
test.skip(isDendrite, "due to Dendrite bug https://github.com/element-hq/dendrite/issues/2970");
test.describe("reactions", () => {
test.describe("in the main timeline", () => {
test("Receiving a reaction to a message does not make a room unread", async ({
@@ -9,8 +9,10 @@ Please see LICENSE files in the repository root for full details.
/* See readme.md for tips on writing these tests. */
import { test } from ".";
import { isDendrite } from "../../plugins/homeserver/dendrite";
test.describe("Read receipts", { tag: "@mergequeue" }, () => {
test.skip(isDendrite, "due to Dendrite bug https://github.com/element-hq/dendrite/issues/2970");
test.describe("reactions", () => {
test.describe("thread roots", () => {
test("A reaction to a thread root does not make the room unread", async ({
@@ -12,8 +12,10 @@ import { expect } from "../../element-web-test";
import { ElementAppPage } from "../../pages/ElementAppPage";
import { Bot } from "../../pages/bot";
import { test } from ".";
import { isDendrite } from "../../plugins/homeserver/dendrite";
test.describe("Read receipts", { tag: "@mergequeue" }, () => {
test.skip(isDendrite, "due to Dendrite bug https://github.com/element-hq/dendrite/issues/2970");
test.use({
displayName: "Mae",
botCreateOpts: { displayName: "Other User" },
@@ -9,8 +9,11 @@ Please see LICENSE files in the repository root for full details.
/* See readme.md for tips on writing these tests. */
import { test } from ".";
import { isDendrite } from "../../plugins/homeserver/dendrite";
test.describe("Read receipts", { tag: "@mergequeue" }, () => {
test.skip(isDendrite, "due to Dendrite bug https://github.com/element-hq/dendrite/issues/2970");
test.describe("redactions", () => {
test.describe("in threads", () => {
test("Redacting the threaded message pointed to by my receipt leaves the room read", async ({
@@ -9,8 +9,11 @@ Please see LICENSE files in the repository root for full details.
/* See readme.md for tips on writing these tests. */
import { test } from ".";
import { isDendrite } from "../../plugins/homeserver/dendrite";
test.describe("Read receipts", { tag: "@mergequeue" }, () => {
test.skip(isDendrite, "due to Dendrite bug https://github.com/element-hq/dendrite/issues/2970");
test.describe("redactions", () => {
test.describe("in the main timeline", () => {
test("Redacting the message pointed to by my receipt leaves the room read", async ({
@@ -9,8 +9,11 @@ Please see LICENSE files in the repository root for full details.
/* See readme.md for tips on writing these tests. */
import { test } from ".";
import { isDendrite } from "../../plugins/homeserver/dendrite";
test.describe("Read receipts", { tag: "@mergequeue" }, () => {
test.skip(isDendrite, "due to Dendrite bug https://github.com/element-hq/dendrite/issues/2970");
test.describe("redactions", () => {
test.describe("thread roots", () => {
test("Redacting a thread root after it was read leaves the room read", async ({