Performance tests (#1301)

* Add signposts to performance tests.

- Update flow to include support for the migration screen.

* If the welcome screen shows, click on the button.

* Ensure a clean simulator each run.

* Add accessibility identifier for migration screen if required.

* Handle walking into the room and back out again.

* use iphone 14 pro to match what's used in xcode.

* Remove ApplicationTests as duplicated in LoginTests.

We measure app startup time in LoginTests as part of the flow - we may as well avoid spending 60s doing only that measurement in ApplicationTests

* Sleep 10s, the ui is otherwise showing up in random order.

* Revert "Remove ApplicationTests as duplicated in LoginTests."

This reverts commit 8670710315bcd0d6c3c3046f534b32b4c728b837.

* Update script to parse out correct values from results file.

* Allow cancellation of password prompt in any order.

* Remove test timeout, performance tests will always take a while.

* Adjust parsing further

* Remove ApplicationTests.

* Move to a more elegant way to wait for something to disappear.

* Linting.

* Fix unit tests.

---------

Co-authored-by: Doug <douglase@element.io>
Co-authored-by: Doug <6060466+pixlwave@users.noreply.github.com>
This commit is contained in:
Michael Kaye
2023-07-11 10:32:24 +01:00
committed by GitHub
parent 28fa1140b7
commit 5e97a4a0fa
23 changed files with 207 additions and 69 deletions
+5 -4
View File
@@ -143,15 +143,16 @@ end
lane :integration_tests do
create_simulator_if_necessary(
name: "iPhone 13 Pro",
type: "com.apple.CoreSimulator.SimDeviceType.iPhone-13-Pro"
name: "iPhone 14 Pro",
type: "com.apple.CoreSimulator.SimDeviceType.iPhone-14-Pro"
)
run_tests(
scheme: "IntegrationTests",
devices: ["iPhone 13 Pro"],
devices: ["iPhone 14 Pro"],
ensure_devices_found: true,
result_bundle: true,
reset_simulator: true,
include_simulator_logs: true
)
@@ -391,7 +392,7 @@ private_lane :create_simulator_if_necessary do |options|
UI.user_error!("Invalid simulator type") unless !simulator_type.to_s.empty?
# Use use a `(` here to avoid matching `iPhone 13 Pro` on `iPhone 13 Pro Max` for example
# Use use a `(` here to avoid matching `iPhone 14 Pro` on `iPhone 14 Pro Max` for example
begin sh("xcrun simctl list devices | grep '#{simulator_name} ('")
UI.success "Simulator already exists"
rescue