Commit graph

13 commits

Author SHA1 Message Date
Travis Ralston
988cd335db
Set Synapse log level to DEBUG for cypress tests temporarily (#9173)
Until we narrow down the remaining flake cases this will give us a chance at identifying real-world flake issues.

`DEBUG` is the noisiest level.
2022-08-11 06:19:51 +00:00
kegsay
147ec49ff5
cypress: log stdout/stderr when docker exec fails (#9154)
Otherwise you cannot debug anything with errors like:
```
> Command failed: docker exec 134c9a0afd7dadd0b82ce69b4d72d3d6d8ca1b211540d4390a88357b68fa03b9 pg_isready -U postgres
```
Now we include the stdout/err prior to logging this.
2022-08-09 15:22:32 +01:00
Michael Telatynski
d5db131eef
Spike AXE A11Y testing in Cypress (#9111)
* Spike AXE A11Y testing in Cypress

* Fix NewRoomIntro breaking html/aria list rules

* Fix HeaderButtons breaking aria role semantics rules

* missing type

* Switch left panel from aside to nav and include space panel

* Give the page a main heading of the room name when viewing a room

* Use header landmark on RoomHeader

* Improve aria attributes on composer when autocomplete is closed

* Fix aria-owns on RoomHeader

* Give Spinner an aria role

* Give server picker help button an aria label

* Improve auth aria attributes and semantics

* Improve heading semantics in use case selection screen

* Fix autocomplete attribute to be valid

* Fix heading semantics on login page

* Improve Cypress axe testing

* Add axe tests

* Stop synapse after the timeline tests

* Await spinners to fade before percy snapshotting timeline tests

* Improve naming of plugin

* Update snapshots

* Fix accidental heading change

* Fix double synapse stoppage

* Fix Cypress timeline avatar assertions to be DPI agnostic

* Fix aria attributes on date separators

* delint

* Update snapshots

* Revert style change

* Skip redundant call
2022-08-01 08:31:14 +01:00
Michael Telatynski
ca1d9729fd
Extract dockery bits from Cypress' synapsedocker plugin (#9103)
* Extract dockery bits from Cypress' synapsedocker plugin

* Update cypress/plugins/docker/index.ts

Co-authored-by: David Baker <dbkr@users.noreply.github.com>

Co-authored-by: David Baker <dbkr@users.noreply.github.com>
2022-07-27 15:22:14 +00:00
Michael Weimann
7e47749ce2
Migrate crypto test to cypress (#8833) 2022-06-30 10:59:25 +02:00
Travis Ralston
4171c008a4
Convert sticker end-to-end tests to Cypress (#8807)
* Convert sticker end-to-end tests to Cypress

Reference materials:
* https://www.cypress.io/blog/2020/02/12/working-with-iframes-in-cypress/
* https://github.com/cypress-io/cypress/issues/136
* https://docs.cypress.io/api/commands/origin#Other-limitations

Ideally we'd be able to use `cy.origin()` to jump into the iframe, but it's explicitly not supported. Instead we disable web security as instructed by cypress because it's our only reasonable option here. Thankfully, disabling web security doesn't appear to remove the crypto libraries from the browser so we can still function in that respect.

Rationale for why we can't just serve the sticker picker off the app domain is included in the code.

* Appease the linter

* More linter appeasement
2022-06-10 09:24:15 -06:00
Michael Telatynski
9c44e6c1de
Cypress: only pass -u arg to docker on *nix systems (#8773) 2022-06-07 08:11:25 +00:00
Michael Telatynski
f3f14afbbf
Move spaces tests from Puppeteer to Cypress (#8645)
* Move spaces tests from Puppeteer to Cypress

* Add missing fixture

* Tweak synapsedocker to not double error on a docker failure

* Fix space hierarchy loading race condition

Fixes https://github.com/matrix-org/element-web-rageshakes/issues/10345

* Fix race condition when creating public space with url update code

* Try Electron once more due to perms issues around clipboard

* Try set browser permissions properly

* Try to enable clipboard another way

* Try electron again

* Try electron again again

* Switch to built-in cypress feature for file uploads

* Mock clipboard instead

* TMPDIR ftw?

* uid:gid pls

* Clipboard tests can now run on any browser due to mocking

* Test Enter as well as button for space creation

* Make the test actually work

* Update cypress/support/util.ts

Co-authored-by: Eric Eastwood <erice@element.io>

Co-authored-by: Eric Eastwood <erice@element.io>
2022-05-26 10:19:00 +01:00
J. Ryan Stinnett
c122c5cd3b
Add basic performance testing via Cypress (#8586) 2022-05-17 15:38:45 +01:00
Michael Telatynski
ad4d3f9a88
Move threads e2e tests over to cypress (#8501)
* Add non-consent (default) Synapse template

* Add consent test

* Add create room test

* Stash work

* Initial threads tests

* fix

* Delete old threads e2e tests, plan new ones

* Fix typed s'more

* Try something else

* specify d.ts

* Fix types once and for all?

* Fix the consent tests

* Iterate threads test harness

* Fix dispatcher types

* Iterate threads test

* fix typing

* Alternative import attempt

* let it break let it break let it break

* Tweak types

* Stash

* delint and update docs

* null-guard scrollIntoView

* Iterate threads test

* Apply suggestions from code review
2022-05-10 17:09:31 +00:00
Michael Telatynski
77a437f30a
Create more cypress tests and utilities (#8494) 2022-05-04 14:11:33 +00:00
Michael Telatynski
c616bd7a62
Attempt to fix the cypress flake (#8492) 2022-05-04 11:07:37 +00:00
David Baker
b8013fc52a
Add a Cypress Test 🌲 (#8295)
* A first, maybe working cypress test

Plus cypress plugins to manage synapses in docker containers

* Fix yaml

* This file is important

* try & find where it's put the artifact

* Download artifact to a directory

* pics or it didn't happen

* Add conditional, otherwise no artifacts on failure...

* Try increasing timeout

also actually give the test a name

* Try in chrome

* Get docker logs to see why it's failing

also document the chrome setting

* Try changing mode on homeserver.yaml

* debug

* More debugging

* more file permissions debugging

* ARGH

* more debug

* sigh

* Eugh, that's not how arguments work

* Add the option to really allow open registration

and remove debug logging / comment fixes

* failure to yaml

* Upload docker logs as artifacts

and temporarily remove contional to test

* Put the conditional back

* Upgrade types in end to end tests

to be compatible with fs-extra types

* Try reducing timeout a bit

also make password more... sensible

* Hex is not octal

* Remove file mode

Seems to be unnecessary since the signing key is perfectly fine

* Give the log files extensions

* Rename workflow file now it also does tests

* Add cypress scripts

* copyright headers

* Use ? operator

Co-authored-by: Travis Ralston <travisr@matrix.org>

* Use develop synapse image

* Tidy up any remaining synapses after each spec run

Also:
 * Move the synapseStart / synapseStop functions out to the top level
   so they can be reused
 * Add a tsconfig file
 * Give the containers names

* Don't upload video on test pass

We don't upload it anyway so tell cypress not to so it can not
bother encoding them

* Enable linting on cypress files

and fix existing lint errors

* Type check cypress files

and make it pass the type checks, specifically:
 * Upgrade sinon fake timers to a version that has the right types
 * Set module resolution
 * Type check cypress files separately

* Rename workflow file again

Probably better to just call it an element web build

* Don't plus + characters in container name

* Fix yaml

* Stream logs to file

* Add note to end to end tester to sya what's been ported

* Put docker rm in finally block

Co-authored-by: Travis Ralston <travisr@matrix.org>
2022-04-14 10:41:58 +01:00