Disable Percy (#11852)

* Disable Percy

While visual regression testing is without doubt an essential part of
quality assurance, the way we're currently using Percy brings little
to no value while causing mainenance overhead. Therefore, we're disabling it
until we have figure out https://github.com/vector-im/wat-internal/issues/36
and https://github.com/vector-im/wat-internal/issues/56.

* Also disable nightly build

* Comment out Percy step instead of fiddling with the output value

* Appease the frigging linter

.github/workflows/cypress.yaml

@@ -78,16 +78,19 @@ jobs:
                       core.setOutput("author", response.data.author.name);
                       core.setOutput("email", response.data.author.email);
 
-            # Only run Percy when it is demanded or we are running the daily build
+            # Percy is disabled while we're figuring out https://github.com/vector-im/wat-internal/issues/36
-            - name: Enable Percy
+            # and https://github.com/vector-im/wat-internal/issues/56. We're hoping to turn it back on or switch
-              id: percy
+            # to an alternative in the future.
-              if: |
+            # # Only run Percy when it is demanded or we are running the daily build
-                  github.event.workflow_run.event == 'schedule' ||
+            # - name: Enable Percy
-                  (
+            #   id: percy
-                    github.event.workflow_run.event == 'merge_group' &&
+            #   if: |
-                    contains(fromJSON(steps.prdetails.outputs.data).labels.*.name, 'X-Needs-Percy')
+            #       github.event.workflow_run.event == 'schedule' ||
-                  )
+            #       (
-              run: echo "value=1" >> $GITHUB_OUTPUT
+            #         github.event.workflow_run.event == 'merge_group' &&
+            #         contains(fromJSON(steps.prdetails.outputs.data).labels.*.name, 'X-Needs-Percy')
+            #       )
+            #   run: echo "value=1" >> $GITHUB_OUTPUT
 
             - name: Generate unique ID 💎
               id: uuid

.github/workflows/element-web.yaml

@@ -3,8 +3,12 @@
 # as an artifact and run integration tests.
 name: Element Web - Build
 on:
-    schedule:
+    # We only need the nightly run for Percy which is disabled while we're
-        - cron: "17 4 * * 1-5" # every weekday at 04:17 UTC
+    # figuring out https://github.com/vector-im/wat-internal/issues/36 and
+    # https://github.com/vector-im/wat-internal/issues/56. We're hoping to
+    # turn it back on or switch to an alternative in the future.
+    # schedule:
+    #     - cron: "17 4 * * 1-5" # every weekday at 04:17 UTC
     pull_request: {}
     merge_group:
         types: [checks_requested]

README.md

@@ -2,7 +2,14 @@
 ![Tests](https://github.com/matrix-org/matrix-react-sdk/actions/workflows/tests.yml/badge.svg)
 ![Static Analysis](https://github.com/matrix-org/matrix-react-sdk/actions/workflows/static_analysis.yaml/badge.svg)
 [![matrix-react-sdk](https://img.shields.io/endpoint?url=https://dashboard.cypress.io/badge/simple/ppvnzg/develop&style=flat&logo=cypress)](https://dashboard.cypress.io/projects/ppvnzg/runs)
+
+<!--
+Percy is disabled while we're figuring out https://github.com/vector-im/wat-internal/issues/36
+and https://github.com/vector-im/wat-internal/issues/56. We're hoping to turn it back on or switch
+to an alternative in the future.
 [![This project is using Percy.io for visual regression testing.](https://percy.io/static/images/percy-badge.svg)](https://percy.io/dfde73bd/matrix-react-sdk)
+-->
+
 [![Localazy](https://img.shields.io/endpoint?url=https%3A%2F%2Fconnect.localazy.com%2Fstatus%2Felement-web%2Fdata%3Fcontent%3Dall%26title%3Dlocalazy%26logo%3Dtrue)](https://localazy.com/p/element-web)
 [![Quality Gate Status](https://sonarcloud.io/api/project_badges/measure?project=matrix-react-sdk&metric=alert_status)](https://sonarcloud.io/summary/new_code?id=matrix-react-sdk)
 [![Coverage](https://sonarcloud.io/api/project_badges/measure?project=matrix-react-sdk&metric=coverage)](https://sonarcloud.io/summary/new_code?id=matrix-react-sdk)

docs/cypress.md

@@ -238,6 +238,10 @@ should generally try to adhere to them.
 
 ## Screenshot testing with Percy
 
+**⚠️ Percy is disabled while we're figuring out https://github.com/vector-im/wat-internal/issues/36**
+**and https://github.com/vector-im/wat-internal/issues/56. We're hoping to turn it back on or switch**
+**to an alternative in the future.**
+
 We also support visual testing via [Percy](https://percy.io). Within many of our
 Cypress tests you can see lines calling `cy.percySnapshot()`. This creates a
 screenshot and uses Percy to check whether it has changed from the last time