PerformanceTesting.rst (3564B)
1 ================================ 2 Service Worker Performance Tests 3 ================================ 4 5 Our performance tests are mochitests running in the `mozperftest 6 <https://firefox-source-docs.mozilla.org/testing/perfdocs/mozperftest.html>`_ 7 harness. Tests reside under `dom/serviceworkers/test/performance 8 <https://searchfox.org/mozilla-central/source/dom/serviceworkers/test/performance>`_, 9 itemized in perftest.toml. Beyond the standard mochitest machinery, 10 performance tests define a ``perfMetadata`` variable at the top level, and call: 11 12 ``info("perfMetrics", JSON.stringify(metrics));`` 13 14 to report their results, where *metrics* is a map from testpoint name to scalar 15 numeric value. See the `performance scripts documentation 16 <https://firefox-source-docs.mozilla.org/testing/perfdocs/writing.html#mochitest>`_ 17 for more. 18 19 They can be run via mach perftest, or as normal mochitests via mach test. 20 (Currently we can’t run the full manifest, see `bug 1865852 21 <https://bugzilla.mozilla.org/show_bug.cgi?id=1865852>`_.) 22 23 Adding new tests 24 ================ 25 26 Add files to `perftest.toml 27 <https://searchfox.org/mozilla-central/source/dom/serviceworkers/test/performance/perftest.toml>`_ 28 as usual for mochitests. 29 30 Modify linux.yml, macosx.yml, and windows11.yml under `taskcluster/kinds/perftest 31 <https://searchfox.org/mozilla-central/source/taskcluster/kinds/perftest>`_. 32 Currently, each test needs to be added individually to the run command (`here 33 <https://searchfox.org/mozilla-central/rev/91cc8848427fdbbeb324e6ca56a0d08d32d3c308/taskcluster/ci/perftest/linux.yml#121-149>`_, 34 for example). kind.yml can be ignored–it provides some defaults. 35 36 Add your new test to `perfdocs/config.yml 37 <https://searchfox.org/mozilla-central/source/python/mozperftest/perfdocs/config.yml>`_. 38 39 Modify the generated documentation using: 40 41 ``$ ./mach lint -l perfdocs . --fix --warnings --outgoing`` 42 43 There's currently a `bug <https://bugzilla.mozilla.org/show_bug.cgi?id=1872613>`_ 44 which will likely cause the command to fail. Running it a second time should 45 succeed. 46 47 Profiler Markers 48 ================ 49 50 Profiler markers can be used to collect timing data. Markers of known name can be inspected from the perftest. AUTO_PROFILER_MARKER_UNTYPED must be used, as we need both the start-time and end-time of the marker. For example: 51 52 .. code-block:: cpp 53 54 AUTO_PROFILER_MARKER_UNTYPED("interesting thing #1", DOM, {}; 55 AUTO_PROFILER_MARKER_UNTYPED("interesting thing #2", DOM, {}); 56 57 can be inspected from the perftest: 58 59 .. code-block:: js 60 61 await startProfiler(); 62 interestingThings(); 63 let pdata = await stopProfiler(); 64 let duration_ms = inspectProfile(pdata, [ 65 "interesting thing #1", 66 "interesting thing #2" 67 ]); 68 69 Staging tests in try jobs 70 ========================= 71 72 ``$ ./mach try fuzzy --full`` 73 74 Look for ``‘service-worker`` to find things like: 75 76 | >perftest-windows-service-worker 77 | >perftest-macosx-service-worker 78 | >perftest-linux-service-worker 79 | 80 81 Results 82 ======= 83 84 Results can be found in treeherder on `mozilla-central 85 <https://treeherder.mozilla.org/jobs?repo=mozilla-central&searchStr=perftest>`_ 86 and `autoland 87 <https://treeherder.mozilla.org/jobs?repo=autoland&searchStr=perftest>`_. Look 88 for linux-sw, macosx-sw, and win-sw (`example 89 <https://treeherder.mozilla.org/perfherder/graphs?series=mozilla-central,4967140,1,15&selected=4967140,1814245176>`_). 90 These symbol names are defined in the .yml files under taskcluster/kinds/perftest. 91 92 Contacts 93 ======== 94 | DOM LWS 95 | `Gregory Mierzwinski <https://people.mozilla.org/p/sparky>`_ (Performance Tools)