debugging_firefox_with_pernosco.rst (3432B)
1 Debugging Firefox with Pernosco 2 =============================== 3 4 This page is intended to help Firefox/Gecko developers get started using `Pernosco <https://pernos.co/>`_ 5 to debug Firefox. 6 7 Mozilla employees have a paid subscription to Pernosco, and are encouraged to use it. 8 Contributors can use Pernosco as well. `A free account <https://pernos.co/pricing/>`_ 9 comes with 5 uploads to see if it is suitable for your workflow. 10 11 Prerequisites 12 ------------- 13 14 - You need the ability to produce an :code:`rr` trace. See :doc:`debugging_firefox_with_rr` 15 - Employees, you can also generate traces for Pernosco for some CI job failures via 16 Treeherder. 17 18 Workflow 19 -------- 20 21 At a high level, working with Pernosco consists of recording a trace then uploading 22 the trace for processing. 23 24 Install your Pernosco configuration 25 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 26 27 If you are using your own Pernoso account, follow the instructions 28 `here <https://pernos.co/account>`_ to install your credentials. 29 30 Mozilla employees, see the `Pernosco install and configuration doc 31 <https://docs.google.com/document/d/1cb-mkNyJYMudpS0a-R6zxy0kOMIRUSKrxxL-E9zFGKE/edit>`_ 32 for credentials. 33 34 In general, you will want: 35 36 * Your email in :code:`~/.config/pernosco/user` 37 * Your group in :code:`~/.config/pernosco/group` 38 * Your key in :code:`~/.config/pernosco/user_secret_key` 39 40 Gather a Local Trace 41 ~~~~~~~~~~~~~~~~~~~~ 42 43 In general, to gather a trace follow the instruction for recording :code:`rr` traces 44 in :doc:`debugging_firefox_with_rr`. However, to gather a local rr trace that’s 45 compatible with Pernosco you need to disable some incompatible CPU features (currently 46 only AVX-512) 47 48 On an Intel machine, you can disable this at the command line. To simplify things a 49 shell script :code:`pernosco-record` can be created like so, 50 51 .. code:: bash 52 53 #!/bin/bash 54 55 rr record --disable-avx-512 "$@" 56 57 replacing plain :code:`rr` in command lines (:code:`mach --debugger pernosco-record`) 58 59 On AMD Zen Machines, you can disable AVX-512 and 5 level page tables at boot. 60 61 On a Debian derived distro this involves editing :code:`/etc/default/grub` and 62 adding :code:`clearcpuid=304,no5lvl` to :code:`GRUB_CMDLINE_LINUX_DEFAULT`, then 63 running :code:`sudo update-grub` after. 64 65 Upload the trace 66 ~~~~~~~~~~~~~~~~ 67 68 Find the trace you’re interested in :code:`~/.local/share/rr/<trace>`, and call :code:`pernosco-submit` 69 70 .. code:: bash 71 72 pernosco-submit upload ~/.local/share/rr/<trace> PATH_TO_CENTRAL --title "Name Of Trace" 73 74 where :code:`PATH TO CENTRAL` is the path to the source of your build, checked out 75 to the revision matching the build. 76 77 The last trace recorded is always symlinked, so you can use :code:`~/.local/share/rr/latest-trace` too. 78 79 Wait for email 80 ~~~~~~~~~~~~~~ 81 82 After the trace is processed, you'll get an email with a link to your trace. Good 83 hunting! 84 85 Treeherder Workflow 86 ------------------- 87 88 You can also try to get traces generated via the Treeherder interface. On a failing 89 test that is eligible, in "Artifacts and Debugging Tools", you'll see a link to 90 91 .. image:: ../img/record-this-failure.png 92 :width: 35 % 93 94 This link brings you to the self-service reproducer: 95 96 .. image:: ../img/self-service.png 97 :width: 60 % 98 99 Select a test and a count, and whether or not chaos-mode should be used, then click 100 reproduce. At this point there is a **long** wait; the page will notify you when 101 reproduction has started with an :code:`alert` dialog.