README (3999B)
1 Performing a (re)bootstrapping of symbols scraping process 2 ========================================================== 3 4 Whenever for any reason the symbol scraping process might have been faulty long 5 enough, we can end up (currently) in a situation where the recorded status of 6 `SHA256SUMS.zip` on the TaskCluster index is inconsistent with what we really 7 processed. 8 9 This document aims at explaining what needs to be done and where to recover 10 from that state (this is based on the experience from bug 1893156). 11 12 First, you need to identify since how long the problem has been present. As of 13 now there is no really better tooling than processing manually the cron tasks 14 logs and see when it started to fail. 15 16 Once you have identified a date, the next step is to work on the bootstrapping 17 content. As visible in 18 https://searchfox.org/mozilla-central/rev/f6e3b81aac49e602f06c204f9278da30993cdc8a/taskcluster/docker/system-symbols-linux-scraper/run.sh#62, 19 the first source of truth is the gh-pages branch of the symbol-scrapers github 20 repository: https://github.com/mozilla/symbol-scrapers/tree/gh-pages. This 21 source of truth is evaluated ONLY if the TaskCluster index is NOT present. The 22 route is being computed from the running task's definition: 23 https://searchfox.org/mozilla-central/rev/f6e3b81aac49e602f06c204f9278da30993cdc8a/taskcluster/docker/system-symbols-linux-scraper/run.sh#14 24 from which we ONLY consider the `latest` alias. 25 26 As of today the index is for example for debian: 27 index.gecko.v2.mozilla-central.latest.system-symbols.debian and thus one can 28 explore the content at 29 https://firefox-ci-tc.services.mozilla.com/tasks/index/gecko.v2.mozilla-central.latest.system-symbols.debian, 30 other means of browsing including pushdate allows to find e.g., 31 https://firefox-ci-tc.services.mozilla.com/tasks/index/gecko.v2.mozilla-central.pushdate.2024.04.20.20240420094034.system-symbols/debian 32 from which we can get a link to the sums file: 33 https://firefox-ci-tc.services.mozilla.com/api/index/v1/task/gecko.v2.mozilla-central.pushdate.2024.04.20.20240420094034.system-symbols.debian/artifacts/public%2Fbuild%2FSHA256SUMS.zip 34 35 Once you have identified WHEN the problem arose, you can take the above URL 36 (adapting with the correct date) and adapting to the various distributions. 37 38 Make sure you have an uptodate git clone of the mozilla/symbol-scrapers 39 repository, checkout a new branch out of the gh-pages tree, and you can proceed 40 to the data extraction following (example with a different date): 41 for distro in alpine archlinux debian fedora firefox-flatpak firefox-snap gnome-sdk-snap mint opensuse ubuntu; do 42 wget https://firefox-ci-tc.services.mozilla.com/api/index/v1/task/gecko.v2.mozilla-central.pushdate.2024.02.07.latest.system-symbols.$distro/artifacts/public%2Fbuild%2FSHA256SUMS.zip -O $distro/SHA256SUMS.zip; 43 done; 44 mv archlinux/SHA256SUMS.zip arch/ 45 46 Please note that there's a slight difference in naming, archlinux vs arch. 47 Please note other distros might have been added since so you need to adapt. 48 49 Send a pull request once this is OK, make it reviewed or merge it. 50 51 As of now, the content of the boostrapping process is in the state we want, but 52 if you run a symbols scraping task, it will still pull data from the 53 TaskCluster index. This time, you need to use the index that refers to latest 54 and NOT the pushdate or another one, so the index used in the example SHOULD be 55 good. You just have to run deleteTask (with the appropriate credentials if you 56 have them, or ask releng for help in #firefox-ci): 57 for distro in alpine archlinux debian fedora firefox-flatpak firefox-snap gnome-sdk-snap mint opensuse ubuntu; do 58 taskcluster api index deleteTask gecko.v2.mozilla-central.latest.system-symbols.$distro 59 done; 60 61 From there, HTTP queries to (for the debian example) 62 https://firefox-ci-tc.services.mozilla.com/tasks/index/gecko.v2.mozilla-central.latest.system-symbols.debian 63 would return 404, which will make the symbol scraping tasks search its data on 64 GitHub.