triage-bugzilla.rst (12969B)
1 Triage for Bugzilla 2 =================== 3 4 Expectations 5 ------------ 6 7 All teams working on Firefox using either or both Mozilla-central and 8 Bugzilla are expected to follow the following process. 9 10 Components 11 ~~~~~~~~~~ 12 13 You should understand the list of components that your team is responsible for. 14 Each component must have a Triage Owner. (File a bug to update a component's 15 Triage Owner, or see the `Rotating triage <#rotating-triage>`__ section). 16 17 Triage Owners 18 ~~~~~~~~~~~~~ 19 20 Triage Owners are responsible for ensuring that bugs are triaged within the 21 expected timeframe, as well as acting as the primary point of contact for triage 22 related questions. While it's their responsibility to ensure triage happens, it 23 doesn't necessarily mean only they can or should perform triage. 24 25 A good starting point is for anyone senior enough in the team to triage bugs as 26 they are filed, leaving bugs that need discussion untriaged. Then schedule a 27 weekly triage meeting to discuss and triage bugs where required. 28 29 Triage 30 ~~~~~~ 31 32 Incoming bugs could be serious and we may need to react quickly. Users that have 33 invested the time to inform us of a bug would like to feel that we are 34 listening. 35 36 - All new bugs should be quickly assessed on a daily basis to ensure that 37 security bugs can be actioned quickly. 38 - All new bugs should be fully triaged, or under active investigation, within 39 one week of being created. 40 41 Important bugs monitoring and fixing 42 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 43 44 Bug severities are there for a reason. Tracked bugs must be taken particularly 45 seriously, too. Regressions are important to users, regressed functionality is 46 an indication that the product is getting worse not better. They should, 47 best-case, never hit release but be fixed in beta. 48 49 - All new S1 and sec-critical bugs should get the full attention of anyone that 50 can reasonably help. They should be assigned within 2 business days. 51 - All new S2 and sec-high bugs should be assigned (with caveats) and monitored 52 closely (weekly team meeting) 53 - Tracked bugs should be monitored closely (weekly team meeting) 54 - Regressions should be monitored closely (weekly team meeting) 55 56 All of this can be found on the “Important” tab of https://bugdash.moz.tools/ 57 (after selecting your components). 58 59 \*::General components 60 ~~~~~~~~~~~~~~~~~~~~~~ 61 62 We can't expect users to know the details of our component structure so we may 63 need to help routing bugs to the right place. 64 65 - Some teams have \*::General components. New bugs in these components should 66 be triaged into more specific components, within a week (unless we're 67 actively gathering information to decide where it belongs). Also note that 68 sometimes meta bugs don't have a better component, and in these cases, 69 \*::General is an appropriate long term home. 70 - (Core::General has its own `Definitions for Triage Owner Rotations <https://docs.google.com/spreadsheets/d/1EK6iCtdD8KP4UflIHscuZo6W5er2vy_TX7vsmaaBVd4/edit>`__) 71 72 Backlogs 73 ~~~~~~~~ 74 75 Backlogs are a feature of all software projects and we're never going to get to 76 zero bugs. However, given our already sizable backlogs, we should at least 77 ensure that our backlogs are not getting any worse. We have some tools to track 78 this on https://bugdash.moz.tools/. It has an "Overview" tab that shows you the 79 maintenance trends. Ideally over the past 12 weeks, the "Maint Effect" (i.e. 80 `Maintenance Effectiveness <https://docs.google.com/document/d/1y2dUDZI5U3xvY0jMY1LfIDARc5b_QB9mS2DV7MWrfa0/edit>`__) 81 should be over 100%, and the "Burn Down" (i.e. the time to zero bugs) should not 82 be "∞". 83 84 What is a Triaged Bug 85 --------------------- 86 87 The new definition of Triaged will be Firefox-related bugs of type 88 ``defect`` where the component is not 89 ``UNTRIAGED``, and a :ref:`Severity <Defect Severity>` value not equal 90 to ``--`` or ``N/A``. 91 92 Bugs of type Task or Enhancement may have a Severity of ``N/A``, 93 but defects must have a Severity that is neither ``--`` nor 94 ``N/A``. 95 96 Why Triage 97 ---------- 98 99 We want to make sure that we looked at every defect and a severity has 100 been defined. This way, we make sure that we did not miss any critical 101 issues during the development and stabilization cycles. 102 103 Staying on top of the bugs in your component means: 104 105 - You get ahead of critical regressions and crashes which could trigger 106 a point release if uncaught 107 108 - And you don’t want to spend your holiday with the Release 109 Management team (not that they don’t like you) 110 111 - Your bug queue is not overwhelming 112 113 - Members of your team do not see the bug queue and get the 114 ‘wiggins’ 115 116 Who Triages 117 ----------- 118 119 Engineering managers and directors are responsible for naming the 120 individuals responsible for triaging :ref:`all types of bugs <Bug Types>` in a component. 121 122 We use Bugzilla to manage this. See the `list of triage 123 owners <https://bugzilla.mozilla.org/page.cgi?id=triage_owners.html>`__. 124 125 If you need to change who is responsible for triaging a bug in a 126 component, please `file a bug against bugzilla.mozilla.org in the 127 Administration 128 component <https://bugzilla.mozilla.org/enter_bug.cgi?product=bugzilla.mozilla.org&component=Administration>`__. 129 When a new component is created, a triage owner **must** be named. 130 131 Rotating triage 132 ~~~~~~~~~~~~~~~ 133 134 Some components are monitored by a rotation of triagers. In those cases, 135 the triage owner on Bugzilla will be automatically updated to reflect the 136 person on the rotation. The rotations are managed as calendars. 137 138 If you wish to set up a rotation for triaging one or more components, 139 add a link to your rotation calendar in the `triage rotations spreadsheet <https://docs.google.com/spreadsheets/d/1EK6iCtdD8KP4UflIHscuZo6W5er2vy_TX7vsmaaBVd4>`__. 140 141 Firefox::General and Toolkit::General 142 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 143 144 Bugs in Firefox::General are fitted with Bug Bug’s model to see if 145 there’s another component with a high likelihood of fit, and if a 146 threshold confidence is achieved, the bug is moved to that component. 147 148 Members of the community also review bugs in this component and try to 149 move them. 150 151 What Do You Triage 152 ------------------ 153 154 As a triage owner the queries you should be following for your component 155 are: 156 157 - All open bugs, in your components without a pending ``needinfo`` flag 158 which do not have a valid value of severity set 159 - All bugs with active review requests in your component which have not 160 been modified in five days 161 - All bugs with reviewed, but unlanded patches in your components 162 - All bugs with a needinfo request unanswered for more than 10 days 163 164 Note: For bugs filed or reproduced by QA, the QA team will set an initial bug severity. 165 166 Changing a bug's severity requires a comment explaining the rationale. 167 168 There’s a tool with these queries to help you find bugs 169 https://bugdash.moz.tools/ and the source is at 170 https://github.com/mozilla/bugdash/. 171 172 If a bug is an enhancement it needs a priority set and a target release 173 or program milestone. These bugs are normally reviewed by product 174 managers. Enhancements can lead to release notes and QA needed that we 175 also need to know about 176 177 If a bug is a task resulting in a changeset, release managers will need 178 to known when this work will be done. A task such as refactoring fragile 179 code can be risky. 180 181 Weekly or More Frequently (depending on the component) find un-triaged 182 bugs in the components you triage. 183 184 Decide the :ref:`Severity <Defect Severity>` for each untriaged bug 185 (you can override what’s already been set.) 186 187 These bugs are reviewed in the weekly Regression Triage meeting 188 189 - Bugs of type ``defect`` with the ``regression`` keyword without 190 ``status-firefoxNN`` decisions 191 - Bugs of type ``defect`` with the ``regression`` keyword without 192 a regression range 193 194 Automatic Bug Updates 195 ~~~~~~~~~~~~~~~~~~~~~ 196 197 When a bug is tracked for a release, i.e. the ``tracking_firefoxNN`` 198 flag is set to ``+`` or ``blocking`` triage decisions will be overridden, 199 or made as follows: 200 201 - If a bug is tracked for or blocking beta, release or ESR, its 202 priority will be set to ``P1`` 203 - If a bug is tracked for or blocking nightly, its priority will be set 204 to ``P2`` 205 206 Because bugs can be bumped in priority it’s essential that triage owners 207 review their 208 `P1 <https://bugzilla.mozilla.org/buglist.cgi?priority=P1&f1=triage_owner&o1=equals&resolution=---&v1=%25user%25>`__ 209 and 210 `P2 <https://bugzilla.mozilla.org/buglist.cgi?priority=P2&f1=triage_owner&o1=equals&resolution=---&v1=%25user%25>`__ 211 bugs frequently. 212 213 Assumptions 214 ~~~~~~~~~~~ 215 216 If a bug's release status in Firefox version N was ``affected`` or ``wontfix``, 217 its Severity is ``S3`` or ``S4`` and its Priority is ``P3`` or lower (backlog,) 218 then its release status in Firefox version N+1, if the bug is still open, 219 is considered to be ``wontfix``. 220 221 Questions and Edge Cases 222 ------------------------ 223 224 This bug is a feature request 225 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 226 227 Set the bug’s type to ``enhancement``, add the ``feature`` keyword if 228 relevant, and state to ``NEW``. Set the bug's Severity to ``N/A``. This 229 bug will be excluded from future triage queries. 230 231 This bug is a task, not a defect 232 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 233 234 Set the bug’s type to ``task``, and state to ``NEW``. Set the bug's 235 Severity to ``N/A``. This bug will be excluded from future triage queries. 236 237 238 If you are not sure of a bug’s type, check :ref:`our rules for bug 239 types <Bug Types>`. 240 241 This bug’s state is ``UNCONFIRMED`` 242 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 243 244 Are there steps to reproduce? If not, needinfo the person who filed the 245 bug, requesting steps to reproduce. You are not obligated to wait 246 forever for a response, and bugs for which open requests for information 247 go unanswered can be ``RESOLVED`` as ``INCOMPLETE``. 248 249 I need help reproducing the bug 250 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 251 252 Set a needinfo for the QA managers, Softvision project managers, or the 253 QA owner of the component of the bug. 254 255 I don’t have enough information to make a decision 256 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 257 258 If you don’t have a reproduction or confirmation, or have questions 259 about how to proceed, ``needinfo`` the person who filed the bug, or 260 someone who can answer. 261 262 The ``stalled`` keyword 263 ~~~~~~~~~~~~~~~~~~~~~~~ 264 265 The extreme case of not-enough-information is one which cannot be 266 answered with a ``needinfo`` request. The reporter has shared all they 267 know about the bug, we are out of strategies to take to resolve it, but 268 the bug should be kept open. 269 270 Mark the bug as stalled by adding the ``stalled`` keyword to it. The 271 keyword will remove it from the list of bugs to be triaged. 272 273 If a patch lands on a ``stalled`` bug, automation will remove the 274 keyword. Otherwise, when the ``keyword`` is removed, the bug will have 275 its priority reset to ``--`` and the components triage owner notified by 276 automation. 277 278 Bugs which remain ``stalled`` for long periods of time should be 279 reviewed, and closed if necessary. 280 281 Bug is in the wrong Component 282 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 283 284 If the bug has a Severity of ``S3``, ``S4``, or ``N/A`` move the what 285 you think is the correct component, or needinfo the person 286 responsible for the component to ask them. 287 288 If the bug has a Severity of ``S1`` or ``S2`` then notify Release Management 289 and contact the triage owner of the component for which you think it belongs to. 290 We cannot lose track of a high severity bug because it is in the wrong component. 291 292 My project is on GitHub 293 ~~~~~~~~~~~~~~~~~~~~~~~ 294 295 We have :ref:`a guide for GitHub projects to follow <GitHub Metadata Recommendations>` when 296 triaging. (Note: this guide needs updating.) 297 298 Summary 299 ------- 300 301 Multiple times weekly 302 ~~~~~~~~~~~~~~~~~~~~~ 303 304 Use queries for the components you are responsible for in 305 https://github.com/mozilla/bugdash/ to find bugs in 306 need of triage. 307 308 For each untriaged bug: 309 310 - Assign a Severity 311 - **Do not** assign a ``defect`` a Severity of 312 ``N/A`` 313 314 You can, but are not required to set the bug's :ref:`Priority <Priority Definitions>`. 315 316 Watch open needinfo flags 317 ~~~~~~~~~~~~~~~~~~~~~~~~~ 318 319 Don’t let open needinfo flags linger for more than two weeks. 320 321 Close minor bugs with unresponded needinfo flags. 322 323 Follow up on needinfo flag requests. 324 325 `BugDash <https://github.com/mozilla/bugdash/>`__ will help you find these. 326 327 End of Iteration/Release Cycle 328 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 329 330 Any open ``S1`` or ``S2`` bugs at the end of the release cycle 331 will require review by engineering and release management. A 332 policy on this is forthcoming. 333 334 Optional 335 ^^^^^^^^ 336 337 (The guidelines on bug priority are under review.) 338 339 Are there open P1s? Revisit their priority, 340 and move to them to the backlog (``P3``) or ``P2``. 341 342 Are there ``P2`` bugs that should move to ``P1`` 343 for the next cycle? 344 345 Are there ``P2`` you now know are lower priority, 346 move to ``P3``. 347 348 Are there ``P3`` bugs you now know you won’t get to? 349 Either demote to ``P5`` (will accept patch) or 350 resolve as ``WONTFIX``. 351 352 Getting help 353 ------------ 354 355 - Ask in #bug-handling on chat.mozilla.org