attributes.rst (18749B)
1 =============== 2 Task Attributes 3 =============== 4 5 Tasks can be filtered, for example to support "try" pushes which only perform a 6 subset of the task graph or to link dependent tasks. This filtering is the 7 difference between a full task graph and a target task graph. 8 9 Filtering takes place on the basis of attributes. Each task has a dictionary 10 of attributes and filters over those attributes can be expressed in Python. A 11 task may not have a value for every attribute. 12 13 The attributes, and acceptable values, are defined here. In general, attribute 14 names and values are the short, lower-case form, with underscores. 15 16 kind 17 ==== 18 19 A task's ``kind`` attribute gives the name of the kind that generated it, e.g., 20 ``build`` or ``spidermonkey``. 21 22 run_on_repo_type 23 ================ 24 25 The types of repositories where this task should be in the target task set. Typically 26 "hg" (the default), "git" or both. 27 28 This attribute is temporary and will be used during the transition from hg.mozilla.org 29 to Github. 30 31 run_on_projects 32 =============== 33 34 The projects where this task should be in the target task set. This is how 35 requirements like "only run this on autoland" get implemented. 36 37 .. note:: 38 39 Please use this configuration. Running a job for all projects can quickly add up 40 in term of cost while not providing any value for some projects. 41 42 `run-on-projects` can use either aliases or project names. 43 44 These are the aliases: 45 46 * `integration` -- integration repository (autoland) 47 * `trunk` -- integration repository plus mozilla-central 48 * `release` -- release repositories (beta, release, esr) including mozilla-central 49 * `all` -- everywhere (the default) 50 51 Project names are the repositories. They can be: 52 53 * `autoland` 54 * `mozilla-central` 55 * `mozilla-beta` 56 * `mozilla-release` 57 * `mozilla-esr140` 58 * ... A partial list can be found in taskcluster/gecko_taskgraph/util/attributes.py 59 60 For try, this attribute applies only if ``-p all`` is specified. All jobs can 61 be specified by name regardless of ``run_on_projects``. 62 63 If ``run_on_projects`` is set to an empty list, then the task will not run 64 anywhere, unless its build platform is specified explicitly in try syntax. 65 66 67 .. note:: 68 69 As `try` pushes don't use filter_for_projects by design, there isn't a way 70 to define that a task will run on `try`. 71 72 73 .. note:: 74 75 A given task `[taskA]` may not respect `run-on-projects` if there another task 76 `[taskB]` which is scheduled to run (such as via run-on-projects) which depends it 77 `[taskA]`. Because by nature of `TaskB` running we must run `TaskA`. 78 79 See `bug 1640603 <https://bugzilla.mozilla.org/show_bug.cgi?id=1640603#c5>`_ as example. 80 81 run_on_hg_branches 82 ================== 83 84 On a given project, the mercurial branch where this task should be in the target 85 task set. This is how requirements like "only run this RELBRANCH" get implemented. 86 These are either the regular expression of a branch (e.g.: ``GECKOVIEW_\d+_RELBRANCH``) 87 or the following alias: 88 89 * `all` -- everywhere (the default) 90 91 Like ``run_on_projects``, the same behavior applies if it is set to an empty list. 92 93 task_duplicates 94 =============== 95 96 This is used to indicate that we want multiple copies of the task created. 97 This feature is used to track down intermittent job failures. 98 99 If this value is set to N, the task-creation machinery will create a total of N 100 copies of the task. Only the first copy will be included in the taskgraph 101 output artifacts, although all tasks will be contained in the same taskGroup. 102 103 While most attributes are considered read-only, target task methods may alter 104 this attribute of tasks they include in the target set. 105 106 build_platform 107 ============== 108 109 The build platform defines the platform for which the binary was built. It is 110 set for both build and test jobs, although test jobs may have a different 111 ``test_platform``. 112 113 build_type 114 ========== 115 116 The type of build being performed. This is a subdivision of ``build_platform``, 117 used for different kinds of builds that target the same platform. Values are 118 119 * ``debug`` 120 * ``opt`` 121 122 test_platform 123 ============= 124 125 The test platform defines the platform on which tests are run. It is only 126 defined for test jobs and may differ from ``build_platform`` when the same binary 127 is tested on several platforms (for example, on several versions of Windows). 128 This applies for both talos and unit tests. 129 130 Unlike build_platform, the test platform is represented in a slash-separated 131 format, e.g., ``linux64/opt``. 132 133 unittest_suite 134 ============== 135 136 This is the unit test suite being run in a unit test task. For example, 137 ``mochitest`` or ``cppunittest``. 138 139 unittest_category 140 ================= 141 142 This is the high-level category of test the suite corresponds to. This is 143 usually the test harness used to run the suite. 144 145 unittest_try_name 146 ================= 147 148 This is the name used to refer to a unit test via try syntax. It 149 may not match ``unittest_suite``. 150 151 unittest_variant 152 ================ 153 154 The configuration variant the test suite is running with. If set, this usually 155 means the tests are running with a special pref enabled. These are defined in 156 ``taskgraph.transforms.test.TEST_VARIANTS``. 157 158 talos_try_name 159 ============== 160 161 This is the name used to refer to a talos job via try syntax. 162 163 raptor_try_name 164 =============== 165 166 This is the name used to refer to a raptor job via try syntax. 167 168 job_try_name 169 ============ 170 171 This is the name used to refer to a "job" via try syntax (``-j``). Note that for 172 some kinds, ``-j`` also matches against ``build_platform``. 173 174 test_chunk 175 ========== 176 177 This is the chunk number of a chunked test suite. Note that this is a string! 178 179 test_manifests 180 ============== 181 182 A list of the test manifests that run in this task. 183 184 image_name 185 ========== 186 187 For the ``docker_image`` kind, this attribute contains the docker image name. 188 189 nightly 190 ======= 191 192 Signals whether the task is part of a nightly graph. Useful when filtering 193 out nightly tasks from full task set at target stage. 194 195 shippable 196 ========= 197 Signals whether the task is considered "shippable", that it should get signed and is ok to 198 be used for nightlies or releases. 199 200 all_locales 201 =========== 202 203 For the ``l10n`` and ``shippable-l10n`` kinds, this attribute contains the list 204 of relevant locales for the platform. 205 206 all_locales_with_changesets 207 =========================== 208 209 Contains a dict of l10n changesets, mapped by locales (same as in ``all_locales``). 210 211 l10n_chunk 212 ========== 213 For the ``l10n`` and ``shippable-l10n`` kinds, this attribute contains the chunk 214 number of the job. Note that this is a string! 215 216 chunk_locales 217 ============= 218 For the ``l10n`` and ``shippable-l10n`` kinds, this attribute contains an array of 219 the individual locales this chunk is responsible for processing. 220 221 locale 222 ====== 223 For jobs that operate on only one locale, we set the attribute ``locale`` to the 224 specific locale involved. Currently this is only in l10n versions of the 225 ``beetmover`` and ``balrog`` kinds. 226 227 signed 228 ====== 229 Signals that the output of this task contains signed artifacts. 230 231 stub-installer 232 ============== 233 Signals to the build system that this build is expected to have a stub installer 234 present, and informs followon tasks to expect it. 235 236 repackage_type 237 ============== 238 This is the type of repackage. Can be ``repackage`` or 239 ``repackage_signing``. 240 241 fetch-artifact 242 ============== 243 244 For fetch jobs, this is the path to the artifact for that fetch operation. 245 246 fetch-alias 247 =========== 248 An alias that can be used instead of the real fetch job name in fetch 249 stanzas for jobs. 250 251 toolchain-artifact 252 ================== 253 For toolchain jobs, this is the path to the artifact for that toolchain. 254 255 toolchain-resources 256 =================== 257 For toolchain jobs, this is the list of paths that contribute to the toolchain's 258 digest hash. These include the build script, any resources specified in the task 259 definition, and the tooltool manifest (if present). 260 261 toolchain-extract 262 ================= 263 Control whether toolchain should be automatically extracted after download. 264 Default is true. 265 266 toolchain-alias 267 =============== 268 An alias that can be used instead of the real toolchain job name in fetch 269 stanzas for jobs. 270 271 toolchain-env 272 ============= 273 Extra environment variables that will be set on the worker when fetching this 274 toolchain. 275 276 toolchain-command 277 ================= 278 An extra attribute used to communicate to the build system bootstrap code the 279 command used to build the toolchain. This is useful because the commands are 280 formatted differently depending on the worker type, sometimes inconveniently 281 to parse afterwards. 282 283 always_target 284 ============= 285 286 Tasks with this attribute will be included in the ``target_task_graph`` if 287 ``parameters["tasks_for"]`` is ``hg-push``, regardless of any target task 288 filtering that occurs. When a task is included in this manner (i.e it otherwise 289 would have been filtered out), it will be considered for optimization even if 290 the ``optimize_target_tasks`` parameter is False. 291 292 This is meant to be used for tasks which a developer would almost always want to 293 run. Typically these tasks will be short running and have a high risk of causing 294 a backout. For example ``lint`` or ``python-unittest`` tasks. 295 296 shipping_product 297 ================ 298 For release promotion jobs, this is the product we are shipping. 299 300 shipping_phase 301 ============== 302 For release promotion jobs, this is the shipping phase (build, promote, push, ship). 303 During the build phase, we build and sign shippable builds. During the promote phase, 304 we generate l10n repacks and push to the candidates directory. During the push phase, 305 we push to the releases directory. During the ship phase, we update bouncer, push to 306 Google Play, version bump, mark as shipped in ship-it. 307 308 Using the "snowman model", we depend on previous graphs if they're defined. So if we 309 ask for a ``push`` (the head of the snowman) and point at the body and base, we only 310 build the head. If we don't point at the body and base, we build the whole snowman 311 (build, promote, push). 312 313 artifact_prefix 314 =============== 315 Most taskcluster artifacts are public, so we've hardcoded ``public/build`` in a 316 lot of places. To support private artifacts, we've moved this to the 317 ``artifact_prefix`` attribute. It will default to ``public/build`` but will be 318 overridable per-task. 319 320 artifact_map 321 =============== 322 For beetmover jobs, this indicates which yaml file should be used to 323 generate the upstream artifacts and payload instructions to the task. 324 325 release_artifacts 326 ================= 327 A set of artifacts that are candidates for downstream release tasks to operate 328 on. 329 330 batch 331 ===== 332 Used by `perftest` to indicates that a task can be run as a batch. 333 334 perftest_name 335 ============= 336 Used by `perftest` to indicate the test name being run. 337 338 enable-full-crashsymbols 339 ======================== 340 In automation, full crashsymbol package generation is normally disabled. For 341 build kinds where the full crashsymbols should be enabled, set this attribute 342 to True. The full symbol packages will then be generated and uploaded on 343 release branches and on try. 344 345 skip-upload-crashsymbols 346 ======================== 347 Shippable/nightly builds are normally required to set enable-full-crashsymbols, 348 but in some limited corner cases (universal builds), that is not wanted, because 349 the symbols are uploaded independently already. 350 351 upload-generated-sources 352 ======================== 353 generated-sources are normally only uploaded to S3 for shippable/nightly 354 builds. This attributes turns that on for other builds such as macosx 355 before unification. 356 357 cron 358 ==== 359 Indicates that a task is meant to be run via cron tasks, and should not be run 360 on push. 361 362 cached_task 363 =========== 364 Some tasks generate artifacts that are cached between pushes. This is a 365 dictionary with the type and name of the cache, and the unique string used to 366 identify the current version of the artifacts. See :py:mod:`taskgraph.util.cached_task`. 367 368 .. code:: yaml 369 370 cached_task: 371 digest: 66dfc2204600b48d92a049b6a18b83972bb9a92f9504c06608a9c20eb4c9d8ae 372 name: debian7-base 373 type: docker-images.v2 374 375 eager_indexes 376 ============= 377 A list of strings of indexes to populate before the task ever completes. Some tasks (e.g. cached tasks) we 378 want to exist in the index before they even run/complete. Our current use is to allow us to depend on an 379 unfinished cached task in future pushes. This avoids extra overhead from multiple tasks running, and 380 can allow us to have our results in just a bit earlier. 381 382 required_signoffs 383 ================= 384 A list of release signoffs that this kind requires, should the release also 385 require these signoffs. For example, ``mar-signing`` signoffs may be required 386 by some releases in the future; for any releases that require ``mar-signing`` 387 signoffs, the kinds that also require that signoff are marked with this 388 attribute. 389 390 update-channel 391 ============== 392 The update channel the build is configured to use. 393 394 mar-channel-id 395 ============== 396 The mar-channel-id the build is configured to use. 397 398 accepted-mar-channel-ids 399 ======================== 400 The mar-channel-ids this build will accept updates to. It should usually be the same as 401 the value mar_channel_id. If more than one ID is needed, then you should use a 402 comma separated list of values. 403 404 openh264_rev 405 ============ 406 Only used for openh264 plugin builds, used to signify the revision (and thus inform artifact name) of the given build. 407 408 code-review 409 =========== 410 If a task set this boolean attribute to `true`, it will be processed by the code 411 review bot, the task will ran for every new Phabricator diff. 412 Any supported and detected issue will be automatically reported on the 413 Phabricator revision. 414 415 retrigger 416 ========= 417 Whether the task can be retriggered, or if it needs to be re-run. 418 419 disable-push-apk 420 ================ 421 Some GeckoView-only Android tasks produce APKs that shouldn't be 422 pushed to the Google Play Store. Set this to ``true`` to disable 423 pushing. 424 425 disable-build-signing 426 ===================== 427 Some GeckoView-only tasks produce APKs, but not APKs that should be 428 signed. Set this to ``true`` to disable APK signing. 429 430 enable-build-signing 431 ==================== 432 We enable build-signing for ``shippable``, ``nightly``, and ``enable-build-signing`` tasks. 433 434 run-visual-metrics 435 ================== 436 If set to true, will run the visual metrics task on the provided 437 video files. 438 439 skip-verify-test-packaging 440 ========================== 441 If set to true, this task will not be checked to see that 442 MOZ_AUTOMATION_PACKAGE_TESTS is set correctly based on whether or not the task 443 has dependent tests. This should only be used in very unique situations, such 444 as Windows AArch64 builds that copy test packages between build tasks. 445 446 geckodriver 447 =========== 448 If non-empty, declares that the (toolchain) task is a `geckodriver` 449 task that produces a binary that should be signed. 450 451 rebuild-on-release 452 ================== 453 If true, the digest for this task will also depend on if the branch is a 454 release branch. This will cause tasks like toolchains to be rebuilt as they 455 move from e.g. autoland to mozilla-central. 456 457 local-fetch 458 =============== 459 This fetch is used for local development, so should be built on trunk, even 460 if it does not have any in-graph consumers. 461 462 local-toolchain 463 =============== 464 This toolchain is used for local development, so should be built on trunk, even 465 if it does not have any in-graph consumers. 466 467 artifact-build 468 ============== 469 470 This build is an artifact build. 471 472 This deliberately excludes builds that are implemented using the artifact build 473 machinery, but are not primarily intended to short-circuit build time. In 474 particular the Windows aarch64 builds are not marked this way. 475 476 maven_packages 477 =============== 478 List of maven packages produced by the build. 479 480 supports-artifact-builds 481 ======================== 482 483 If false, the task requires a compiled build and will not work with artifact builds. 484 485 primary-kind-dependency 486 ======================= 487 488 For tasks that were derived from a group of dependencies, this attribute 489 denotes which dependent kind is the `primary one`_. 490 491 Typically this is set by the ``taskgraph.transforms.from_deps`` transforms. 492 493 primary-dependency-label 494 ======================== 495 496 This attribute denotes which upstream task is the `primary one`_, and is typically used to indicate which upstream task other attributes should be derived from. 497 498 snap_test_type 499 ============== 500 501 For Snap tests tasks, used to disambiguate task label 502 503 snap_test_release 504 ================= 505 506 For Snap tests tasks, used to disambiguate task label by distro release 507 508 .. _primary one: https://taskcluster-taskgraph.readthedocs.io/en/latest/reference/transforms/from_deps.html#primary-kind 509 510 build-type 511 ========== 512 For android builds, the type of build: typically debug, nightly, beta, release. 513 514 component 515 ========= 516 For android-components builds, the name of the component. 517 518 gradle-project 519 ============== 520 For android-components builds, the complete gradle name of the component. 521 522 apks 523 ==== 524 For android apk build tasks, the list of apk artifact names. 525 526 aab 527 === 528 For android aab build tasks, the aab artifact names. 529 530 release-type 531 ============ 532 For android builds, the type of release: typically debug, nightly, beta, release. 533 534 shipping-product 535 ================ 536 For android tasks, the shipping product: Fenix, Focus, or an android-component name. 537 538 artifacts 539 ========= 540 For android-components tasks, a dictionary mapping extensions to artifact names. 541 542 is_final_chunked_task 543 ===================== 544 Used by taskcluster.android_taskgraph.transforms.chunk to mark the last task in a chunk group. 545 546 legacy 547 ====== 548 Used by Fenix UI tests to select UI tests on legacy Android devices and API levels. 549 550 nightly-test 551 ============ 552 Used by Fenix browsertime tests to mark the task as part of the nightly-test graph. 553 554 screenshots 555 =========== 556 Used by Fenix UI tests to mark the task as part of the screenshots graph. 557 558 abi 559 === 560 Used by android browsertime tasks to track the abi of the product under test. 561 562 apk 563 === 564 Used by android browsertime tasks to track the path to the apk of the product under test. 565 566 test-manifests 567 ============== 568 A list of the test manifests that run in this task. 569 570 lull-schedule 571 ============= 572 Used by performance tasks to schedule them at a specified frequency in a best-effort method. Schedules them when the overall CI load is low for a given platform. Use "w" for weeks, "d" for days, "h" for hours, and "m" for minutes in a string like so to specify the scheduling frequency: 1d, 1w 4h, 2w 4d 1h. 573 574 this_chunk 575 ============= 576 Used by source tests to support chunking and specify a current chunk. 577 578 total_chunks 579 ============= 580 Used by source tests to support chunking and specify a total amount of chunks. 581 582 matrix 583 ====== 584 Used to generate several tasks from a task, using the substitution given in the 585 matrix. 586 587 dxc-crash-symbols 588 ================= 589 590 Indicates that the job produces crash symbols for `dxcompiler.dll`.