index.rst (3397B)
1 .. taskcluster_index: 2 3 Firefox CI and Taskgraph 4 ======================== 5 6 Firefox's `continuous integration`_ (CI) system is made up of three parts. 7 8 First there's `Taskcluster`_, a task execution framework developed by Mozilla. 9 Taskcluster is capable of building complex, scalable and highly customizable CI 10 systems. Taskcluster is more like a set of building blocks that can be used to 11 create CI systems, rather than a fully-fledged CI system itself. 12 13 The second part is the `Firefox CI`_ instance of Taskcluster. This is the 14 Taskcluster deployment responsible for running most of Mozilla's CI needs. This 15 component is comprised of a Kubernetes cluster to run the Taskcluster services 16 (maintained by SRE Services), some customizations to support Taskcluster 17 on ``hg.mozilla.org`` and access control in the `fxci-config`_ repo 18 (maintained by Release Engineering). 19 20 The third part is `Taskgraph`_. Taskgraph is a Python library that can generate 21 a `DAG`_ of tasks and submit them to a Taskcluster instance. This is the 22 component that most Gecko and Mozilla developers will interact with when 23 working with tasks, and will be the focal point of the rest of this 24 documentation. 25 26 .. note:: 27 28 Historically Taskgraph started out inside ``mozilla-central``. It was then 29 forked to standalone `Taskgraph`_ in order to support projects on Github. 30 Over time maintaining two forks grew cumbersome so they are in the process 31 of being merged back together. 32 33 Today the version of Taskgraph under ``taskcluster/gecko_taskgraph`` depends 34 on the standalone version, which is vendored under 35 ``third_party/python/taskcluster_taskgraph``. There is still a lot of 36 duplication between these places, but ``gecko_taskgraph`` is slowly being 37 re-written to consume standalone Taskgraph. 38 39 The ``taskcluster`` directory contains all the files needed to define the graph 40 of tasks that must be executed to build and test the Gecko tree. This is more 41 complex than you think! There are 30,000+ tasks and counting in Gecko's CI 42 graphs. 43 44 Taskgraph supports: 45 46 * A huge array of tasks 47 * Different behavior for different repositories 48 * "Try" pushes, with special means to select a subset of the graph for execution 49 * Optimization -- skipping tasks that have already been performed 50 * Extremely flexible generation of a variety of tasks using an approach of 51 incrementally transforming job descriptions into task definitions. 52 53 The most comprehensive resource on Taskgraph is `Taskgraph's documentation`_. These docs 54 will refer there when appropriate and expand on topics specific to ``gecko_taskgraph`` 55 where necessary. 56 57 If you are reading this with a particular goal in mind and would rather avoid 58 becoming a task-graph expert, check out the :doc:`how-to section <howto/index>`. 59 60 .. _continuous integration: https://en.wikipedia.org/wiki/Continuous_integration 61 .. _Taskcluster: https://taskcluster.net/ 62 .. _Firefox CI: https://firefox-ci-tc.services.mozilla.com/ 63 .. _fxci-config: https://github.com/mozilla-releng/fxci-config 64 .. _Taskgraph: https://github.com/taskcluster/taskgraph 65 .. _DAG: https://en.wikipedia.org/wiki/Directed_acyclic_graph 66 .. _Taskgraph's documentation: https://taskcluster-taskgraph.readthedocs.io/en/latest/ 67 68 .. toctree:: 69 70 taskgraph 71 howto/index 72 transforms/index 73 optimization/index 74 cron 75 try 76 release-promotion 77 versioncontrol 78 config 79 reference