files-metadata.rst (6436B)
1 .. _mozbuild_files_metadata: 2 3 ============== 4 Files Metadata 5 ============== 6 7 :ref:`mozbuild-files` provide a mechanism for attaching metadata to 8 files. Essentially, you define some flags to set on a file or file 9 pattern. Later, some tool or process queries for metadata attached to a 10 file of interest and it does something intelligent with that data. 11 12 Defining Metadata 13 ================= 14 15 Files metadata is defined by using the 16 :ref:`Files Sub-Context <mozbuild_subcontext_Files>` in ``moz.build`` 17 files. e.g.:: 18 19 with Files('**/Makefile.in'): 20 BUG_COMPONENT = ('Firefox Build System', 'General') 21 22 This working example says, *for all Makefile.in files in every directory 23 underneath this one - including this directory - set the Bugzilla 24 component to Firefox Build System :: General*. 25 26 For more info, read the 27 :ref:`docs on Files <mozbuild_subcontext_Files>`. 28 29 How Metadata is Read 30 ==================== 31 32 ``Files`` metadata is extracted in :ref:`mozbuild_fs_reading_mode`. 33 34 Reading starts by specifying a set of files whose metadata you are 35 interested in. For each file, the filesystem is walked to the root 36 of the source directory. Any ``moz.build`` encountered during this 37 walking are marked as relevant to the file. 38 39 Let's say you have the following filesystem content:: 40 41 /moz.build 42 /root_file 43 /dir1/moz.build 44 /dir1/foo 45 /dir1/subdir1/foo 46 /dir2/foo 47 48 For ``/root_file``, the relevant ``moz.build`` files are just 49 ``/moz.build``. 50 51 For ``/dir1/foo`` and ``/dir1/subdir1/foo``, the relevant files are 52 ``/moz.build`` and ``/dir1/moz.build``. 53 54 For ``/dir2``, the relevant file is just ``/moz.build``. 55 56 Once the list of relevant ``moz.build`` files is obtained, each 57 ``moz.build`` file is evaluated. Root ``moz.build`` file first, 58 leaf-most files last. This follows the rules of 59 :ref:`mozbuild_fs_reading_mode`, with the set of evaluated ``moz.build`` 60 files being controlled by filesystem content, not ``DIRS`` variables. 61 62 The file whose metadata is being resolved maps to a set of ``moz.build`` 63 files which in turn evaluates to a list of contexts. For file metadata, 64 we only care about one of these contexts: 65 :ref:`Files <mozbuild_subcontext_Files>`. 66 67 We start with an empty ``Files`` instance to represent the file. As 68 we encounter a *files sub-context*, we see if it is appropriate to 69 this file. If it is, we apply its values. This process is repeated 70 until all *files sub-contexts* have been applied or skipped. The final 71 state of the ``Files`` instance is used to represent the metadata for 72 this particular file. 73 74 It may help to visualize this. Say we have 2 ``moz.build`` files:: 75 76 # /moz.build 77 with Files('*.cpp'): 78 BUG_COMPONENT = ('Core', 'XPCOM') 79 80 with Files('**/*.js'): 81 BUG_COMPONENT = ('Firefox', 'General') 82 83 # /foo/moz.build 84 with Files('*.js'): 85 BUG_COMPONENT = ('Another', 'Component') 86 87 Querying for metadata for the file ``/foo/test.js`` will reveal 3 88 relevant ``Files`` sub-contexts. They are evaluated as follows: 89 90 1. ``/moz.build - Files('*.cpp')``. Does ``/*.cpp`` match 91 ``/foo/test.js``? **No**. Ignore this context. 92 2. ``/moz.build - Files('**/*.js')``. Does ``/**/*.js`` match 93 ``/foo/test.js``? **Yes**. Apply ``BUG_COMPONENT = ('Firefox', 'General')`` 94 to us. 95 3. ``/foo/moz.build - Files('*.js')``. Does ``/foo/*.js`` match 96 ``/foo/test.js``? **Yes**. Apply 97 ``BUG_COMPONENT = ('Another', 'Component')``. 98 99 At the end of execution, we have 100 ``BUG_COMPONENT = ('Another', 'Component')`` as the metadata for 101 ``/foo/test.js``. 102 103 One way to look at file metadata is as a stack of data structures. 104 Each ``Files`` sub-context relevant to a given file is applied on top 105 of the previous state, starting from an empty state. The final state 106 wins. 107 108 .. _mozbuild_files_metadata_finalizing: 109 110 Finalizing Values 111 ================= 112 113 The default behavior of ``Files`` sub-context evaluation is to apply new 114 values on top of old. In most circumstances, this results in desired 115 behavior. However, there are circumstances where this may not be 116 desired. There is thus a mechanism to *finalize* or *freeze* values. 117 118 Finalizing values is useful for scenarios where you want to prevent 119 wildcard matches from overwriting previously-set values. This is useful 120 for one-off files. 121 122 Let's take ``Makefile.in`` files as an example. The build system module 123 policy dictates that ``Makefile.in`` files are part of the ``Build 124 Config`` module and should be reviewed by peers of that module. However, 125 there exist ``Makefile.in`` files in many directories in the source 126 tree. Without finalization, a ``*`` or ``**`` wildcard matching rule 127 would match ``Makefile.in`` files and overwrite their metadata. 128 129 Finalizing of values is performed by setting the ``FINAL`` variable 130 on ``Files`` sub-contexts. See the 131 :ref:`Files documentation <mozbuild_subcontext_Files>` for more. 132 133 Here is an example with ``Makefile.in`` files, showing how it is 134 possible to finalize the ``BUG_COMPONENT`` value.:: 135 136 # /moz.build 137 with Files('**/Makefile.in'): 138 BUG_COMPONENT = ('Firefox Build System', 'General') 139 FINAL = True 140 141 # /foo/moz.build 142 with Files('**'): 143 BUG_COMPONENT = ('Another', 'Component') 144 145 If we query for metadata of ``/foo/Makefile.in``, both ``Files`` 146 sub-contexts match the file pattern. However, since ``BUG_COMPONENT`` is 147 marked as finalized by ``/moz.build``, the assignment from 148 ``/foo/moz.build`` is ignored. The final value for ``BUG_COMPONENT`` 149 is ``('Firefox Build System', 'General')``. 150 151 Here is another example:: 152 153 with Files('*.cpp'): 154 BUG_COMPONENT = ('One-Off', 'For C++') 155 FINAL = True 156 157 with Files('**'): 158 BUG_COMPONENT = ('Regular', 'Component') 159 160 For every files except ``foo.cpp``, the bug component will be resolved 161 as ``Regular :: Component``. However, ``foo.cpp`` has its value of 162 ``One-Off :: For C++`` preserved because it is finalized. 163 164 .. important:: 165 166 ``FINAL`` only applied to variables defined in a context. 167 168 If you want to mark one variable as finalized but want to leave 169 another mutable, you'll need to use 2 ``Files`` contexts. 170 171 Guidelines for Defining Metadata 172 ================================ 173 174 In general, values defined towards the root of the source tree are 175 generic and become more specific towards the leaves. For example, 176 the ``BUG_COMPONENT`` for ``/browser`` might be ``Firefox :: General`` 177 whereas ``/browser/components/preferences`` would list 178 ``Firefox :: Preferences``.