write_build_config.py (95965B)
1 #!/usr/bin/env python3 2 # 3 # Copyright 2014 The Chromium Authors 4 # Use of this source code is governed by a BSD-style license that can be 5 # found in the LICENSE file. 6 7 """Writes a build_config file. 8 9 The build_config file for a target is a json file containing information about 10 how to build that target based on the target's dependencies. This includes 11 things like: the javac classpath, the list of android resources dependencies, 12 etc. It also includes the information needed to create the build_config for 13 other targets that depend on that one. 14 15 Android build scripts should not refer to the build_config directly, and the 16 build specification should instead pass information in using the special 17 file-arg syntax (see build_utils.py:ExpandFileArgs). That syntax allows passing 18 of values in a json dict in a file and looks like this: 19 --python-arg=@FileArg(build_config_path:javac:classpath) 20 21 Note: If paths to input files are passed in this way, it is important that: 22 1. inputs/deps of the action ensure that the files are available the first 23 time the action runs. 24 2. Either (a) or (b) 25 a. inputs/deps ensure that the action runs whenever one of the files changes 26 b. the files are added to the action's depfile 27 28 NOTE: All paths within .build_config files are relative to $OUTPUT_CHROMIUM_DIR. 29 30 This is a technical note describing the format of .build_config files. 31 Please keep it updated when changing this script. For extraction and 32 visualization instructions, see build/android/docs/build_config.md 33 34 ------------- BEGIN_MARKDOWN --------------------------------------------------- 35 The .build_config file format 36 === 37 38 # Introduction 39 40 This document tries to explain the format of `.build_config` generated during 41 the Android build of Chromium. For a higher-level explanation of these files, 42 please read 43 [build/android/docs/build_config.md](build/android/docs/build_config.md). 44 45 # The `deps_info` top-level dictionary: 46 47 All `.build_config` files have a required `'deps_info'` key, whose value is a 48 dictionary describing the target and its dependencies. The latter has the 49 following required keys: 50 51 ## Required keys in `deps_info`: 52 53 * `deps_info['type']`: The target type as a string. 54 55 The following types are known by the internal GN build rules and the 56 build scripts altogether: 57 58 * [java_binary](#target_java_binary) 59 * [java_annotation_processor](#target_java_annotation_processor) 60 * [robolectric_binary](#target_robolectric_binary) 61 * [java_library](#target_java_library) 62 * [android_assets](#target_android_assets) 63 * [android_resources](#target_android_resources) 64 * [android_apk](#target_android_apk) 65 * [android_app_bundle_module](#target_android_app_bundle_module) 66 * [android_app_bundle](#target_android_app_bundle) 67 * [dist_jar](#target_dist_jar) 68 * [dist_aar](#target_dist_aar) 69 * [group](#target_group) 70 71 See later sections for more details of some of these. 72 73 * `deps_info['path']`: Path to the target's `.build_config` file. 74 75 * `deps_info['name']`: Nothing more than the basename of `deps_info['path']` 76 at the moment. 77 78 * `deps_info['deps_configs']`: List of paths to the `.build_config` files of 79 all *direct* dependencies of the current target. 80 81 NOTE: Because the `.build_config` of a given target is always generated 82 after the `.build_config` of its dependencies, the `write_build_config.py` 83 script can use chains of `deps_configs` to compute transitive dependencies 84 for each target when needed. 85 86 ## Optional keys in `deps_info`: 87 88 The following keys will only appear in the `.build_config` files of certain 89 target types: 90 91 * `deps_info['requires_android']`: True to indicate that the corresponding 92 code uses Android-specific APIs, and thus cannot run on the host within a 93 regular JVM. May only appear in Java-related targets. 94 95 * `deps_info['supports_android']`: 96 May appear in Java-related targets, and indicates that 97 the corresponding code doesn't use Java APIs that are not available on 98 Android. As such it may run either on the host or on an Android device. 99 100 * `deps_info['assets']`: 101 Only seen for the [`android_assets`](#target_android_assets) type. See below. 102 103 * `deps_info['package_name']`: Java package name associated with this target. 104 105 NOTE: For `android_resources` targets, 106 this is the package name for the corresponding R class. For `android_apk` 107 targets, this is the corresponding package name. This does *not* appear for 108 other target types. 109 110 * `deps_info['android_manifest']`: 111 Path to an AndroidManifest.xml file related to the current target. 112 113 * `deps_info['base_module_config']`: 114 Only seen for the [`android_app_bundle`](#target_android_app_bundle) type. 115 Path to the base module for the bundle. 116 117 * `deps_info['module_name']`: 118 Only seen for the 119 [`android_app_bundle_module`](#target_android_app_bundle_module) 120 type. The name of the feature module. 121 122 * `deps_info['dependency_zips']`: 123 List of `deps_info['resources_zip']` entries for all `android_resources` 124 dependencies for the current target. 125 126 * `deps_info['extra_package_names']`: 127 Always empty for `android_resources` types. Otherwise, 128 the list of `deps_info['package_name']` entries for all `android_resources` 129 dependencies for the current target. Computed automatically by 130 `write_build_config.py`. 131 132 * `deps_info['dependency_r_txt_files']`: 133 Exists only on dist_aar. It is the list of deps_info['r_text_path'] from 134 transitive dependencies. Computed automatically. 135 136 137 # `.build_config` target types description: 138 139 ## <a name="target_group">Target type `group`</a>: 140 141 This type corresponds to a simple target that is only used to group 142 dependencies. It matches the `java_group()` GN template. Its only top-level 143 `deps_info` keys are `supports_android` (always True), and `deps_configs`. 144 145 146 ## <a name="target_android_resources">Target type `android_resources`</a>: 147 148 This type corresponds to targets that are used to group Android resource files. 149 For example, all `android_resources` dependencies of an `android_apk` will 150 end up packaged into the final APK by the build system. 151 152 It uses the following keys: 153 154 155 * `deps_info['res_sources_path']`: 156 Path to file containing a list of resource source files used by the 157 android_resources target. 158 159 * `deps_info['resources_zip']`: 160 *Required*. Path to the `.resources.zip` file that contains all raw/uncompiled 161 resource files for this target (and also no `R.txt`, `R.java` or `R.class`). 162 163 If `deps_info['res_sources_path']` is missing, this must point to a prebuilt 164 `.aar` archive containing resources. Otherwise, this will point to a zip 165 archive generated at build time, wrapping the sources listed in 166 `deps_info['res_sources_path']` into a single zip file. 167 168 * `deps_info['package_name']`: 169 Java package name that the R class for this target belongs to. 170 171 * `deps_info['android_manifest']`: 172 Optional. Path to the top-level Android manifest file associated with these 173 resources (if not provided, an empty manifest will be used to generate R.txt). 174 175 * `deps_info['resource_overlay']`: 176 Optional. Whether the resources in resources_zip should override resources with 177 the same name. Does not affect the behaviour of any android_resources() 178 dependencies of this target. If a target with resource_overlay=true depends 179 on another target with resource_overlay=true the target with the dependency 180 overrides the other. 181 182 * `deps_info['r_text_path']`: 183 Provide the path to the `R.txt` file that describes the resources wrapped by 184 this target. Normally this file is generated from the content of the resource 185 directories or zip file, but some targets can provide their own `R.txt` file 186 if they want. 187 188 * `deps_info['srcjar_path']`: 189 Path to the `.srcjar` file that contains the auto-generated `R.java` source 190 file corresponding to the content of `deps_info['r_text_path']`. This is 191 *always* generated from the content of `deps_info['r_text_path']` by the 192 `build/android/gyp/process_resources.py` script. 193 194 ## <a name="target_android_assets">Target type `android_assets`</a>: 195 196 This type corresponds to targets used to group Android assets, i.e. liberal 197 files that will be placed under `//assets/` within the final APK. 198 199 These use an `deps_info['assets']` key to hold a dictionary of values related 200 to assets covered by this target. 201 202 * `assets['sources']`: 203 The list of all asset source paths for this target. Each source path can 204 use an optional `:<zipPath>` suffix, where `<zipPath>` is the final location 205 of the assets (relative to `//assets/`) within the APK. 206 207 * `assets['outputs']`: 208 Optional. Some of the sources might be renamed before being stored in the 209 final //assets/ sub-directory. When this happens, this contains a list of 210 all renamed output file paths 211 212 NOTE: When not empty, the first items of `assets['sources']` must match 213 every item in this list. Extra sources correspond to non-renamed sources. 214 215 NOTE: This comes from the `asset_renaming_destinations` parameter for the 216 `android_assets()` GN template. 217 218 * `assets['disable_compression']`: 219 Optional. Will be True to indicate that these assets should be stored 220 uncompressed in the final APK. For example, this is necessary for locale 221 .pak files used by the System WebView feature. 222 223 * `assets['treat_as_locale_paks']`: 224 Optional. Will be True to indicate that these assets are locale `.pak` files 225 (containing localized strings for C++). These are later processed to generate 226 a special ``.build_config`.java` source file, listing all supported Locales in 227 the current build. 228 229 230 ## <a name="target_java_library">Target type `java_library`</a>: 231 232 This type is used to describe target that wrap Java bytecode, either created 233 by compiling sources, or providing them with a prebuilt jar. 234 235 * `deps_info['public_deps_configs']`: List of paths to the `.build_config` files 236 of *direct* dependencies of the current target which are exposed as part of the 237 current target's public API. 238 239 * `deps_info['unprocessed_jar_path']`: 240 Path to the original .jar file for this target, before any kind of processing 241 through Proguard or other tools. For most targets this is generated 242 from sources, with a name like `$target_name.javac.jar`. However, when using 243 a prebuilt jar, this will point to the source archive directly. 244 245 * `deps_info['device_jar_path']`: 246 Path to a file that is the result of processing 247 `deps_info['unprocessed_jar_path']` with various tools (ready to be dexed). 248 249 * `deps_info['host_jar_path']`: 250 Path to a file that is the result of processing 251 `deps_info['unprocessed_jar_path']` with various tools (use by java_binary). 252 253 * `deps_info['interface_jar_path']: 254 Path to the interface jar generated for this library. This corresponds to 255 a jar file that only contains declarations. Generated by running the `ijar` on 256 `deps_info['unprocessed_jar_path']` or the `turbine` tool on source files. 257 258 * `deps_info['dex_path']`: 259 Path to the `.dex` file generated for this target, from 260 `deps_info['device_jar_path']` unless this comes from a prebuilt `.aar` archive. 261 262 * `deps_info['is_prebuilt']`: 263 True to indicate that this target corresponds to a prebuilt `.jar` file. 264 In this case, `deps_info['unprocessed_jar_path']` will point to the source 265 `.jar` file. Otherwise, it will be point to a build-generated file. 266 267 * `deps_info['target_sources_file']`: 268 Path to a single `.sources` file listing all the Java and Kotlin sources that 269 were used to generate the library (simple text format, one `.jar` path per 270 line). 271 272 * `deps_info['lint_android_manifest']`: 273 Path to an AndroidManifest.xml file to use for this lint target. 274 275 * `deps_info['lint_sources']`: 276 The list of all `deps_info['target_sources_file']` entries for all library 277 dependencies that are chromium code. Note: this is a list of files, where each 278 file contains a list of Java and Kotlin source files. This is used for lint. 279 280 * `deps_info['lint_aars']`: 281 List of all aars from transitive java dependencies. This allows lint to collect 282 their custom annotations.zip and run checks like @IntDef on their annotations. 283 284 * `deps_info['lint_srcjars']`: 285 List of all bundled srcjars of all transitive java library targets. Excludes 286 non-chromium java libraries. 287 288 * `deps_info['lint_resource_sources']`: 289 List of all resource sources files belonging to all transitive resource 290 dependencies of this target. Excludes resources owned by non-chromium code. 291 292 * `deps_info['lint_resource_zips']`: 293 List of all resource zip files belonging to all transitive resource dependencies 294 of this target. Excludes resources owned by non-chromium code. 295 296 * `deps_info['javac']`: 297 A dictionary containing information about the way the sources in this library 298 are compiled. Appears also on other Java-related targets. See the [dedicated 299 section about this](#dict_javac) below for details. 300 301 * `deps_info['javac_full_classpath']`: 302 The classpath used when performing bytecode processing. Essentially the 303 collection of all `deps_info['unprocessed_jar_path']` entries for the target 304 and all its dependencies. 305 306 * `deps_info['javac_full_interface_classpath']`: 307 The classpath used when using the errorprone compiler. 308 309 * `deps_info['proguard_enabled"]`: 310 True to indicate that ProGuard processing is enabled for this target. 311 312 * `deps_info['proguard_configs"]`: 313 A list of paths to ProGuard configuration files related to this library. 314 315 * `deps_info['extra_classpath_jars']: 316 For some Java related types, a list of extra `.jar` files to use at build time 317 but not at runtime. 318 319 ## <a name="target_java_binary">Target type `java_binary`</a>: 320 321 This type corresponds to a Java binary, which is nothing more than a 322 `java_library` target that also provides a main class name. It thus inherits 323 all entries from the `java_library` type, and adds: 324 325 * `deps_info['main_class']`: 326 Name of the main Java class that serves as an entry point for the binary. 327 328 * `deps_info['device_classpath']`: 329 The classpath used when running a Java or Android binary. Essentially the 330 collection of all `deps_info['device_jar_path']` entries for the target and all 331 its dependencies. 332 333 * `deps_info['all_dex_files']`: 334 The list of paths to all `deps_info['dex_path']` entries for all libraries 335 that comprise this APK. Valid only for debug builds. 336 337 * `deps_info['preferred_dep']`: 338 Whether the target should be the preferred dep. This is usually the case when we 339 have a java_group that depends on either the public or internal dep accordingly, 340 and it is better to depend on the group rather than the underlying dep. Another 341 case is for android_library_factory targets, the factory target should be 342 preferred instead of the actual implementation. 343 344 ## <a name="target_robolectric_binary">Target type `robolectric_binary`</a>: 345 346 A target type for JUnit-specific binaries. Identical to 347 [`java_binary`](#target_java_binary) in the context of `.build_config` files, 348 except the name. 349 350 351 ## <a name="target_java_annotation_processor">Target type \ 352 `java_annotation_processor`</a>: 353 354 A target type for Java annotation processors. Identical to 355 [`java_binary`](#target_java_binary) in the context of `.build_config` files, 356 except the name, except that it requires a `deps_info['main_class']` entry. 357 358 359 ## <a name="target_android_apk">Target type `android_apk`</a>: 360 361 Corresponds to an Android APK. Inherits from the 362 [`java_binary`](#target_java_binary) type and adds: 363 364 * `deps_info['apk_path']`: 365 Path to the raw, unsigned, APK generated by this target. 366 367 * `deps_info['incremental_apk_path']`: 368 Path to the raw, unsigned, incremental APK generated by this target. 369 370 * `deps_info['incremental_install_json_path']`: 371 Path to the JSON file with per-apk details for incremental install. 372 See `build/android/gyp/incremental/write_installer_json.py` for more 373 details about its content. 374 375 * `deps_info['dist_jar']['all_interface_jars']`: 376 For `android_apk` and `dist_jar` targets, a list of all interface jar files 377 that will be merged into the final `.jar` file for distribution. 378 379 * `deps_info['final_dex']['path']`: 380 Path to the final classes.dex file (or classes.zip in case of multi-dex) 381 for this APK - only used for proguarded builds. 382 383 * `native['libraries']` 384 List of native libraries for the primary ABI to be embedded in this APK. 385 E.g. [ "libchrome.so" ] (i.e. this doesn't include any ABI sub-directory 386 prefix). 387 388 * `native['java_libraries_list']` 389 The same list as `native['libraries']` as a string holding a Java source 390 fragment, e.g. `"{\"chrome\"}"`, without any `lib` prefix, and `.so` 391 suffix (as expected by `System.loadLibrary()`). 392 393 * `native['second_abi_libraries']` 394 List of native libraries for the secondary ABI to be embedded in this APK. 395 Empty if only a single ABI is supported. 396 397 * `native['loadable_modules']` 398 A list of native libraries to store within the APK, in addition to those from 399 `native['libraries']`. These correspond to things like the Chromium linker 400 or instrumentation libraries. 401 402 * `native['secondary_abi_loadable_modules']` 403 Secondary ABI version of loadable_modules 404 405 * `native['library_always_compress']` 406 A list of library files that we always compress. 407 408 * `assets` 409 A list of assets stored compressed in the APK. Each entry has the format 410 `<source-path>:<destination-path>`, where `<source-path>` is relative to 411 `$CHROMIUM_OUTPUT_DIR`, and `<destination-path>` is relative to `//assets/` 412 within the APK. 413 414 NOTE: Not to be confused with the `deps_info['assets']` dictionary that 415 belongs to `android_assets` targets only. 416 417 * `uncompressed_assets` 418 A list of uncompressed assets stored in the APK. Each entry has the format 419 `<source-path>:<destination-path>` too. 420 421 * `locales_java_list` 422 A string holding a Java source fragment that gives the list of locales stored 423 uncompressed as android assets. 424 425 * `extra_android_manifests` 426 A list of `deps_configs['android_manifest]` entries, for all resource 427 dependencies for this target. I.e. a list of paths to manifest files for 428 all the resources in this APK. These will be merged with the root manifest 429 file to generate the final one used to build the APK. 430 431 * `java_resources_jars` 432 This is a list of `.jar` files whose *Java* resources should be included in 433 the final APK. For example, this is used to copy the `.res` files from the 434 EMMA Coverage tool. The copy will omit any `.class` file and the top-level 435 `//meta-inf/` directory from the input jars. Everything else will be copied 436 into the final APK as-is. 437 438 NOTE: This has nothing to do with *Android* resources. 439 440 * `deps_info['proguard_all_configs']`: 441 The collection of all 'deps_info['proguard_configs']` values from this target 442 and all its dependencies. 443 444 * `deps_info['proguard_classpath_jars']`: 445 The collection of all 'deps_info['extra_classpath_jars']` values from all 446 dependencies. 447 448 * `deps_info['proguard_under_test_mapping']`: 449 Applicable to apks with proguard enabled that have an apk_under_test. This is 450 the path to the apk_under_test's output proguard .mapping file. 451 452 ## <a name="target_android_app_bundle_module">Target type \ 453 `android_app_bundle_module`</a>: 454 455 Corresponds to an Android app bundle module. Very similar to an APK and 456 inherits the same fields, except that this does not generate an installable 457 file (see `android_app_bundle`), and for the following omitted fields: 458 459 * `deps_info['apk_path']`, `deps_info['incremental_apk_path']` and 460 `deps_info['incremental_install_json_path']` are omitted. 461 462 * top-level `dist_jar` is omitted as well. 463 464 In addition to `android_apk` targets though come these new fields: 465 466 * `deps_info['proto_resources_path']`: 467 The path of an zip archive containing the APK's resources compiled to the 468 protocol buffer format (instead of regular binary xml + resources.arsc). 469 470 * `deps_info['module_rtxt_path']`: 471 The path of the R.txt file generated when compiling the resources for the bundle 472 module. 473 474 * `deps_info['module_pathmap_path']`: 475 The path of the pathmap file generated when compiling the resources for the 476 bundle module, if resource path shortening is enabled. 477 478 * `deps_info['base_allowlist_rtxt_path']`: 479 Optional path to an R.txt file used as a allowlist for base string resources. 480 This means that any string resource listed in this file *and* in 481 `deps_info['module_rtxt_path']` will end up in the base split APK of any 482 `android_app_bundle` target that uses this target as its base module. 483 484 This ensures that such localized strings are available to all bundle installs, 485 even when language based splits are enabled (e.g. required for WebView strings 486 inside the Monochrome bundle). 487 488 489 ## <a name="target_android_app_bundle">Target type `android_app_bundle`</a> 490 491 This target type corresponds to an Android app bundle, and is built from one 492 or more `android_app_bundle_module` targets listed as dependencies. 493 494 495 ## <a name="target_dist_aar">Target type `dist_aar`</a>: 496 497 This type corresponds to a target used to generate an `.aar` archive for 498 distribution. The archive's content is determined by the target's dependencies. 499 500 This always has the following entries: 501 502 * `deps_info['supports_android']` (always True). 503 * `deps_info['requires_android']` (always True). 504 * `deps_info['proguard_configs']` (optional). 505 506 507 ## <a name="target_dist_jar">Target type `dist_jar`</a>: 508 509 This type is similar to [`dist_aar`](#target_dist_aar) but is not 510 Android-specific, and used to create a `.jar` file that can be later 511 redistributed. 512 513 This always has the following entries: 514 515 * `deps_info['proguard_enabled']` (False by default). 516 * `deps_info['proguard_configs']` (optional). 517 * `deps_info['supports_android']` (True by default). 518 * `deps_info['requires_android']` (False by default). 519 520 521 522 ## <a name="dict_javac">The `deps_info['javac']` dictionary</a>: 523 524 This dictionary appears in Java-related targets (e.g. `java_library`, 525 `android_apk` and others), and contains information related to the compilation 526 of Java sources, class files, and jars. 527 528 * `javac['classpath']` 529 The classpath used to compile this target when annotation processors are 530 present. 531 532 * `javac['interface_classpath']` 533 The classpath used to compile this target when annotation processors are 534 not present. These are also always used to known when a target needs to be 535 rebuilt. 536 537 * `javac['processor_classpath']` 538 The classpath listing the jars used for annotation processors. I.e. sent as 539 `-processorpath` when invoking `javac`. 540 541 * `javac['processor_classes']` 542 The list of annotation processor main classes. I.e. sent as `-processor' when 543 invoking `javac`. 544 545 ## <a name="android_app_bundle">Target type `android_app_bundle`</a>: 546 547 This type corresponds to an Android app bundle (`.aab` file). 548 549 --------------- END_MARKDOWN --------------------------------------------------- 550 """ 551 552 import collections 553 import itertools 554 import json 555 import optparse 556 import os 557 import shutil 558 import sys 559 import xml.dom.minidom 560 561 from util import build_utils 562 from util import resource_utils 563 import action_helpers # build_utils adds //build to sys.path. 564 565 566 # Types that should never be used as a dependency of another build config. 567 _ROOT_TYPES = ('android_apk', 'java_binary', 'java_annotation_processor', 568 'robolectric_binary', 'android_app_bundle') 569 # Types that should not allow code deps to pass through. 570 _RESOURCE_TYPES = ('android_assets', 'android_resources', 'system_java_library') 571 572 # Cache of path -> JSON dict. 573 _dep_config_cache = {} 574 575 576 class OrderedSet(collections.OrderedDict): 577 @staticmethod 578 def fromkeys(iterable): 579 out = OrderedSet() 580 out.update(iterable) 581 return out 582 583 def add(self, key): 584 self[key] = True 585 586 def remove(self, key): 587 if key in self: 588 del self[key] 589 590 def update(self, iterable): 591 for v in iterable: 592 self.add(v) 593 594 def difference_update(self, iterable): 595 for v in iterable: 596 self.remove(v) 597 598 599 def _ExtractMarkdownDocumentation(input_text): 600 """Extract Markdown documentation from a list of input strings lines. 601 602 This generates a list of strings extracted from |input_text|, by looking 603 for '-- BEGIN_MARKDOWN --' and '-- END_MARKDOWN --' line markers.""" 604 in_markdown = False 605 result = [] 606 for line in input_text.splitlines(): 607 if in_markdown: 608 if '-- END_MARKDOWN --' in line: 609 in_markdown = False 610 else: 611 result.append(line) 612 else: 613 if '-- BEGIN_MARKDOWN --' in line: 614 in_markdown = True 615 616 return result 617 618 619 class AndroidManifest: 620 def __init__(self, path): 621 self.path = path 622 dom = xml.dom.minidom.parse(path) 623 manifests = dom.getElementsByTagName('manifest') 624 assert len(manifests) == 1 625 self.manifest = manifests[0] 626 627 def GetInstrumentationElements(self): 628 instrumentation_els = self.manifest.getElementsByTagName('instrumentation') 629 if len(instrumentation_els) == 0: 630 return None 631 return instrumentation_els 632 633 def CheckInstrumentationElements(self, expected_package): 634 instrs = self.GetInstrumentationElements() 635 if not instrs: 636 raise Exception('No <instrumentation> elements found in %s' % self.path) 637 for instr in instrs: 638 instrumented_package = instr.getAttributeNS( 639 'http://schemas.android.com/apk/res/android', 'targetPackage') 640 if instrumented_package != expected_package: 641 raise Exception( 642 'Wrong instrumented package. Expected %s, got %s' 643 % (expected_package, instrumented_package)) 644 645 def GetPackageName(self): 646 return self.manifest.getAttribute('package') 647 648 649 def GetDepConfigRoot(path): 650 if not path in _dep_config_cache: 651 with open(path) as jsonfile: 652 _dep_config_cache[path] = json.load(jsonfile) 653 return _dep_config_cache[path] 654 655 656 def GetDepConfig(path): 657 return GetDepConfigRoot(path)['deps_info'] 658 659 660 def DepsOfType(wanted_type, configs): 661 return [c for c in configs if c['type'] == wanted_type] 662 663 664 def DepPathsOfType(wanted_type, config_paths): 665 return [p for p in config_paths if GetDepConfig(p)['type'] == wanted_type] 666 667 668 def GetAllDepsConfigsInOrder(deps_config_paths, filter_func=None): 669 def apply_filter(paths): 670 if filter_func: 671 return [p for p in paths if filter_func(GetDepConfig(p))] 672 return paths 673 674 def discover(path): 675 config = GetDepConfig(path) 676 all_deps = config['deps_configs'] + config.get('public_deps_configs', []) 677 return apply_filter(all_deps) 678 679 deps_config_paths = apply_filter(deps_config_paths) 680 deps_config_paths = build_utils.GetSortedTransitiveDependencies( 681 deps_config_paths, discover) 682 return deps_config_paths 683 684 685 def GetObjectByPath(obj, key_path): 686 """Given an object, return its nth child based on a key path. 687 """ 688 return GetObjectByPath(obj[key_path[0]], key_path[1:]) if key_path else obj 689 690 691 def RemoveObjDups(obj, base, *key_path): 692 """Remove array items from an object[*kep_path] that are also 693 contained in the base[*kep_path] (duplicates). 694 """ 695 base_target = set(GetObjectByPath(base, key_path)) 696 target = GetObjectByPath(obj, key_path) 697 target[:] = [x for x in target if x not in base_target] 698 699 700 class Deps: 701 def __init__(self, direct_deps_config_paths): 702 self._all_deps_config_paths = GetAllDepsConfigsInOrder( 703 direct_deps_config_paths) 704 self._direct_deps_configs = [ 705 GetDepConfig(p) for p in direct_deps_config_paths 706 ] 707 self._all_deps_configs = [ 708 GetDepConfig(p) for p in self._all_deps_config_paths 709 ] 710 self._direct_deps_config_paths = direct_deps_config_paths 711 712 def All(self, wanted_type=None): 713 if wanted_type is None: 714 return self._all_deps_configs 715 return DepsOfType(wanted_type, self._all_deps_configs) 716 717 def Direct(self, wanted_type=None): 718 if wanted_type is None: 719 return self._direct_deps_configs 720 return DepsOfType(wanted_type, self._direct_deps_configs) 721 722 def AllConfigPaths(self): 723 return self._all_deps_config_paths 724 725 def GradlePrebuiltJarPaths(self): 726 ret = [] 727 728 def helper(cur): 729 for config in cur.Direct('java_library'): 730 if config['is_prebuilt'] or config['gradle_treat_as_prebuilt']: 731 if config['unprocessed_jar_path'] not in ret: 732 ret.append(config['unprocessed_jar_path']) 733 734 helper(self) 735 return ret 736 737 def GradleLibraryProjectDeps(self): 738 ret = [] 739 740 def helper(cur): 741 for config in cur.Direct('java_library'): 742 if config['is_prebuilt']: 743 pass 744 elif config['gradle_treat_as_prebuilt']: 745 all_deps = config['deps_configs'] + config.get( 746 'public_deps_configs', []) 747 helper(Deps(all_deps)) 748 elif config not in ret: 749 ret.append(config) 750 751 helper(self) 752 return ret 753 754 755 def _MergeAssets(all_assets): 756 """Merges all assets from the given deps. 757 758 Returns: 759 A tuple of: (compressed, uncompressed, locale_paks) 760 |compressed| and |uncompressed| are lists of "srcPath:zipPath". srcPath is 761 the path of the asset to add, and zipPath is the location within the zip 762 (excluding assets/ prefix). 763 |locale_paks| is a set of all zipPaths that have been marked as 764 treat_as_locale_paks=true. 765 """ 766 compressed = {} 767 uncompressed = {} 768 locale_paks = set() 769 for asset_dep in all_assets: 770 entry = asset_dep['assets'] 771 disable_compression = entry.get('disable_compression') 772 treat_as_locale_paks = entry.get('treat_as_locale_paks') 773 dest_map = uncompressed if disable_compression else compressed 774 other_map = compressed if disable_compression else uncompressed 775 outputs = entry.get('outputs', []) 776 for src, dest in itertools.zip_longest(entry['sources'], outputs): 777 if not dest: 778 dest = os.path.basename(src) 779 # Merge so that each path shows up in only one of the lists, and that 780 # deps of the same target override previous ones. 781 other_map.pop(dest, 0) 782 dest_map[dest] = src 783 if treat_as_locale_paks: 784 locale_paks.add(dest) 785 786 def create_list(asset_map): 787 # Sort to ensure deterministic ordering. 788 items = sorted(asset_map.items()) 789 return [f'{src}:{dest}' for dest, src in items] 790 791 return create_list(compressed), create_list(uncompressed), locale_paks 792 793 794 def _SuffixAssets(suffix_names, suffix, assets): 795 new_assets = [] 796 for x in assets: 797 src_path, apk_subpath = x.split(':', 1) 798 if apk_subpath in suffix_names: 799 apk_subpath += suffix 800 new_assets.append(f'{src_path}:{apk_subpath}') 801 return new_assets 802 803 804 def _ResolveGroupsAndPublicDeps(config_paths): 805 """Returns a list of configs with all groups inlined.""" 806 807 def helper(config_path): 808 config = GetDepConfig(config_path) 809 if config['type'] == 'group': 810 # Groups combine public_deps with deps_configs, so no need to check 811 # public_config_paths separately. 812 return config['deps_configs'] 813 if config['type'] == 'android_resources': 814 # android_resources targets do not support public_deps, but instead treat 815 # all resource deps as public deps. 816 return DepPathsOfType('android_resources', config['deps_configs']) 817 818 return config.get('public_deps_configs', []) 819 820 return build_utils.GetSortedTransitiveDependencies(config_paths, helper) 821 822 823 def _DepsFromPaths(dep_paths, 824 target_type, 825 filter_root_targets=True, 826 recursive_resource_deps=False): 827 """Resolves all groups and trims dependency branches that we never want. 828 829 E.g. When a resource or asset depends on an apk target, the intent is to 830 include the .apk as a resource/asset, not to have the apk's classpath added. 831 832 This method is meant to be called to get the top nodes (i.e. closest to 833 current target) that we could then use to get a full transitive dependants 834 list (eg using Deps#all). So filtering single elements out of this list, 835 filters whole branches of dependencies. By resolving groups (i.e. expanding 836 them to their constituents), depending on a group is equivalent to directly 837 depending on each element of that group. 838 """ 839 blocklist = [] 840 allowlist = [] 841 842 # Don't allow root targets to be considered as a dep. 843 if filter_root_targets: 844 blocklist.extend(_ROOT_TYPES) 845 846 # Don't allow java libraries to cross through assets/resources. 847 if target_type in _RESOURCE_TYPES: 848 allowlist.extend(_RESOURCE_TYPES) 849 # Pretend that this target directly depends on all of its transitive 850 # dependencies. 851 if recursive_resource_deps: 852 dep_paths = GetAllDepsConfigsInOrder(dep_paths) 853 # Exclude assets if recursive_resource_deps is set. The 854 # recursive_resource_deps arg is used to pull resources into the base 855 # module to workaround bugs accessing resources in isolated DFMs, but 856 # assets should be kept in the DFMs. 857 blocklist.append('android_assets') 858 859 return _DepsFromPathsWithFilters(dep_paths, blocklist, allowlist) 860 861 862 def _FilterConfigPaths(dep_paths, blocklist=None, allowlist=None): 863 if not blocklist and not allowlist: 864 return dep_paths 865 configs = [GetDepConfig(p) for p in dep_paths] 866 if blocklist: 867 configs = [c for c in configs if c['type'] not in blocklist] 868 if allowlist: 869 configs = [c for c in configs if c['type'] in allowlist] 870 871 return [c['path'] for c in configs] 872 873 874 def _DepsFromPathsWithFilters(dep_paths, blocklist=None, allowlist=None): 875 """Resolves all groups and trims dependency branches that we never want. 876 877 See _DepsFromPaths. 878 879 |blocklist| if passed, are the types of direct dependencies we do not care 880 about (i.e. tips of branches that we wish to prune). 881 882 |allowlist| if passed, are the only types of direct dependencies we care 883 about (i.e. we wish to prune all other branches that do not start from one of 884 these). 885 """ 886 # Filter both before and after so that public_deps of blocked targets are not 887 # added. 888 allowlist_with_groups = None 889 if allowlist: 890 allowlist_with_groups = set(allowlist) 891 allowlist_with_groups.add('group') 892 dep_paths = _FilterConfigPaths(dep_paths, blocklist, allowlist_with_groups) 893 dep_paths = _ResolveGroupsAndPublicDeps(dep_paths) 894 dep_paths = _FilterConfigPaths(dep_paths, blocklist, allowlist) 895 896 return Deps(dep_paths) 897 898 899 def _ExtractSharedLibsFromRuntimeDeps(runtime_deps_file): 900 ret = [] 901 with open(runtime_deps_file) as f: 902 for line in f: 903 line = line.rstrip() 904 if not line.endswith('.so'): 905 continue 906 # Only unstripped .so files are listed in runtime deps. 907 # Convert to the stripped .so by going up one directory. 908 ret.append(os.path.normpath(line.replace('lib.unstripped/', ''))) 909 ret.reverse() 910 return ret 911 912 913 def _CreateJavaLibrariesList(library_paths): 914 """Returns a java literal array with the "base" library names: 915 e.g. libfoo.so -> foo 916 """ 917 names = ['"%s"' % os.path.basename(s)[3:-3] for s in library_paths] 918 return ('{%s}' % ','.join(sorted(set(names)))) 919 920 921 def _CreateJavaLocaleListFromAssets(assets, locale_paks): 922 """Returns a java literal array from a list of locale assets. 923 924 Args: 925 assets: A list of all APK asset paths in the form 'src:dst' 926 locale_paks: A list of asset paths that correponds to the locale pak 927 files of interest. Each |assets| entry will have its 'dst' part matched 928 against it to determine if they are part of the result. 929 Returns: 930 A string that is a Java source literal array listing the locale names 931 of the corresponding asset files, without directory or .pak suffix. 932 E.g. '{"en-GB", "en-US", "es-ES", "fr", ... }' 933 """ 934 assets_paths = [a.split(':')[1] for a in assets] 935 locales = [os.path.basename(a)[:-4] for a in assets_paths if a in locale_paks] 936 return '{%s}' % ','.join('"%s"' % l for l in sorted(locales)) 937 938 939 def _AddJarMapping(jar_to_target, configs): 940 for config in configs: 941 jar = config.get('unprocessed_jar_path') 942 if jar: 943 jar_to_target[jar] = config['gn_target'] 944 for jar in config.get('extra_classpath_jars', []): 945 jar_to_target[jar] = config['gn_target'] 946 947 948 def _CompareClasspathPriority(dep): 949 return 1 if dep.get('low_classpath_priority') else 0 950 951 952 def _DedupFeatureModuleSharedCode(uses_split_arg, modules, 953 field_names_to_dedup): 954 child_to_ancestors = collections.defaultdict(list) 955 if uses_split_arg: 956 for split_pair in uses_split_arg: 957 child, parent = split_pair.split(':') 958 assert child in modules 959 assert parent in modules 960 child_to_ancestors[child] = [parent] 961 962 # Create a full list of ancestors for each module. 963 for name in modules: 964 if name == 'base': 965 continue 966 curr_name = name 967 while curr_name in child_to_ancestors: 968 parent = child_to_ancestors[curr_name][0] 969 if parent not in child_to_ancestors[name]: 970 child_to_ancestors[name].append(parent) 971 curr_name = parent 972 973 if curr_name != 'base': 974 child_to_ancestors[name].append('base') 975 976 # Strip out duplicates from ancestors. 977 for name, module in modules.items(): 978 if name == 'base': 979 continue 980 # Make sure we get all ancestors, not just direct parent. 981 for ancestor in child_to_ancestors[name]: 982 for f in field_names_to_dedup: 983 if f in module: 984 RemoveObjDups(module, modules[ancestor], f) 985 986 # Strip out duplicates from siblings/cousins. 987 for f in field_names_to_dedup: 988 _PromoteToCommonAncestor(modules, child_to_ancestors, f) 989 990 991 def _PromoteToCommonAncestor(modules, child_to_ancestors, field_name): 992 module_to_fields_set = {} 993 for module_name, module in modules.items(): 994 if field_name in module: 995 module_to_fields_set[module_name] = set(module[field_name]) 996 997 seen = set() 998 dupes = set() 999 for fields in module_to_fields_set.values(): 1000 new_dupes = seen & fields 1001 if new_dupes: 1002 dupes |= new_dupes 1003 seen |= fields 1004 1005 for d in dupes: 1006 owning_modules = [] 1007 for module_name, fields in module_to_fields_set.items(): 1008 if d in fields: 1009 owning_modules.append(module_name) 1010 assert len(owning_modules) >= 2 1011 # Rely on the fact that ancestors are inserted from closest to 1012 # farthest, where "base" should always be the last element. 1013 # Arbitrarily using the first owning module - any would work. 1014 for ancestor in child_to_ancestors[owning_modules[0]]: 1015 ancestor_is_shared_with_all = True 1016 for o in owning_modules[1:]: 1017 if ancestor not in child_to_ancestors[o]: 1018 ancestor_is_shared_with_all = False 1019 break 1020 if ancestor_is_shared_with_all: 1021 common_ancestor = ancestor 1022 break 1023 for o in owning_modules: 1024 module_to_fields_set[o].remove(d) 1025 module_to_fields_set[common_ancestor].add(d) 1026 1027 for module_name, module in modules.items(): 1028 if field_name in module: 1029 module[field_name] = sorted(list(module_to_fields_set[module_name])) 1030 1031 1032 def _CopyBuildConfigsForDebugging(debug_dir): 1033 shutil.rmtree(debug_dir, ignore_errors=True) 1034 os.makedirs(debug_dir) 1035 for src_path in _dep_config_cache: 1036 dst_path = os.path.join(debug_dir, src_path) 1037 assert dst_path.startswith(debug_dir), dst_path 1038 os.makedirs(os.path.dirname(dst_path), exist_ok=True) 1039 shutil.copy(src_path, dst_path) 1040 print(f'Copied {len(_dep_config_cache)} .build_config.json into {debug_dir}') 1041 1042 1043 def main(argv): 1044 parser = optparse.OptionParser() 1045 action_helpers.add_depfile_arg(parser) 1046 parser.add_option('--build-config', help='Path to build_config output.') 1047 parser.add_option('--store-deps-for-debugging-to', 1048 help='Path to copy all transitive build config files to.') 1049 parser.add_option( 1050 '--type', 1051 help='Type of this target (e.g. android_library).') 1052 parser.add_option('--gn-target', help='GN label for this target') 1053 parser.add_option( 1054 '--deps-configs', 1055 help='GN-list of dependent build_config files.') 1056 parser.add_option( 1057 '--annotation-processor-configs', 1058 help='GN-list of build_config files for annotation processors.') 1059 1060 # android_resources options 1061 parser.add_option('--srcjar', help='Path to target\'s resources srcjar.') 1062 parser.add_option('--resources-zip', help='Path to target\'s resources zip.') 1063 parser.add_option('--package-name', 1064 help='Java package name for these resources.') 1065 parser.add_option('--android-manifest', 1066 help='Path to the root android manifest.') 1067 parser.add_option('--merged-android-manifest', 1068 help='Path to the merged android manifest.') 1069 parser.add_option('--resource-dirs', action='append', default=[], 1070 help='GYP-list of resource dirs') 1071 parser.add_option( 1072 '--res-sources-path', 1073 help='Path to file containing a list of paths to resources.') 1074 parser.add_option( 1075 '--resource-overlay', 1076 action='store_true', 1077 help='Whether resources passed in via --resources-zip should override ' 1078 'resources with the same name') 1079 parser.add_option( 1080 '--recursive-resource-deps', 1081 action='store_true', 1082 help='Whether deps should be walked recursively to find resource deps.') 1083 1084 # android_assets options 1085 parser.add_option('--asset-sources', help='List of asset sources.') 1086 parser.add_option('--asset-renaming-sources', 1087 help='List of asset sources with custom destinations.') 1088 parser.add_option('--asset-renaming-destinations', 1089 help='List of asset custom destinations.') 1090 parser.add_option('--disable-asset-compression', action='store_true', 1091 help='Whether to disable asset compression.') 1092 parser.add_option('--treat-as-locale-paks', action='store_true', 1093 help='Consider the assets as locale paks in BuildConfig.java') 1094 1095 # java library and group options 1096 parser.add_option('--preferred-dep', 1097 action='store_true', 1098 help='Whether the target should be preferred as a dep.') 1099 1100 # java library options 1101 parser.add_option('--public-deps-configs', 1102 help='GN list of config files of deps which are exposed as ' 1103 'part of the target\'s public API.') 1104 parser.add_option('--aar-path', help='Path to containing .aar file.') 1105 parser.add_option('--device-jar-path', help='Path to .jar for dexing.') 1106 parser.add_option('--host-jar-path', help='Path to .jar for java_binary.') 1107 parser.add_option('--unprocessed-jar-path', 1108 help='Path to the .jar to use for javac classpath purposes.') 1109 parser.add_option( 1110 '--interface-jar-path', 1111 help='Path to the interface .jar to use for javac classpath purposes.') 1112 parser.add_option('--is-prebuilt', action='store_true', 1113 help='Whether the jar was compiled or pre-compiled.') 1114 parser.add_option('--target-sources-file', help='Path to .sources file') 1115 parser.add_option('--bundled-srcjars', 1116 help='GYP-list of .srcjars that have been included in this java_library.') 1117 parser.add_option('--supports-android', action='store_true', 1118 help='Whether this library supports running on the Android platform.') 1119 parser.add_option('--requires-android', action='store_true', 1120 help='Whether this library requires running on the Android platform.') 1121 parser.add_option('--bypass-platform-checks', action='store_true', 1122 help='Bypass checks for support/require Android platform.') 1123 parser.add_option('--extra-classpath-jars', 1124 help='GYP-list of .jar files to include on the classpath when compiling, ' 1125 'but not to include in the final binary.') 1126 parser.add_option( 1127 '--low-classpath-priority', 1128 action='store_true', 1129 help='Indicates that the library should be placed at the end of the ' 1130 'classpath.') 1131 parser.add_option( 1132 '--mergeable-android-manifests', 1133 help='GN-list of AndroidManifest.xml to include in manifest merging.') 1134 parser.add_option('--gradle-treat-as-prebuilt', action='store_true', 1135 help='Whether this library should be treated as a prebuilt library by ' 1136 'generate_gradle.py.') 1137 parser.add_option('--main-class', 1138 help='Main class for java_binary or java_annotation_processor targets.') 1139 parser.add_option('--java-resources-jar-path', 1140 help='Path to JAR that contains java resources. Everything ' 1141 'from this JAR except meta-inf/ content and .class files ' 1142 'will be added to the final APK.') 1143 parser.add_option( 1144 '--non-chromium-code', 1145 action='store_true', 1146 help='True if a java library is not chromium code, used for lint.') 1147 1148 # robolectric_library options 1149 parser.add_option('--is-robolectric', 1150 action='store_true', 1151 help='Whether this is a host side android test library.') 1152 1153 # android library options 1154 parser.add_option('--dex-path', help='Path to target\'s dex output.') 1155 1156 # native library options 1157 parser.add_option('--shared-libraries-runtime-deps', 1158 help='Path to file containing runtime deps for shared ' 1159 'libraries.') 1160 parser.add_option( 1161 '--loadable-modules', 1162 action='append', 1163 help='GN-list of native libraries for primary ' 1164 'android-abi. Can be specified multiple times.', 1165 default=[]) 1166 parser.add_option('--secondary-abi-shared-libraries-runtime-deps', 1167 help='Path to file containing runtime deps for secondary ' 1168 'abi shared libraries.') 1169 parser.add_option( 1170 '--secondary-abi-loadable-modules', 1171 action='append', 1172 help='GN-list of native libraries for secondary ' 1173 'android-abi. Can be specified multiple times.', 1174 default=[]) 1175 parser.add_option( 1176 '--native-lib-placeholders', 1177 action='append', 1178 help='GN-list of native library placeholders to add.', 1179 default=[]) 1180 parser.add_option( 1181 '--secondary-native-lib-placeholders', 1182 action='append', 1183 help='GN-list of native library placeholders to add ' 1184 'for the secondary android-abi.', 1185 default=[]) 1186 parser.add_option('--uncompress-shared-libraries', default=False, 1187 action='store_true', 1188 help='Whether to store native libraries uncompressed') 1189 parser.add_option( 1190 '--library-always-compress', 1191 help='The list of library files that we always compress.') 1192 1193 # apk options 1194 parser.add_option('--apk-path', help='Path to the target\'s apk output.') 1195 parser.add_option('--incremental-apk-path', 1196 help="Path to the target's incremental apk output.") 1197 parser.add_option('--incremental-install-json-path', 1198 help="Path to the target's generated incremental install " 1199 "json.") 1200 parser.add_option( 1201 '--suffix-apk-assets-used-by', 1202 help='Path to the build config of the apk whose asset list should be ' 1203 'suffixed.') 1204 parser.add_option( 1205 '--tested-apk-config', 1206 help='Path to the build config of the tested apk (for an instrumentation ' 1207 'test apk).') 1208 parser.add_option( 1209 '--proguard-enabled', 1210 action='store_true', 1211 help='Whether proguard is enabled for this apk or bundle module.') 1212 parser.add_option( 1213 '--proguard-configs', 1214 help='GN-list of proguard flag files to use in final apk.') 1215 parser.add_option( 1216 '--proguard-mapping-path', help='Path to jar created by ProGuard step') 1217 1218 # apk options that are static library specific 1219 parser.add_option( 1220 '--static-library-dependent-configs', 1221 help='GN list of .build_configs of targets that use this target as a ' 1222 'static library.') 1223 1224 # options shared between android_resources and apk targets 1225 parser.add_option('--r-text-path', help='Path to target\'s R.txt file.') 1226 1227 parser.add_option('--fail', 1228 help='GN-list of error message lines to fail with.') 1229 1230 parser.add_option('--final-dex-path', 1231 help='Path to final input classes.dex (or classes.zip) to ' 1232 'use in final apk.') 1233 parser.add_option('--res-size-info', help='Path to .ap_.info') 1234 parser.add_option('--apk-proto-resources', 1235 help='Path to resources compiled in protocol buffer format ' 1236 ' for this apk.') 1237 parser.add_option( 1238 '--module-pathmap-path', 1239 help='Path to pathmap file for resource paths in a bundle module.') 1240 parser.add_option( 1241 '--base-allowlist-rtxt-path', 1242 help='Path to R.txt file for the base resources allowlist.') 1243 1244 parser.add_option('--generate-markdown-format-doc', action='store_true', 1245 help='Dump the Markdown .build_config format documentation ' 1246 'then exit immediately.') 1247 1248 parser.add_option('--module-name', help='The name of this feature module.') 1249 parser.add_option( 1250 '--base-module-build-config', 1251 help='Path to the base module\'s build config ' 1252 'if this is a feature module.') 1253 parser.add_option('--parent-module-build-config', 1254 help='Path to the parent module\'s build config ' 1255 'when not using base module as parent.') 1256 1257 parser.add_option( 1258 '--module-build-configs', 1259 help='For bundles, the paths of all non-async module .build_configs ' 1260 'for modules that are part of the bundle.') 1261 parser.add_option( 1262 '--uses-split', 1263 action='append', 1264 help='List of name pairs separated by : mapping a feature module to a ' 1265 'dependent feature module.') 1266 1267 parser.add_option( 1268 '--trace-events-jar-dir', 1269 help='Directory of rewritten .jar files for trace event rewriting.') 1270 1271 parser.add_option('--version-name', help='Version name for this APK.') 1272 parser.add_option('--version-code', help='Version code for this APK.') 1273 1274 # dist_jar options 1275 parser.add_option('--use-interface-jars', action='store_true') 1276 parser.add_option('--direct-deps-only', action='store_true') 1277 1278 options, args = parser.parse_args(argv) 1279 1280 if args: 1281 parser.error('No positional arguments should be given.') 1282 1283 if options.generate_markdown_format_doc: 1284 doc_lines = _ExtractMarkdownDocumentation(__doc__) 1285 for line in doc_lines: 1286 print(line) 1287 return 0 1288 1289 if options.fail: 1290 parser.error('\n'.join(action_helpers.parse_gn_list(options.fail))) 1291 1292 lib_options = ['unprocessed_jar_path', 'interface_jar_path'] 1293 device_lib_options = ['device_jar_path', 'dex_path'] 1294 required_options_map = { 1295 'android_apk': ['build_config'] + lib_options + device_lib_options, 1296 'android_app_bundle_module': 1297 ['build_config', 'res_size_info'] + lib_options + device_lib_options, 1298 'android_assets': ['build_config'], 1299 'android_resources': ['build_config', 'resources_zip'], 1300 'dist_aar': ['build_config'], 1301 'dist_jar': ['build_config'], 1302 'group': ['build_config'], 1303 'java_annotation_processor': ['build_config', 'main_class'], 1304 'java_binary': ['build_config'], 1305 'java_library': ['build_config', 'host_jar_path'] + lib_options, 1306 'robolectric_binary': ['build_config'], 1307 'system_java_library': ['build_config', 'unprocessed_jar_path'], 1308 'android_app_bundle': ['build_config', 'module_build_configs'], 1309 } 1310 required_options = required_options_map.get(options.type) 1311 if not required_options: 1312 raise Exception('Unknown type: <%s>' % options.type) 1313 1314 build_utils.CheckOptions(options, parser, required_options) 1315 1316 if options.type != 'android_app_bundle_module': 1317 if options.apk_proto_resources: 1318 raise Exception('--apk-proto-resources can only be used with ' 1319 '--type=android_app_bundle_module') 1320 if options.module_pathmap_path: 1321 raise Exception('--module-pathmap-path can only be used with ' 1322 '--type=android_app_bundle_module') 1323 if options.base_allowlist_rtxt_path: 1324 raise Exception('--base-allowlist-rtxt-path can only be used with ' 1325 '--type=android_app_bundle_module') 1326 if options.module_name: 1327 raise Exception('--module-name can only be used with ' 1328 '--type=android_app_bundle_module') 1329 1330 is_apk_or_module_target = options.type in ('android_apk', 1331 'android_app_bundle_module') 1332 1333 if not is_apk_or_module_target: 1334 if options.library_always_compress: 1335 raise Exception( 1336 '--library-always-compress can only be used with --type=android_apk ' 1337 'or --type=android_app_bundle_module') 1338 1339 if options.device_jar_path and not options.dex_path: 1340 raise Exception('java_library that supports Android requires a dex path.') 1341 if any(getattr(options, x) for x in lib_options): 1342 for attr in lib_options: 1343 if not getattr(options, attr): 1344 raise('Expected %s to be set.' % attr) 1345 1346 if options.requires_android and not options.supports_android: 1347 raise Exception( 1348 '--supports-android is required when using --requires-android') 1349 1350 is_java_target = options.type in ('java_binary', 'robolectric_binary', 1351 'java_annotation_processor', 'java_library', 1352 'android_apk', 'dist_aar', 'dist_jar', 1353 'system_java_library', 1354 'android_app_bundle_module') 1355 1356 deps_configs_paths = action_helpers.parse_gn_list(options.deps_configs) 1357 public_deps_configs_paths = action_helpers.parse_gn_list( 1358 options.public_deps_configs) 1359 deps_configs_paths += public_deps_configs_paths 1360 deps = _DepsFromPaths(deps_configs_paths, 1361 options.type, 1362 recursive_resource_deps=options.recursive_resource_deps) 1363 public_deps = _DepsFromPaths(public_deps_configs_paths, options.type) 1364 processor_deps = _DepsFromPaths(action_helpers.parse_gn_list( 1365 options.annotation_processor_configs or ''), 1366 options.type, 1367 filter_root_targets=False) 1368 1369 all_inputs = (deps.AllConfigPaths() + processor_deps.AllConfigPaths()) 1370 1371 if options.recursive_resource_deps: 1372 # Include java_library targets since changes to these targets can remove 1373 # resource deps from the build, which would require rebuilding this target's 1374 # build config file: crbug.com/1168655. 1375 recursive_java_deps = _DepsFromPathsWithFilters( 1376 GetAllDepsConfigsInOrder(deps_configs_paths), 1377 allowlist=['java_library']) 1378 all_inputs.extend(recursive_java_deps.AllConfigPaths()) 1379 1380 system_library_deps = deps.Direct('system_java_library') 1381 all_deps = deps.All() 1382 all_library_deps = deps.All('java_library') 1383 1384 direct_resources_deps = deps.Direct('android_resources') 1385 if options.type == 'java_library': 1386 # For Java libraries, restrict to resource targets that are direct deps, or 1387 # are indirect via other resource targets. 1388 # The indirect-through-other-targets ones are picked up because 1389 # _ResolveGroupsAndPublicDeps() treats resource deps of resource targets as 1390 # public_deps. 1391 all_resources_deps = direct_resources_deps 1392 else: 1393 all_resources_deps = deps.All('android_resources') 1394 1395 if options.type == 'android_resources' and options.recursive_resource_deps: 1396 # android_resources targets that want recursive resource deps also need to 1397 # collect package_names from all library deps. This ensures the R.java files 1398 # for these libraries will get pulled in along with the resources. 1399 android_resources_library_deps = _DepsFromPathsWithFilters( 1400 deps_configs_paths, allowlist=['java_library']).All('java_library') 1401 1402 base_module_build_config = None 1403 if options.base_module_build_config: 1404 base_module_build_config = GetDepConfigRoot( 1405 options.base_module_build_config) 1406 parent_module_build_config = base_module_build_config 1407 if options.parent_module_build_config: 1408 parent_module_build_config = GetDepConfigRoot( 1409 options.parent_module_build_config) 1410 1411 # Initialize some common config. 1412 # Any value that needs to be queryable by dependents must go within deps_info. 1413 config = { 1414 'deps_info': { 1415 'name': os.path.basename(options.build_config), 1416 'path': options.build_config, 1417 'type': options.type, 1418 'gn_target': options.gn_target, 1419 'chromium_code': not options.non_chromium_code, 1420 }, 1421 # Info needed only by generate_gradle.py. 1422 'gradle': {} 1423 } 1424 deps_info = config['deps_info'] 1425 gradle = config['gradle'] 1426 1427 # The paths we record as deps can differ from deps_config_paths: 1428 # 1) Paths can be removed when blocked by _ROOT_TYPES / _RESOURCE_TYPES. 1429 # 2) Paths can be added when promoted from group deps or public_deps of deps. 1430 # Deps are promoted from groups/public_deps in order to make the filtering 1431 # of 1) work through group() targets (which themselves are not resource 1432 # targets, but should be treated as such when depended on by a resource 1433 # target. A more involved filtering implementation could work to maintain 1434 # the semantics of 1) without the need to promote deps, but we've avoided 1435 # such an undertaking so far. 1436 public_deps_set = set() 1437 if public_deps_configs_paths: 1438 deps_info['public_deps_configs'] = [d['path'] for d in public_deps.Direct()] 1439 public_deps_set = set(deps_info['public_deps_configs']) 1440 1441 deps_info['deps_configs'] = [ 1442 d['path'] for d in deps.Direct() if d['path'] not in public_deps_set 1443 ] 1444 1445 if options.type == 'android_apk' and options.tested_apk_config: 1446 tested_apk_deps = Deps([options.tested_apk_config]) 1447 tested_apk_config = tested_apk_deps.Direct()[0] 1448 gradle['apk_under_test'] = tested_apk_config['name'] 1449 1450 if options.type == 'android_app_bundle_module': 1451 deps_info['module_name'] = options.module_name 1452 1453 # Required for generating gradle files. 1454 if options.type == 'java_library': 1455 deps_info['is_prebuilt'] = bool(options.is_prebuilt) 1456 deps_info['gradle_treat_as_prebuilt'] = options.gradle_treat_as_prebuilt 1457 1458 if options.preferred_dep: 1459 deps_info['preferred_dep'] = bool(options.preferred_dep) 1460 1461 if options.android_manifest: 1462 deps_info['android_manifest'] = options.android_manifest 1463 1464 if options.merged_android_manifest: 1465 deps_info['merged_android_manifest'] = options.merged_android_manifest 1466 1467 if options.bundled_srcjars: 1468 deps_info['bundled_srcjars'] = action_helpers.parse_gn_list( 1469 options.bundled_srcjars) 1470 1471 if options.target_sources_file: 1472 deps_info['target_sources_file'] = options.target_sources_file 1473 1474 if is_java_target: 1475 if options.main_class: 1476 deps_info['main_class'] = options.main_class 1477 1478 dependent_prebuilt_jars = deps.GradlePrebuiltJarPaths() 1479 dependent_prebuilt_jars.sort() 1480 if dependent_prebuilt_jars: 1481 gradle['dependent_prebuilt_jars'] = dependent_prebuilt_jars 1482 1483 dependent_android_projects = [] 1484 dependent_java_projects = [] 1485 for c in deps.GradleLibraryProjectDeps(): 1486 if c['requires_android']: 1487 dependent_android_projects.append(c['path']) 1488 else: 1489 dependent_java_projects.append(c['path']) 1490 1491 gradle['dependent_android_projects'] = dependent_android_projects 1492 gradle['dependent_java_projects'] = dependent_java_projects 1493 1494 if options.r_text_path: 1495 deps_info['r_text_path'] = options.r_text_path 1496 1497 if is_apk_or_module_target or options.type in ('group', 'java_library', 1498 'robolectric_binary'): 1499 if options.apk_proto_resources: 1500 deps_info['proto_resources_path'] = options.apk_proto_resources 1501 1502 deps_info['version_name'] = options.version_name 1503 deps_info['version_code'] = options.version_code 1504 if options.module_pathmap_path: 1505 deps_info['module_pathmap_path'] = options.module_pathmap_path 1506 else: 1507 # Ensure there is an entry, even if it is empty, for modules 1508 # that have not enabled resource path shortening. Otherwise 1509 # build_utils.ExpandFileArgs fails. 1510 deps_info['module_pathmap_path'] = '' 1511 1512 if options.base_allowlist_rtxt_path: 1513 deps_info['base_allowlist_rtxt_path'] = options.base_allowlist_rtxt_path 1514 else: 1515 # Ensure there is an entry, even if it is empty, for modules 1516 # that don't need such a allowlist. 1517 deps_info['base_allowlist_rtxt_path'] = '' 1518 1519 if is_java_target: 1520 deps_info['requires_android'] = bool(options.requires_android) 1521 deps_info['supports_android'] = bool(options.supports_android) 1522 1523 # robolectric is special in that its an android target that runs on host. 1524 # You are allowed to depend on both android |deps_require_android| and 1525 # non-android |deps_not_support_android| targets. 1526 if not options.bypass_platform_checks and not options.is_robolectric: 1527 deps_require_android = direct_resources_deps + [ 1528 d for d in deps.Direct() if d.get('requires_android', False) 1529 ] 1530 deps_not_support_android = [ 1531 d for d in deps.Direct() if not d.get('supports_android', True) 1532 ] 1533 1534 if deps_require_android and not options.requires_android: 1535 raise Exception( 1536 'Some deps require building for the Android platform:\n' + 1537 '\n'.join('* ' + d['gn_target'] for d in deps_require_android)) 1538 1539 if deps_not_support_android and options.supports_android: 1540 raise Exception('Not all deps support the Android platform:\n' + 1541 '\n'.join('* ' + d['gn_target'] 1542 for d in deps_not_support_android)) 1543 1544 if is_apk_or_module_target or options.type == 'dist_jar': 1545 all_dex_files = [c['dex_path'] for c in all_library_deps] 1546 1547 if is_java_target: 1548 # Classpath values filled in below (after applying tested_apk_config). 1549 config['javac'] = {} 1550 if options.aar_path: 1551 deps_info['aar_path'] = options.aar_path 1552 if options.unprocessed_jar_path: 1553 deps_info['unprocessed_jar_path'] = options.unprocessed_jar_path 1554 deps_info['interface_jar_path'] = options.interface_jar_path 1555 if options.device_jar_path: 1556 deps_info['device_jar_path'] = options.device_jar_path 1557 if options.host_jar_path: 1558 deps_info['host_jar_path'] = options.host_jar_path 1559 if options.dex_path: 1560 deps_info['dex_path'] = options.dex_path 1561 if is_apk_or_module_target: 1562 all_dex_files.append(options.dex_path) 1563 if options.low_classpath_priority: 1564 deps_info['low_classpath_priority'] = True 1565 if options.type == 'android_apk': 1566 deps_info['apk_path'] = options.apk_path 1567 deps_info['incremental_apk_path'] = options.incremental_apk_path 1568 deps_info['incremental_install_json_path'] = ( 1569 options.incremental_install_json_path) 1570 1571 if options.type == 'android_assets': 1572 all_asset_sources = [] 1573 if options.asset_renaming_sources: 1574 all_asset_sources.extend( 1575 action_helpers.parse_gn_list(options.asset_renaming_sources)) 1576 if options.asset_sources: 1577 all_asset_sources.extend( 1578 action_helpers.parse_gn_list(options.asset_sources)) 1579 1580 deps_info['assets'] = { 1581 'sources': all_asset_sources 1582 } 1583 if options.asset_renaming_destinations: 1584 deps_info['assets']['outputs'] = (action_helpers.parse_gn_list( 1585 options.asset_renaming_destinations)) 1586 if options.disable_asset_compression: 1587 deps_info['assets']['disable_compression'] = True 1588 if options.treat_as_locale_paks: 1589 deps_info['assets']['treat_as_locale_paks'] = True 1590 1591 if options.type == 'android_resources': 1592 deps_info['resources_zip'] = options.resources_zip 1593 if options.resource_overlay: 1594 deps_info['resource_overlay'] = True 1595 if options.srcjar: 1596 deps_info['srcjar'] = options.srcjar 1597 if options.android_manifest: 1598 manifest = AndroidManifest(options.android_manifest) 1599 deps_info['package_name'] = manifest.GetPackageName() 1600 if options.package_name: 1601 deps_info['package_name'] = options.package_name 1602 deps_info['res_sources_path'] = '' 1603 if options.res_sources_path: 1604 deps_info['res_sources_path'] = options.res_sources_path 1605 1606 if (options.requires_android 1607 and options.type == 'java_library') or options.is_robolectric: 1608 if options.package_name: 1609 deps_info['package_name'] = options.package_name 1610 1611 if options.type in ('android_resources', 'android_apk', 'robolectric_binary', 1612 'dist_aar', 'android_app_bundle_module', 'java_library'): 1613 dependency_zips = [] 1614 dependency_zip_overlays = [] 1615 for c in all_resources_deps: 1616 if not c['resources_zip']: 1617 continue 1618 1619 dependency_zips.append(c['resources_zip']) 1620 if c.get('resource_overlay'): 1621 dependency_zip_overlays.append(c['resources_zip']) 1622 1623 extra_package_names = [] 1624 1625 if options.type != 'android_resources': 1626 extra_package_names = [ 1627 c['package_name'] for c in all_resources_deps if 'package_name' in c 1628 ] 1629 if options.package_name: 1630 extra_package_names += [options.package_name] 1631 1632 # android_resources targets which specified recursive_resource_deps may 1633 # have extra_package_names. 1634 for resources_dep in all_resources_deps: 1635 extra_package_names.extend(resources_dep['extra_package_names']) 1636 1637 # In final types (i.e. apks and modules) that create real R.java files, 1638 # they must collect package names from java_libraries as well. 1639 # https://crbug.com/1073476 1640 if options.type != 'java_library': 1641 extra_package_names.extend([ 1642 c['package_name'] for c in all_library_deps if 'package_name' in c 1643 ]) 1644 elif options.recursive_resource_deps: 1645 # Pull extra_package_names from library deps if recursive resource deps 1646 # are required. 1647 extra_package_names = [ 1648 c['package_name'] for c in android_resources_library_deps 1649 if 'package_name' in c 1650 ] 1651 config['deps_info']['includes_recursive_resources'] = True 1652 1653 if options.type in ('dist_aar', 'java_library'): 1654 r_text_files = [ 1655 c['r_text_path'] for c in all_resources_deps if 'r_text_path' in c 1656 ] 1657 deps_info['dependency_r_txt_files'] = r_text_files 1658 1659 if options.type == 'android_apk' and options.tested_apk_config: 1660 config['deps_info']['arsc_package_name'] = ( 1661 tested_apk_config['package_name']) 1662 # We should not shadow the actual R.java files of the apk_under_test by 1663 # creating new R.java files with the same package names in the tested apk. 1664 extra_package_names = [ 1665 package for package in extra_package_names 1666 if package not in tested_apk_config['extra_package_names'] 1667 ] 1668 if options.res_size_info: 1669 config['deps_info']['res_size_info'] = options.res_size_info 1670 1671 # Safe to sort: Build checks that non-overlay resource have no overlap. 1672 dependency_zips.sort() 1673 config['deps_info']['dependency_zips'] = dependency_zips 1674 config['deps_info']['dependency_zip_overlays'] = dependency_zip_overlays 1675 # Order doesn't matter, so make stable. 1676 extra_package_names.sort() 1677 config['deps_info']['extra_package_names'] = extra_package_names 1678 1679 # These are .jars to add to javac classpath but not to runtime classpath. 1680 extra_classpath_jars = action_helpers.parse_gn_list( 1681 options.extra_classpath_jars) 1682 if extra_classpath_jars: 1683 extra_classpath_jars.sort() 1684 deps_info['extra_classpath_jars'] = extra_classpath_jars 1685 1686 mergeable_android_manifests = action_helpers.parse_gn_list( 1687 options.mergeable_android_manifests) 1688 mergeable_android_manifests.sort() 1689 if mergeable_android_manifests: 1690 deps_info['mergeable_android_manifests'] = mergeable_android_manifests 1691 1692 extra_proguard_classpath_jars = [] 1693 proguard_configs = action_helpers.parse_gn_list(options.proguard_configs) 1694 if proguard_configs: 1695 # Make a copy of |proguard_configs| since it's mutated below. 1696 deps_info['proguard_configs'] = list(proguard_configs) 1697 1698 1699 if is_java_target: 1700 classpath_direct_deps = deps.Direct() 1701 classpath_direct_library_deps = deps.Direct('java_library') 1702 1703 # The classpath used to compile this target when annotation processors are 1704 # present. 1705 javac_classpath = set(c['unprocessed_jar_path'] 1706 for c in classpath_direct_library_deps) 1707 # The classpath used to compile this target when annotation processors are 1708 # not present. These are also always used to know when a target needs to be 1709 # rebuilt. 1710 javac_interface_classpath = set(c['interface_jar_path'] 1711 for c in classpath_direct_library_deps) 1712 1713 # Preserve order of |all_library_deps|. Move low priority libraries to the 1714 # end of the classpath. 1715 all_library_deps_sorted_for_classpath = sorted( 1716 all_library_deps[::-1], key=_CompareClasspathPriority) 1717 1718 # The classpath used for bytecode-rewritting. 1719 javac_full_classpath = OrderedSet.fromkeys( 1720 c['unprocessed_jar_path'] 1721 for c in all_library_deps_sorted_for_classpath) 1722 # The classpath used for error prone. 1723 javac_full_interface_classpath = OrderedSet.fromkeys( 1724 c['interface_jar_path'] for c in all_library_deps_sorted_for_classpath) 1725 1726 # Adding base module to classpath to compile against its R.java file 1727 if base_module_build_config: 1728 javac_full_classpath.add( 1729 base_module_build_config['deps_info']['unprocessed_jar_path']) 1730 javac_full_interface_classpath.add( 1731 base_module_build_config['deps_info']['interface_jar_path']) 1732 # Turbine now compiles headers against only the direct classpath, so the 1733 # base module's interface jar must be on the direct interface classpath. 1734 javac_interface_classpath.add( 1735 base_module_build_config['deps_info']['interface_jar_path']) 1736 1737 for dep in classpath_direct_deps: 1738 if 'extra_classpath_jars' in dep: 1739 javac_classpath.update(dep['extra_classpath_jars']) 1740 javac_interface_classpath.update(dep['extra_classpath_jars']) 1741 for dep in all_deps: 1742 if 'extra_classpath_jars' in dep: 1743 javac_full_classpath.update(dep['extra_classpath_jars']) 1744 javac_full_interface_classpath.update(dep['extra_classpath_jars']) 1745 1746 # TODO(agrieve): Might be less confusing to fold these into bootclasspath. 1747 # Deps to add to the compile-time classpath (but not the runtime classpath). 1748 # These are jars specified by input_jars_paths that almost never change. 1749 # Just add them directly to all the classpaths. 1750 if options.extra_classpath_jars: 1751 javac_classpath.update(extra_classpath_jars) 1752 javac_interface_classpath.update(extra_classpath_jars) 1753 javac_full_classpath.update(extra_classpath_jars) 1754 javac_full_interface_classpath.update(extra_classpath_jars) 1755 1756 if is_java_target or options.type == 'android_app_bundle': 1757 # The classpath to use to run this target (or as an input to ProGuard). 1758 device_classpath = [] 1759 # dist_jar configs should not list their device jar in their own classpath 1760 # since the classpath is used to create the device jar itself. 1761 if (is_java_target and options.device_jar_path 1762 and options.type != 'dist_jar'): 1763 device_classpath.append(options.device_jar_path) 1764 device_classpath.extend( 1765 c.get('device_jar_path') for c in all_library_deps 1766 if c.get('device_jar_path')) 1767 if options.type == 'android_app_bundle': 1768 for d in deps.Direct('android_app_bundle_module'): 1769 device_classpath.extend(c for c in d.get('device_classpath', []) 1770 if c not in device_classpath) 1771 1772 all_dist_jar_deps = deps.All('dist_jar') 1773 1774 # We allow lint to be run on android_apk targets, so we collect lint 1775 # artifacts for them. 1776 # We allow lint to be run on android_app_bundle targets, so we need to 1777 # collect lint artifacts for the android_app_bundle_module targets that the 1778 # bundle includes. Different android_app_bundle targets may include different 1779 # android_app_bundle_module targets, so the bundle needs to be able to 1780 # de-duplicate these lint artifacts. 1781 if options.type in ('android_app_bundle_module', 'android_apk'): 1782 # Collect all sources and resources at the apk/bundle_module level. 1783 lint_aars = set() 1784 lint_srcjars = set() 1785 lint_sources = set() 1786 lint_resource_sources = set() 1787 lint_resource_zips = set() 1788 1789 if options.target_sources_file: 1790 lint_sources.add(options.target_sources_file) 1791 if options.bundled_srcjars: 1792 lint_srcjars.update(deps_info['bundled_srcjars']) 1793 for c in all_library_deps: 1794 if c['chromium_code'] and c['requires_android']: 1795 if 'target_sources_file' in c: 1796 lint_sources.add(c['target_sources_file']) 1797 lint_srcjars.update(c['bundled_srcjars']) 1798 if 'aar_path' in c: 1799 lint_aars.add(c['aar_path']) 1800 1801 if options.res_sources_path: 1802 lint_resource_sources.add(options.res_sources_path) 1803 if options.resources_zip: 1804 lint_resource_zips.add(options.resources_zip) 1805 for c in all_resources_deps: 1806 if c['chromium_code']: 1807 # Prefer res_sources_path to resources_zips so that lint errors have 1808 # real paths and to avoid needing to extract during lint. 1809 if c['res_sources_path']: 1810 lint_resource_sources.add(c['res_sources_path']) 1811 else: 1812 lint_resource_zips.add(c['resources_zip']) 1813 1814 deps_info['lint_aars'] = sorted(lint_aars) 1815 deps_info['lint_srcjars'] = sorted(lint_srcjars) 1816 deps_info['lint_sources'] = sorted(lint_sources) 1817 deps_info['lint_resource_sources'] = sorted(lint_resource_sources) 1818 deps_info['lint_resource_zips'] = sorted(lint_resource_zips) 1819 deps_info['lint_extra_android_manifests'] = [] 1820 1821 if options.type == 'android_apk': 1822 assert options.android_manifest, 'Android APKs must define a manifest' 1823 deps_info['lint_android_manifest'] = options.android_manifest 1824 1825 if options.type == 'android_app_bundle': 1826 module_config_paths = action_helpers.parse_gn_list( 1827 options.module_build_configs) 1828 module_configs = [GetDepConfig(c) for c in module_config_paths] 1829 module_configs_by_name = {d['module_name']: d for d in module_configs} 1830 per_module_fields = [ 1831 'device_classpath', 'trace_event_rewritten_device_classpath', 1832 'all_dex_files', 'assets' 1833 ] 1834 lint_aars = set() 1835 lint_srcjars = set() 1836 lint_sources = set() 1837 lint_resource_sources = set() 1838 lint_resource_zips = set() 1839 lint_extra_android_manifests = set() 1840 config['modules'] = {} 1841 modules = config['modules'] 1842 for n, c in module_configs_by_name.items(): 1843 if n == 'base': 1844 assert 'base_module_config' not in deps_info, ( 1845 'Must have exactly 1 base module!') 1846 deps_info['package_name'] = c['package_name'] 1847 deps_info['version_code'] = c['version_code'] 1848 deps_info['version_name'] = c['version_name'] 1849 deps_info['base_module_config'] = c['path'] 1850 deps_info['lint_android_manifest'] = c['android_manifest'] 1851 else: 1852 # All manifests nodes are merged into the main manfiest by lint.py. 1853 lint_extra_android_manifests.add(c['android_manifest']) 1854 1855 lint_extra_android_manifests.update(c['extra_android_manifests']) 1856 lint_aars.update(c['lint_aars']) 1857 lint_srcjars.update(c['lint_srcjars']) 1858 lint_sources.update(c['lint_sources']) 1859 lint_resource_sources.update(c['lint_resource_sources']) 1860 lint_resource_zips.update(c['lint_resource_zips']) 1861 module = modules[n] = {} 1862 for f in per_module_fields: 1863 if f in c: 1864 module[f] = c[f] 1865 deps_info['lint_aars'] = sorted(lint_aars) 1866 deps_info['lint_srcjars'] = sorted(lint_srcjars) 1867 deps_info['lint_sources'] = sorted(lint_sources) 1868 deps_info['lint_resource_sources'] = sorted(lint_resource_sources) 1869 deps_info['lint_resource_zips'] = sorted(lint_resource_zips) 1870 deps_info['lint_extra_android_manifests'] = sorted( 1871 lint_extra_android_manifests) 1872 1873 _DedupFeatureModuleSharedCode(options.uses_split, modules, 1874 per_module_fields) 1875 1876 system_jars = [c['unprocessed_jar_path'] for c in system_library_deps] 1877 system_interface_jars = [c['interface_jar_path'] for c in system_library_deps] 1878 if system_library_deps: 1879 config['android'] = {} 1880 config['android']['sdk_interface_jars'] = system_interface_jars 1881 config['android']['sdk_jars'] = system_jars 1882 1883 if options.type in ('android_apk', 'dist_aar', 1884 'dist_jar', 'android_app_bundle_module', 'android_app_bundle'): 1885 for c in all_deps: 1886 proguard_configs.extend(c.get('proguard_configs', [])) 1887 extra_proguard_classpath_jars.extend(c.get('extra_classpath_jars', [])) 1888 if options.type == 'android_app_bundle': 1889 for c in deps.Direct('android_app_bundle_module'): 1890 proguard_configs.extend(p for p in c.get('proguard_configs', [])) 1891 if options.type == 'android_app_bundle': 1892 for d in deps.Direct('android_app_bundle_module'): 1893 extra_proguard_classpath_jars.extend( 1894 c for c in d.get('proguard_classpath_jars', []) 1895 if c not in extra_proguard_classpath_jars) 1896 1897 if options.type == 'android_app_bundle': 1898 deps_proguard_enabled = [] 1899 deps_proguard_disabled = [] 1900 for d in deps.Direct('android_app_bundle_module'): 1901 if not d['device_classpath']: 1902 # We don't care about modules that have no Java code for proguarding. 1903 continue 1904 if d['proguard_enabled']: 1905 deps_proguard_enabled.append(d['name']) 1906 else: 1907 deps_proguard_disabled.append(d['name']) 1908 if deps_proguard_enabled and deps_proguard_disabled: 1909 raise Exception('Deps %s have proguard enabled while deps %s have ' 1910 'proguard disabled' % (deps_proguard_enabled, 1911 deps_proguard_disabled)) 1912 deps_info['proguard_enabled'] = bool(options.proguard_enabled) 1913 1914 if options.proguard_mapping_path: 1915 deps_info['proguard_mapping_path'] = options.proguard_mapping_path 1916 1917 # The java code for an instrumentation test apk is assembled differently for 1918 # ProGuard vs. non-ProGuard. 1919 # 1920 # Without ProGuard: Each library's jar is dexed separately and then combined 1921 # into a single classes.dex. A test apk will include all dex files not already 1922 # present in the apk-under-test. At runtime all test code lives in the test 1923 # apk, and the program code lives in the apk-under-test. 1924 # 1925 # With ProGuard: Each library's .jar file is fed into ProGuard, which outputs 1926 # a single .jar, which is then dexed into a classes.dex. A test apk includes 1927 # all jar files from the program and the tests because having them separate 1928 # doesn't work with ProGuard's whole-program optimizations. Although the 1929 # apk-under-test still has all of its code in its classes.dex, none of it is 1930 # used at runtime because the copy of it within the test apk takes precidence. 1931 1932 if options.type == 'android_apk' and options.tested_apk_config: 1933 if tested_apk_config['proguard_enabled']: 1934 assert options.proguard_enabled, ('proguard must be enabled for ' 1935 'instrumentation apks if it\'s enabled for the tested apk.') 1936 # Mutating lists, so no need to explicitly re-assign to dict. 1937 proguard_configs.extend( 1938 p for p in tested_apk_config['proguard_all_configs']) 1939 extra_proguard_classpath_jars.extend( 1940 p for p in tested_apk_config['proguard_classpath_jars']) 1941 tested_apk_config = GetDepConfig(options.tested_apk_config) 1942 deps_info['proguard_under_test_mapping'] = ( 1943 tested_apk_config['proguard_mapping_path']) 1944 elif options.proguard_enabled: 1945 # Not sure why you'd want to proguard the test apk when the under-test apk 1946 # is not proguarded, but it's easy enough to support. 1947 deps_info['proguard_under_test_mapping'] = '' 1948 1949 # Add all tested classes to the test's classpath to ensure that the test's 1950 # java code is a superset of the tested apk's java code 1951 device_classpath_extended = list(device_classpath) 1952 device_classpath_extended.extend( 1953 p for p in tested_apk_config['device_classpath'] 1954 if p not in device_classpath) 1955 # Include in the classpath classes that are added directly to the apk under 1956 # test (those that are not a part of a java_library). 1957 javac_classpath.add(tested_apk_config['unprocessed_jar_path']) 1958 javac_interface_classpath.add(tested_apk_config['interface_jar_path']) 1959 javac_full_classpath.add(tested_apk_config['unprocessed_jar_path']) 1960 javac_full_interface_classpath.add(tested_apk_config['interface_jar_path']) 1961 javac_full_classpath.update(tested_apk_config['javac_full_classpath']) 1962 javac_full_interface_classpath.update( 1963 tested_apk_config['javac_full_interface_classpath']) 1964 1965 # Exclude .jar files from the test apk that exist within the apk under test. 1966 tested_apk_library_deps = tested_apk_deps.All('java_library') 1967 tested_apk_dex_files = {c['dex_path'] for c in tested_apk_library_deps} 1968 all_dex_files = [p for p in all_dex_files if p not in tested_apk_dex_files] 1969 tested_apk_jar_files = set(tested_apk_config['device_classpath']) 1970 device_classpath = [ 1971 p for p in device_classpath if p not in tested_apk_jar_files 1972 ] 1973 1974 if options.type in ('android_apk', 'dist_aar', 'dist_jar', 1975 'android_app_bundle_module', 'android_app_bundle'): 1976 deps_info['proguard_all_configs'] = sorted(set(proguard_configs)) 1977 deps_info['proguard_classpath_jars'] = sorted( 1978 set(extra_proguard_classpath_jars)) 1979 1980 if options.type in ('dist_jar', 'java_binary', 'robolectric_binary'): 1981 # The classpath to use to run this target. 1982 host_classpath = [] 1983 if options.host_jar_path: 1984 host_classpath.append(options.host_jar_path) 1985 host_classpath.extend(c['host_jar_path'] for c in all_library_deps) 1986 # Collect all the dist_jar host jars. 1987 dist_jar_host_jars = [ 1988 c['host_jar_path'] for c in all_dist_jar_deps if 'host_jar_path' in c 1989 ] 1990 # Collect all the jars that went into the dist_jar host jars. 1991 dist_jar_host_classpath = set() 1992 for c in all_dist_jar_deps: 1993 dist_jar_host_classpath.update(c['host_classpath']) 1994 # Remove the jars that went into the dist_jar host jars. 1995 host_classpath = [ 1996 p for p in host_classpath if p not in dist_jar_host_classpath 1997 ] 1998 # Add the dist_jar host jars themselves instead. 1999 host_classpath += dist_jar_host_jars 2000 deps_info['host_classpath'] = host_classpath 2001 2002 if is_java_target: 2003 2004 def _CollectListsFromDeps(deps, key_name): 2005 combined = set() 2006 for config in deps: 2007 combined.update(config.get(key_name, [])) 2008 return combined 2009 2010 dist_jar_device_classpath = _CollectListsFromDeps(all_dist_jar_deps, 2011 'device_classpath') 2012 dist_jar_javac_full_classpath = _CollectListsFromDeps( 2013 all_dist_jar_deps, 'javac_full_classpath') 2014 dist_jar_javac_full_interface_classpath = _CollectListsFromDeps( 2015 all_dist_jar_deps, 'javac_full_interface_classpath') 2016 dist_jar_child_dex_files = _CollectListsFromDeps(all_dist_jar_deps, 2017 'all_dex_files') 2018 2019 def _CollectSinglesFromDeps(deps, key_name): 2020 return [config[key_name] for config in deps if key_name in config] 2021 2022 dist_jar_device_jars = _CollectSinglesFromDeps(all_dist_jar_deps, 2023 'device_jar_path') 2024 dist_jar_combined_dex_files = _CollectSinglesFromDeps( 2025 all_dist_jar_deps, 'dex_path') 2026 dist_jar_interface_jars = _CollectSinglesFromDeps(all_dist_jar_deps, 2027 'interface_jar_path') 2028 dist_jar_unprocessed_jars = _CollectSinglesFromDeps(all_dist_jar_deps, 2029 'unprocessed_jar_path') 2030 2031 device_classpath = [ 2032 p for p in device_classpath if p not in dist_jar_device_classpath 2033 ] 2034 device_classpath += dist_jar_device_jars 2035 2036 javac_full_classpath.difference_update(dist_jar_javac_full_classpath) 2037 javac_full_classpath.update(dist_jar_unprocessed_jars) 2038 2039 javac_full_interface_classpath.difference_update( 2040 dist_jar_javac_full_interface_classpath) 2041 javac_full_interface_classpath.update(dist_jar_interface_jars) 2042 2043 javac_interface_classpath.update(dist_jar_interface_jars) 2044 javac_classpath.update(dist_jar_unprocessed_jars) 2045 2046 if is_apk_or_module_target or options.type == 'dist_jar': 2047 all_dex_files = [ 2048 p for p in all_dex_files if p not in dist_jar_child_dex_files 2049 ] 2050 all_dex_files += dist_jar_combined_dex_files 2051 2052 if options.final_dex_path: 2053 config['final_dex'] = {'path': options.final_dex_path} 2054 if is_apk_or_module_target or options.type == 'dist_jar': 2055 # Dependencies for the final dex file of an apk. 2056 deps_info['all_dex_files'] = all_dex_files 2057 2058 if is_java_target: 2059 config['javac']['classpath'] = sorted(javac_classpath) 2060 config['javac']['interface_classpath'] = sorted(javac_interface_classpath) 2061 # Direct() will be of type 'java_annotation_processor', and so not included 2062 # in All('java_library'). 2063 # Annotation processors run as part of the build, so need host_jar_path. 2064 config['javac']['processor_classpath'] = [ 2065 c['host_jar_path'] for c in processor_deps.Direct() 2066 if c.get('host_jar_path') 2067 ] 2068 config['javac']['processor_classpath'] += [ 2069 c['host_jar_path'] for c in processor_deps.All('java_library') 2070 ] 2071 config['javac']['processor_classes'] = sorted( 2072 c['main_class'] for c in processor_deps.Direct()) 2073 deps_info['javac_full_classpath'] = list(javac_full_classpath) 2074 deps_info['javac_full_interface_classpath'] = list( 2075 javac_full_interface_classpath) 2076 elif options.type == 'android_app_bundle': 2077 # bundles require javac_full_classpath to create .aab.jar.info and require 2078 # javac_full_interface_classpath for lint. 2079 javac_full_classpath = OrderedSet() 2080 javac_full_interface_classpath = OrderedSet() 2081 for d in deps.Direct('android_app_bundle_module'): 2082 javac_full_classpath.update(d['javac_full_classpath']) 2083 javac_full_interface_classpath.update(d['javac_full_interface_classpath']) 2084 javac_full_classpath.add(d['unprocessed_jar_path']) 2085 javac_full_interface_classpath.add(d['interface_jar_path']) 2086 deps_info['javac_full_classpath'] = list(javac_full_classpath) 2087 deps_info['javac_full_interface_classpath'] = list( 2088 javac_full_interface_classpath) 2089 2090 if options.type in ('android_apk', 'android_app_bundle', 2091 'android_app_bundle_module', 'dist_aar', 'dist_jar'): 2092 deps_info['device_classpath'] = device_classpath 2093 if options.trace_events_jar_dir: 2094 trace_event_rewritten_device_classpath = [] 2095 for jar_path in device_classpath: 2096 file_path = jar_path.replace('../', '') 2097 file_path = file_path.replace('obj/', '') 2098 file_path = file_path.replace('gen/', '') 2099 file_path = file_path.replace('.jar', '.tracing_rewritten.jar') 2100 rewritten_jar_path = os.path.join(options.trace_events_jar_dir, 2101 file_path) 2102 trace_event_rewritten_device_classpath.append(rewritten_jar_path) 2103 2104 deps_info['trace_event_rewritten_device_classpath'] = ( 2105 trace_event_rewritten_device_classpath) 2106 2107 if options.tested_apk_config: 2108 deps_info['device_classpath_extended'] = device_classpath_extended 2109 2110 if options.type == 'dist_jar': 2111 if options.direct_deps_only: 2112 if options.use_interface_jars: 2113 dist_jars = config['javac']['interface_classpath'] 2114 else: 2115 dist_jars = sorted(c['device_jar_path'] 2116 for c in classpath_direct_library_deps) 2117 else: 2118 if options.use_interface_jars: 2119 dist_jars = [c['interface_jar_path'] for c in all_library_deps] 2120 else: 2121 dist_jars = deps_info['device_classpath'] 2122 2123 config['dist_jar'] = { 2124 'jars': dist_jars, 2125 } 2126 2127 if is_apk_or_module_target: 2128 manifest = AndroidManifest(options.android_manifest) 2129 deps_info['package_name'] = manifest.GetPackageName() 2130 if not options.tested_apk_config and manifest.GetInstrumentationElements(): 2131 # This must then have instrumentation only for itself. 2132 manifest.CheckInstrumentationElements(manifest.GetPackageName()) 2133 2134 library_paths = [] 2135 java_libraries_list = None 2136 if options.shared_libraries_runtime_deps: 2137 library_paths = _ExtractSharedLibsFromRuntimeDeps( 2138 options.shared_libraries_runtime_deps) 2139 java_libraries_list = _CreateJavaLibrariesList(library_paths) 2140 all_inputs.append(options.shared_libraries_runtime_deps) 2141 2142 secondary_abi_library_paths = [] 2143 if options.secondary_abi_shared_libraries_runtime_deps: 2144 secondary_abi_library_paths = _ExtractSharedLibsFromRuntimeDeps( 2145 options.secondary_abi_shared_libraries_runtime_deps) 2146 secondary_abi_library_paths.sort() 2147 paths_without_parent_dirs = [ 2148 p for p in secondary_abi_library_paths if os.path.sep not in p 2149 ] 2150 if paths_without_parent_dirs: 2151 sys.stderr.write('Found secondary native libraries from primary ' 2152 'toolchain directory. This is a bug!\n') 2153 sys.stderr.write('\n'.join(paths_without_parent_dirs)) 2154 sys.stderr.write('\n\nIt may be helpful to run: \n') 2155 sys.stderr.write(' gn path out/Default //chrome/android:' 2156 'monochrome_secondary_abi_lib //base:base\n') 2157 sys.exit(1) 2158 2159 all_inputs.append(options.secondary_abi_shared_libraries_runtime_deps) 2160 2161 native_library_placeholder_paths = action_helpers.parse_gn_list( 2162 options.native_lib_placeholders) 2163 native_library_placeholder_paths.sort() 2164 2165 secondary_native_library_placeholder_paths = action_helpers.parse_gn_list( 2166 options.secondary_native_lib_placeholders) 2167 secondary_native_library_placeholder_paths.sort() 2168 2169 loadable_modules = action_helpers.parse_gn_list(options.loadable_modules) 2170 loadable_modules.sort() 2171 secondary_abi_loadable_modules = action_helpers.parse_gn_list( 2172 options.secondary_abi_loadable_modules) 2173 secondary_abi_loadable_modules.sort() 2174 2175 config['native'] = { 2176 'libraries': 2177 library_paths, 2178 'native_library_placeholders': 2179 native_library_placeholder_paths, 2180 'secondary_abi_libraries': 2181 secondary_abi_library_paths, 2182 'secondary_native_library_placeholders': 2183 secondary_native_library_placeholder_paths, 2184 'java_libraries_list': 2185 java_libraries_list, 2186 'library_always_compress': 2187 options.library_always_compress, 2188 'loadable_modules': 2189 loadable_modules, 2190 'secondary_abi_loadable_modules': 2191 secondary_abi_loadable_modules, 2192 } 2193 2194 # Collect java resources 2195 java_resources_jars = [d['java_resources_jar'] for d in all_library_deps 2196 if 'java_resources_jar' in d] 2197 if options.tested_apk_config: 2198 tested_apk_resource_jars = [d['java_resources_jar'] 2199 for d in tested_apk_library_deps 2200 if 'java_resources_jar' in d] 2201 java_resources_jars = [jar for jar in java_resources_jars 2202 if jar not in tested_apk_resource_jars] 2203 java_resources_jars.sort() 2204 config['java_resources_jars'] = java_resources_jars 2205 2206 if is_apk_or_module_target or options.type == 'robolectric_binary': 2207 # android_resources deps which had recursive_resource_deps set should not 2208 # have the manifests from the recursively collected deps added to this 2209 # module. This keeps the manifest declarations in the child DFMs, since they 2210 # will have the Java implementations. 2211 def ExcludeRecursiveResourcesDeps(config): 2212 return not config.get('includes_recursive_resources', False) 2213 2214 extra_manifest_deps = [ 2215 GetDepConfig(p) for p in GetAllDepsConfigsInOrder( 2216 deps_configs_paths, filter_func=ExcludeRecursiveResourcesDeps) 2217 ] 2218 # Manifests are listed from highest priority to lowest priority. 2219 # Ensure directly manfifests come first, and then sort the rest by name. 2220 # https://developer.android.com/build/manage-manifests#merge_priorities 2221 deps_info['extra_android_manifests'] = list(mergeable_android_manifests) 2222 manifests_from_deps = [] 2223 for c in extra_manifest_deps: 2224 manifests_from_deps += c.get('mergeable_android_manifests', []) 2225 manifests_from_deps.sort(key=lambda p: (os.path.basename(p), p)) 2226 deps_info['extra_android_manifests'] += manifests_from_deps 2227 2228 assets, uncompressed_assets, locale_paks = _MergeAssets( 2229 deps.All('android_assets')) 2230 deps_info['assets'] = assets 2231 deps_info['uncompressed_assets'] = uncompressed_assets 2232 deps_info['locales_java_list'] = _CreateJavaLocaleListFromAssets( 2233 uncompressed_assets, locale_paks) 2234 if options.suffix_apk_assets_used_by: 2235 if options.suffix_apk_assets_used_by == options.build_config: 2236 target_config = deps_info 2237 else: 2238 target_config = GetDepConfig(options.suffix_apk_assets_used_by) 2239 all_assets = (target_config['assets'] + 2240 target_config['uncompressed_assets']) 2241 suffix = '+' + target_config['package_name'] + '+' 2242 suffix_names = { 2243 x.split(':', 1)[1].replace(suffix, '') 2244 for x in all_assets 2245 } 2246 deps_info['assets'] = _SuffixAssets(suffix_names, suffix, assets) 2247 deps_info['uncompressed_assets'] = _SuffixAssets(suffix_names, suffix, 2248 uncompressed_assets) 2249 config['apk_assets_suffixed_list'] = ','.join( 2250 f'"assets/{x}"' for x in sorted(suffix_names)) 2251 config['apk_assets_suffix'] = suffix 2252 2253 if options.java_resources_jar_path: 2254 deps_info['java_resources_jar'] = options.java_resources_jar_path 2255 2256 # DYNAMIC FEATURE MODULES: 2257 # There are two approaches to dealing with modules dependencies: 2258 # 1) Perform steps in android_apk_or_module(), with only the knowledge of 2259 # ancesstor splits. Our implementation currently allows only for 2 levels: 2260 # base -> parent -> leaf 2261 # Bundletool normally fails if two leaf nodes merge the same manifest or 2262 # resources. The fix is to add the common dep to the chrome or base module 2263 # so that our deduplication logic will work. 2264 # RemoveObjDups() implements this approach. 2265 # 2) Perform steps in android_app_bundle(), with knowledge of full set of 2266 # modules. This is required for dex because it can handle the case of two 2267 # leaf nodes having the same dep, and promoting that dep to their common 2268 # parent. 2269 # _DedupFeatureModuleSharedCode() implements this approach. 2270 if base_module_build_config: 2271 ancestors = [base_module_build_config] 2272 if parent_module_build_config is not base_module_build_config: 2273 ancestors += [parent_module_build_config] 2274 for ancestor in ancestors: 2275 RemoveObjDups(config, ancestor, 'deps_info', 'dependency_zips') 2276 RemoveObjDups(config, ancestor, 'deps_info', 'dependency_zip_overlays') 2277 RemoveObjDups(config, ancestor, 'deps_info', 'extra_android_manifests') 2278 RemoveObjDups(config, ancestor, 'deps_info', 'extra_package_names') 2279 2280 if is_java_target: 2281 jar_to_target = {} 2282 _AddJarMapping(jar_to_target, [deps_info]) 2283 _AddJarMapping(jar_to_target, all_deps) 2284 if base_module_build_config: 2285 _AddJarMapping(jar_to_target, [base_module_build_config['deps_info']]) 2286 if parent_module_build_config is not base_module_build_config: 2287 _AddJarMapping(jar_to_target, [parent_module_build_config['deps_info']]) 2288 if options.tested_apk_config: 2289 _AddJarMapping(jar_to_target, [tested_apk_config]) 2290 for jar, target in zip(tested_apk_config['javac_full_classpath'], 2291 tested_apk_config['javac_full_classpath_targets']): 2292 jar_to_target[jar] = target 2293 2294 # Used by check_for_missing_direct_deps.py to give better error message 2295 # when missing deps are found. Both javac_full_classpath_targets and 2296 # javac_full_classpath must be in identical orders, as they get passed as 2297 # separate arrays and then paired up based on index. 2298 config['deps_info']['javac_full_classpath_targets'] = [ 2299 jar_to_target[x] for x in deps_info['javac_full_classpath'] 2300 ] 2301 2302 build_utils.WriteJson(config, options.build_config, only_if_changed=True) 2303 2304 if options.depfile: 2305 action_helpers.write_depfile(options.depfile, options.build_config, 2306 sorted(set(all_inputs))) 2307 2308 if options.store_deps_for_debugging_to: 2309 GetDepConfig(options.build_config) # Add it to cache. 2310 _CopyBuildConfigsForDebugging(options.store_deps_for_debugging_to) 2311 2312 return 0 2313 2314 2315 if __name__ == '__main__': 2316 sys.exit(main(sys.argv[1:]))