ipdl.rst (90816B)
1 IPDL: Inter-Thread and Inter-Process Message Passing 2 ==================================================== 3 4 The Idea 5 -------- 6 7 **IPDL**, the "Inter-[thread|process] Protocol Definition Language", is the 8 Mozilla-specific language that allows code to communicate between system 9 threads or processes in a standardized, efficient, safe, secure and 10 platform-agnostic way. IPDL communications take place between *parent* and 11 *child* objects called *actors*. The architecture is inspired by the `actor 12 model <https://en.wikipedia.org/wiki/Actor_model>`_. 13 14 .. note:: 15 IPDL actors differ from the actor model in one significant way -- all 16 IPDL communications are *only* between a parent and its only child. 17 18 The actors that constitute a parent/child pair are called **peers**. Peer 19 actors communicate through an **endpoint**, which is an end of a message pipe. 20 An actor is explicitly bound to its endpoint, which in turn is bound to a 21 particular thread soon after it is constructed. An actor never changes its 22 endpoint and may only send and receive predeclared **messages** from/to that 23 endpoint, on that thread. Violations result in runtime errors. A thread may 24 be bound to many otherwise unrelated actors but an endpoint supports 25 **top-level** actors and any actors they **manage** (see below). 26 27 .. note:: 28 More precisely, endpoints can be bound to any ``nsISerialEventTarget``, 29 which are themselves associated with a specific thread. By default, 30 IPDL will bind to the current thread's "main" serial event target, 31 which, if it exists, is retrieved with ``GetCurrentSerialEventTarget``. 32 For the sake of clarity, this document will frequently refer to actors 33 as bound to threads, although the more precise interpretation of serial 34 event targets is also always valid. 35 36 .. note:: 37 Internally, we use the "Ports" component of the `Chromium Mojo`_ library 38 to *multiplex* multiple endpoints (and, therefore, multiple top-level 39 actors). This means that the endpoints communicate over the same native 40 pipe, which conserves limited OS resources. The implications of this are 41 discussed in `IPDL Best Practices`_. 42 43 Parent and child actors may be bound to threads in different processes, in 44 different threads in the same process, or even in the same thread in the same 45 process. That last option may seem unreasonable but actors are versatile and 46 their layout can be established at run-time so this could theoretically arise 47 as the result of run-time choices. One large example of this versatility is 48 ``PCompositorBridge`` actors, which in different cases connect endpoints in the 49 main process and the GPU process (for UI rendering on Windows), in a content 50 process and the GPU process (for content rendering on Windows), in the main 51 process and the content process (for content rendering on Mac, where there is 52 no GPU process), or between threads on the main process (UI rendering on Mac). 53 For the most part, this does not require elaborate or redundant coding; it 54 just needs endpoints to be bound judiciously at runtime. The example in 55 :ref:`Connecting With Other Processes` shows one way this can be done. It 56 also shows that, without proper plain-language documentation of *all* of the 57 ways endpoints are configured, this can quickly lead to unmaintainable code. 58 Be sure to document your endpoint bindings thoroughly!!! 59 60 .. _Chromium Mojo: https://chromium.googlesource.com/chromium/src/+/refs/heads/main/mojo/core/README.md#Port 61 62 The Approach 63 ------------ 64 65 The actor framework will schedule tasks to run on its associated event target, 66 in response to messages it receives. Messages are specified in an IPDL 67 **protocol** file and the response handler tasks are defined per-message by C++ 68 methods. As actors only communicate in pairs, and each is bound to one thread, 69 sending is always done sequentially, never concurrently (same for receiving). 70 This means that it can, and does, guarantee that an actor will always receive 71 messages in the same order they were sent by its related actor -- and that this 72 order is well defined since the related actor can only send from one thread. 73 74 .. warning:: 75 There are a few (rare) exceptions to the message order guarantee. They 76 include `synchronous nested`_ messages and messages with a ``[Priority]`` 77 or ``[Compress]`` annotation. 78 79 An IPDL protocol file specifies the messages that may be sent between parent 80 and child actors, as well as the direction and payload of those messages. 81 Messages look like function calls but, from the standpoint of their caller, 82 they may start and end at any time in the future -- they are *asynchronous*, 83 so they won't block their sending actors or any other components that may be 84 running in the actor's thread's ``MessageLoop``. 85 86 .. note:: 87 Not all IPDL messages are asynchronous. Again, we run into exceptions for 88 messages that are synchronous or `synchronous nested`_. Use of synchronous 89 and nested messages is strongly discouraged but may not always be avoidable. 90 They will be defined later, along with superior alternatives to both that 91 should work in nearly all cases. 92 93 Protocol files are compiled by the *IPDL compiler* in an early stage of the 94 build process. The compiler generates C++ code that reflects the protocol. 95 Specifically, it creates one C++ class that represents the parent actor and one 96 that represents the child. The generated files are then automatically included 97 in the C++ build process. The generated classes contain public methods for 98 sending the protocol messages, which client code will use as the entry-point to 99 IPC communication. The generated methods are built atop our IPC framework, 100 defined in `/ipc <https://searchfox.org/mozilla-central/source/ipc>`_, that 101 standardizes the safe and secure use of sockets, pipes, shared memory, etc on 102 all supported platforms. See `Using The IPDL compiler`_ for more on 103 integration with the build process. 104 105 Client code must be written that subclasses these generated classes, in order 106 to add handlers for the tasks generated to respond to each message. It must 107 also add routines (``ParamTraits``) that define serialization and 108 deserialization for any types used in the payload of a message that aren't 109 already known to the IPDL system. Primitive types, and a bunch of Mozilla 110 types, have predefined ``ParamTraits`` (`here 111 <https://searchfox.org/mozilla-central/source/ipc/glue/IPCMessageUtils.h>`__ 112 and `here 113 <https://searchfox.org/mozilla-central/source/ipc/glue/IPCMessageUtilsSpecializations.h>`__). 114 115 .. note:: 116 Among other things, client code that uses the generated code must include 117 ``chromium-config.mozbuild`` in its ``moz.build`` file. See `Using The 118 IPDL compiler`_ for a complete list of required build changes. 119 120 .. _synchronous nested: `The Rest`_ 121 122 The Steps To Making A New Actor 123 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 124 125 #. Decide what folder you will work in and create: 126 127 #. An IPDL protocol file, named for your actor (e.g. ``PMyActor.ipdl`` -- 128 actor protocols must begin with a ``P``). See `The Protocol Language`_. 129 #. Properly-named source files for your actor's parent and child 130 implementations (e.g. ``MyActorParent.h``, ``MyActorChild.h`` and, 131 optionally, adjacent .cpp files). See `The C++ Interface`_. 132 #. IPDL-specific updates to the ``moz.build`` file. See `Using The IPDL 133 compiler`_. 134 #. Write your actor protocol (.ipdl) file: 135 136 #. Decide whether you need a top-level actor or a managed actor. See 137 `Top Level Actors`_. 138 #. Find/write the IPDL and C++ data types you will use in communication. 139 Write ``ParamTraits`` for C++ data types that don't have them. See 140 `Generating IPDL-Aware C++ Data Types: IPDL Structs and Unions`_ for IPDL 141 structures. See `Referencing Externally Defined Data Types: IPDL 142 Includes`_ and `ParamTraits`_ for C++ data types. 143 #. Write your actor and its messages. See `Defining Actors`_. 144 #. Write C++ code to create and destroy instances of your actor at runtime. 145 146 * For managed actors, see `Actor Lifetimes in C++`_. 147 * For top-level actors, see `Creating Top Level Actors From Other Actors`_. 148 The first actor in a process is a very special exception -- see `Creating 149 First Top Level Actors`_. 150 #. Write handlers for your actor's messages. See `Actors and Messages in 151 C++`_. 152 #. Start sending messages through your actors! Again, see `Actors and Messages 153 in C++`_. 154 155 The Protocol Language 156 --------------------- 157 158 This document will follow the integration of two actors into Firefox -- 159 ``PMyManager`` and ``PMyManaged``. ``PMyManager`` will manage ``PMyManaged``. 160 A good place to start is with the IPDL actor definitions. These are files 161 that are named for the actor (e.g. ``PMyManager.ipdl``) and that declare the 162 messages that a protocol understands. These actors are for demonstration 163 purposes and involve quite a bit of functionality. Most actors will use a very 164 small fraction of these features. 165 166 .. literalinclude:: _static/PMyManager.ipdl 167 :language: c++ 168 :name: PMyManager.ipdl 169 170 .. literalinclude:: _static/PMyManaged.ipdl 171 :language: c++ 172 :name: PMyManaged.ipdl 173 174 These files reference three additional files. ``MyTypes.ipdlh`` is an "IPDL 175 header" that can be included into ``.ipdl`` files as if it were inline, except 176 that it also needs to include any external actors and data types it uses: 177 178 .. literalinclude:: _static/MyTypes.ipdlh 179 :language: c++ 180 :name: MyTypes.ipdlh 181 182 ``MyActorUtils.h`` and ``MyDataTypes.h`` are normal C++ header files that 183 contain definitions for types passed by these messages, as well as instructions 184 for serializing them. They will be covered in `The C++ Interface`_. 185 186 Using The IPDL compiler 187 ~~~~~~~~~~~~~~~~~~~~~~~ 188 189 To build IPDL files, list them (alphabetically sorted) in a ``moz.build`` file. 190 In this example, the ``.ipdl`` and ``.ipdlh`` files would be alongside a 191 ``moz.build`` containing: 192 193 .. code-block:: python 194 195 IPDL_SOURCES += [ 196 "MyTypes.ipdlh", 197 "PMyManaged.ipdl", 198 "PMyManager.ipdl", 199 ] 200 201 UNIFIED_SOURCES += [ 202 "MyManagedChild.cpp", 203 "MyManagedParent.cpp", 204 "MyManagerChild.cpp", 205 "MyManagerParent.cpp", 206 ] 207 208 include("/ipc/chromium/chromium-config.mozbuild") 209 210 ``chromium-config.mozbuild`` sets up paths so that generated IPDL header files 211 are in the proper scope. If it isn't included, the build will fail with 212 ``#include`` errors in both your actor code and some internal ipc headers. For 213 example: 214 215 .. code-block:: 216 217 c:/mozilla-src/mozilla-unified/obj-64/dist/include\ipc/IPCMessageUtils.h(13,10): fatal error: 'build/build_config.h' file not found 218 219 ``.ipdl`` files are compiled to C++ files as one of the earliest post-configure 220 build steps. Those files are, in turn, referenced throughout the source code 221 and build process. From ``PMyManager.ipdl`` the compiler generates two header 222 files added to the build context and exported globally: 223 ``mozilla/myns/PMyManagerParent.h`` and ``mozilla/myns/PMyManagerChild.h``, as 224 discussed in `Namespaces`_ below. These files contain the base classes for the 225 actors. It also makes several other files, including C++ source files and 226 another header, that are automatically included into the build and should not 227 require attention. 228 229 C++ definions of the actors are required for IPDL. They define the actions 230 that are taken in response to messages -- without this, they would have no 231 value. There will be much more on this when we discuss `Actors and Messages in 232 C++`_ but note here that C++ header files named for the actor are required by 233 the IPDL `compiler`. The example would expect 234 ``mozilla/myns/MyManagedChild.h``, ``mozilla/myns/MyManagedParent.h``, 235 ``mozilla/myns/MyManagerChild.h`` and ``mozilla/myns/MyManagerParent.h`` and 236 will not build without them. 237 238 Referencing Externally Defined Data Types: IPDL Includes 239 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 240 241 Let's begin with ``PMyManager.ipdl``. It starts by including types that it 242 will need from other places: 243 244 .. code-block:: cpp 245 246 include protocol PMyManaged; 247 include MyTypes; // for MyActorPair 248 249 using MyActorEnum from "mozilla/myns/MyActorUtils.h"; 250 using struct mozilla::myns::MyData from "mozilla/MyDataTypes.h"; 251 [MoveOnly] using mozilla::myns::MyOtherData from "mozilla/MyDataTypes.h"; 252 [RefCounted] using class mozilla::myns::MyThirdData from "mozilla/MyDataTypes.h"; 253 254 The first line includes a protocol that PMyManager will manage. That protocol 255 is defined in its own ``.ipdl`` file. Cyclic references are expected and pose 256 no concern. 257 258 The second line includes the file ``MyTypes.ipdlh``, which defines types like 259 structs and unions, but in IPDL, which means they have behavior that goes 260 beyond the similar C++ concepts. Details can be found in `Generating 261 IPDL-Aware C++ Data Types: IPDL Structs and Unions`_. 262 263 The final lines include types from C++ headers. Additionally, the [RefCounted] 264 and [MoveOnly] attributes tell IPDL that the types have special functionality 265 that is important to operations. These are the data type attributes currently 266 understood by IPDL: 267 268 ================ ============================================================== 269 ``[RefCounted]`` Type ``T`` is reference counted (by ``AddRef``/``Release``). 270 As a parameter to a message or as a type in IPDL 271 structs/unions, it is referenced as a ``RefPtr<T>``. 272 ``[MoveOnly]`` The type ``T`` is treated as uncopyable. When used as a 273 parameter in a message or an IPDL struct/union, it is as an 274 r-value ``T&&``. 275 ================ ============================================================== 276 277 Finally, note that ``using``, ``using class`` and ``using struct`` are all 278 valid syntax. The ``class`` and ``struct`` keywords are optional. 279 280 Namespaces 281 ~~~~~~~~~~ 282 283 From the IPDL file: 284 285 .. code-block:: cpp 286 287 namespace mozilla { 288 namespace myns { 289 290 // ... data type and actor definitions ... 291 292 } // namespace myns 293 } // namespace mozilla 294 295 296 Namespaces work similar to the way they do in C++. They also mimic the 297 notation, in an attempt to make them comfortable to use. When IPDL actors are 298 compiled into C++ actors, the namespace scoping is carried over. As previously 299 noted, when C++ types are included into IPDL files, the same is true. The most 300 important way in which they differ is that IPDL also uses the namespace to 301 establish the path to the generated files. So, the example defines the IPDL 302 data type ``mozilla::myns::MyUnion`` and the actors 303 ``mozilla::myns::PMyManagerParent`` and ``mozilla::myns::PMyManagerChild``, 304 which can be included from ``mozilla/myns/PMyManagerParent.h``, 305 ``mozilla/myns/PMyManagerParent.h`` and ``mozilla/myns/PMyManagerChild.h``, 306 respectively. The namespace becomes part of the path. 307 308 Generating IPDL-Aware C++ Data Types: IPDL Structs and Unions 309 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 310 311 ``PMyManager.ipdl`` and ``MyTypes.ipdlh`` define: 312 313 .. code-block:: cpp 314 315 [Comparable] union MyUnion { 316 float; 317 MyOtherData; 318 }; 319 320 struct MyActorPair { 321 PMyManaged actor1; 322 nullable PMyManaged actor2; 323 }; 324 325 From these descriptions, IPDL generates C++ classes that approximate the 326 behavior of C++ structs and unions but that come with pre-defined 327 ``ParamTraits`` implementations. These objects can also be used as usual 328 outside of IPDL, although the lack of control over the generated code means 329 they are sometimes poorly suited to use as plain data. See `ParamTraits`_ for 330 details. 331 332 The ``[Comparable]`` attribute tells IPDL to generate ``operator==`` and 333 ``operator!=`` for the new type. In order for it to do that, the fields inside 334 the new type need to define both of those operators. 335 336 Finally, the ``nullable`` keyword indicates that, when serialized, the actor 337 may be null. It is intended to help users avoid null-object dereference 338 errors. It only applies to actor types and may also be attached to parameters 339 in message declarations. 340 341 Defining Actors 342 ~~~~~~~~~~~~~~~ 343 344 The real point of any ``.ipdl`` file is that each defines exactly one actor 345 protocol. The definition always matches the ``.ipdl`` filename. Repeating the 346 one in ``PMyManager.ipdl``: 347 348 .. code-block:: cpp 349 350 [ChildProc=Content] 351 sync protocol PMyManager { 352 manages PMyManaged; 353 354 async PMyManaged(); 355 // ... more message declarations ... 356 }; 357 358 .. important:: 359 A form of reference counting is `always` used internally by IPDL to make 360 sure that it and its clients never address an actor the other component 361 deleted but this becomes fragile, and sometimes fails, when the client code 362 does not respect the reference count. For example, when IPDL detects that 363 a connection died due to a crashed remote process, deleting the actor could 364 leave dangling pointers, so IPDL `cannot` delete it. On the other hand, 365 there are many cases where IPDL is the only entity to have references to 366 some actors (this is very common for one side of a managed actor) so IPDL 367 `must` delete it. If all of those objects were reference counted then 368 there would be no complexity here. Indeed, new actors using 369 ``[ManualDealloc]`` should not be approved without a very compelling 370 reason. New ``[ManualDealloc]`` actors may soon be forbidden. 371 372 The ``sync`` keyword tells IPDL that the actor contains messages that block the 373 sender using ``sync`` blocking, so the sending thread waits for a response to 374 the message. There is more on what it and the other blocking modes mean in 375 `IPDL messages`_. For now, just know that this is redundant information whose 376 value is primarily in making it easy for other developers to know that there 377 are ``sync`` messages defined here. This list gives preliminary definitions of 378 the options for the actor-blocking policy of messages: 379 380 ======================= ======================================================= 381 ``async`` Actor may contain only asynchronous messages. 382 ``sync`` Actor has ``async`` capabilities and adds ``sync`` 383 messages. ``sync`` messages 384 can only be sent from the child actor to the parent. 385 ======================= ======================================================= 386 387 Beyond these protocol blocking strategies, IPDL supports annotations that 388 indicate the actor has messages that may be received in an order other than 389 the one they were sent in. These orderings attempt to handle messages in 390 "message thread" order (as in e.g. mailing lists). These behaviors can be 391 difficult to design for. Their use is discouraged but is sometimes warranted. 392 They will be discussed further in `Nested messages`_. 393 394 ============================== ================================================ 395 ``[NestedUpTo=inside_sync]`` Actor has high priority messages that can be 396 handled while waiting for a ``sync`` response. 397 ``[NestedUpTo=inside_cpow]`` Actor has the highest priority messages that 398 can be handled while waiting for a ``sync`` 399 response. 400 ============================== ================================================ 401 402 In addition, top-level protocols are annotated with which processes each side 403 should be bound into using the ``[ParentProc=*]`` and ``[ChildProc=*]`` 404 attributes. The ``[ParentProc]`` attribute is optional, and defaults to the 405 ``Parent`` process. The ``[ChildProc]`` attribute is required. See `Process 406 Type Attributes`_ for possible values. 407 408 The ``manages`` clause tells IPDL that ``PMyManager`` manages the 409 ``PMyManaged`` actor that was previously ``include`` d. As with any managed 410 protocol, it must also be the case that ``PMyManaged.ipdl`` includes 411 ``PMyManager`` and declares that ``PMyManaged`` is ``managed`` by 412 ``PMyManager``. Recalling the code: 413 414 .. code-block:: cpp 415 416 // PMyManaged.ipdl 417 include protocol PMyManager; 418 // ... 419 420 protocol PMyManaged { 421 manager PMyManager; 422 // ... 423 }; 424 425 An actor has a ``manager`` (e.g. ``PMyManaged``) or else it is a top-level 426 actor (e.g. ``PMyManager``). An actor protocol may be managed by more than one 427 actor type. For example, ``PMyManaged`` could have also been managed by some 428 ``PMyOtherManager`` not shown here. In that case, ``manager`` s are presented 429 in a list, separated by ``or`` -- e.g. ``manager PMyManager or 430 PMyOtherManager``. Of course, an **instance** of a managed actor type has only 431 one manager actor (and is therefore managed by only one of the types of 432 manager). The manager of an instance of a managee is always the actor that 433 constructed that managee. 434 435 Finally, there is the message declaration ``async PMyManaged()``. This message 436 is a constructor for ``MyManaged`` actors; unlike C++ classes, it is found in 437 ``MyManager``. Every manager will need to expose constructors to create its 438 managed types. These constructors are the only way to create an actor that is 439 managed. They can take parameters and return results, like normal messages. 440 The implementation of IPDL constructors are discussed in `Actor Lifetimes in 441 C++`_. 442 443 We haven't discussed a way to construct new top level actors. This is a more 444 advanced topic and is covered separately in `Top Level Actors`_. 445 446 .. _IPDL messages: `Declaring IPDL Messages`_ 447 448 Declaring IPDL Messages 449 ~~~~~~~~~~~~~~~~~~~~~~~ 450 451 The final part of the actor definition is the declaration of messages: 452 453 .. code-block:: cpp 454 455 sync protocol PMyManager { 456 // ... 457 parent: 458 async __delete__(nsString aNote); 459 sync SomeMsg(MyActorPair? aActors, MyData[] aMyData) 460 returns (int32_t x, int32_t y, MyUnion aUnion); 461 async PMyManaged(); 462 both: 463 [Tainted] async AnotherMsg(MyActorEnum aEnum, int32_t a number) 464 returns (MyOtherData aOtherData); 465 }; 466 467 The messages are grouped into blocks by ``parent:``, ``child:`` and ``both:``. 468 These labels work the way ``public:`` and ``private:`` work in C++ -- messages 469 after these descriptors are sent/received (only) in the direction specified. 470 471 .. note:: 472 As a mnemonic to remember which direction they indicate, remember to put 473 the word "to" in front of them. So, for example, ``parent:`` precedes 474 ``__delete__``, meaning ``__delete__`` is sent from the child **to** the 475 parent, and ``both:`` states that ``AnotherMsg`` can be sent **to** either 476 endpoint. 477 478 IPDL messages support the following annotations: 479 480 ======================== ====================================================== 481 ``[Compress]`` Indicates repeated messages of this type will 482 consolidate. 483 ``[Tainted]`` Parameters are required to be validated before using 484 them. 485 ``[Priority=Foo]`` Priority of ``MessageTask`` that runs the C++ message 486 handler. ``Foo`` is one of: ``low``, ``normal``, 487 ``mediumhigh``, ``input``, ``vsync``, or ``control``. 488 See the ``IPC::Message::PriorityValue`` enum. 489 ``[Nested=inside_sync]`` Indicates that the message can sometimes be handled 490 while a sync message waits for a response. 491 ``[Nested=inside_cpow]`` Indicates that the message can sometimes be handled 492 while a sync message waits for a response. 493 ``[LazySend]`` Messages with this annotation will be queued up to be 494 sent together either immediately before a non-LazySend 495 message, or from a direct task. 496 ======================== ====================================================== 497 498 ``[Compress]`` provides crude protection against spamming with a flood of 499 messages. When messages of type ``M`` are compressed, the queue of unprocessed 500 messages between actors will never contain an ``M`` beside another one; they 501 will always be separated by a message of a different type. This is achieved by 502 throwing out the older of the two messages if sending the new one would break 503 the rule. This has been used to throttle pointer events between the main and 504 content processes. 505 506 ``[Compress=all]`` is similar but applies whether or not the messages are 507 adjacent in the message queue. 508 509 ``[Tainted]`` is a C++ mechanism designed to encourage paying attentiton to 510 parameter security. The values of tainted parameters cannot be used until you 511 vouch for their safety. They are discussed in `Actors and Messages in C++`_. 512 513 The ``Nested`` annotations are deeply related to the message's blocking policy 514 that follows it and which was briefly discussed in `Defining Actors`_. See 515 `Nested messages`_ for details. 516 517 ``[LazySend]`` indicates the message doesn't need to be sent immediately, and 518 can be sent later, from a direct task. Worker threads which do not support 519 direct task dispatch will ignore this attribute. Messages with this annotation 520 will still be delivered in-order with other messages, meaning that if a normal 521 message is sent, any queued ``[LazySend]`` messages will be sent first. The 522 attribute allows the transport layer to combine messages to be sent together, 523 potentially reducing thread wake-ups for I/O and receiving threads. 524 525 The following is a complete list of the available blocking policies. It 526 resembles the list in `Defining Actors`_: 527 528 ====================== ======================================================== 529 ``async`` Actor may contain only asynchronous messages. 530 ``sync`` Actor has ``async`` capabilities and adds ``sync`` 531 messages. ``sync`` messages can only be sent from the 532 child actor to the parent. 533 ====================== ======================================================== 534 535 The policy defines whether an actor will wait for a response when it sends a 536 certain type of message. A ``sync`` actor will wait immediately after sending 537 a ``sync`` message, stalling its thread, until a response is received. This is 538 an easy source of browser stalls. It is rarely required that a message be 539 synchronous. New ``sync`` messages are therefore required to get approval from 540 an IPC peer. The IPDL compiler will require such messages to be listed in the 541 file ``sync-messages.ini``. 542 543 The notion that only child actors can send ``sync`` messages was introduced to 544 avoid potential deadlocks. It relies on the belief that a cycle (deadlock) of 545 sync messages is impossible because they all point in one direction. This is 546 no longer the case because any endpoint can be a child `or` parent and some, 547 like the main process, sometimes serve as both. This means that sync messages 548 should be used with extreme care. 549 550 .. note:: 551 The notion of sync messages flowing in one direction is still the main 552 mechanism IPDL uses to avoid deadlock. New actors should avoid violating 553 this rule as the consequences are severe (and complex). Actors that break 554 these rules should not be approved without **extreme** extenuating 555 circumstances. If you think you need this, check with the IPC team on 556 Element first (#ipc). 557 558 An ``async`` actor will not wait. An ``async`` response is essentially 559 identical to sending another ``async`` message back. It may be handled 560 whenever received messages are handled. The value over an ``async`` response 561 message comes in the ergonomics -- async responses are usually handled by C++ 562 lambda functions that are more like continuations than methods. This makes 563 them easier to write and to read. Additionally, they allow a response to 564 return message failure, while there would be no such response if we were 565 expecting to send a new async message back, and it failed. 566 567 Following synchronization is the name of the message and its parameter list. 568 The message ``__delete__`` stands out as strange -- indeed, it terminates the 569 actor's connection. `It does not delete any actor objects itself!` It severs 570 the connections of the actor `and any actors it manages` at both endpoints. An 571 actor will never send or receive any messages after it sends or receives a 572 ``__delete__``. Note that all sends and receives have to happen on a specific 573 *worker* thread for any actor tree so the send/receive order is well defined. 574 Anything sent after the actor processes ``__delete__`` is ignored (send returns 575 an error, messages yet to be received fail their delivery). In other words, 576 some future operations may fail but no unexpected behavior is possible. 577 578 In our example, the child can break the connection by sending ``__delete__`` to 579 the parent. The only thing the parent can do to sever the connection is to 580 fail, such as by crashing. This sort of unidirectional control is both common 581 and desirable. 582 583 ``PMyManaged()`` is a managed actor constructor. Note the asymmetry -- an 584 actor contains its managed actor's constructors but its own destructor. 585 586 The list of parameters to a message is fairly straight-forward. Parameters 587 can be any type that has a C++ ``ParamTraits`` specialization and is imported 588 by a directive. That said, there are some surprises in the list of messages: 589 590 ================= ============================================================= 591 ``int32_t``,... The standard primitive types are included. See `builtin.py`_ 592 for a list. Pointer types are, unsurprisingly, forbidden. 593 ``?`` When following a type T, the parameter is translated into 594 ``Maybe<T>`` in C++. 595 ``[]`` When following a type T, the parameter is translated into 596 ``nsTArray<T>`` in C++. 597 ================= ============================================================= 598 599 Finally, the returns list declares the information sent in response, also as a 600 tuple of typed parameters. As previously mentioned, even ``async`` messages 601 can receive responses. A ``sync`` message will always wait for a response but 602 an ``async`` message will not get one unless it has a ``returns`` clause. 603 604 This concludes our tour of the IPDL example file. The connection to C++ is 605 discussed in the next chapter; messages in particular are covered in `Actors 606 and Messages in C++`_. For suggestions on best practices when designing your 607 IPDL actor approach, see `IPDL Best Practices`_. 608 609 .. _builtin.py: https://searchfox.org/mozilla-central/source/ipc/ipdl/ipdl/builtin.py 610 611 IPDL Syntax Quick Reference 612 ~~~~~~~~~~~~~~~~~~~~~~~~~~~ 613 614 The following is a list of the keywords and operators that have been introduced 615 for use in IPDL files: 616 617 ============================= ================================================= 618 ``include`` Include a C++ header (quoted file name) or 619 ``.ipdlh`` file (unquoted with no file suffix). 620 ``using (class|struct) from`` Similar to ``include`` but imports only a 621 specific data type. 622 ``include protocol`` Include another actor for use in management 623 statements, IPDL data types or as parameters to 624 messages. 625 ``[RefCounted]`` Indicates that the imported C++ data types are 626 reference counted. Refcounted types require a 627 different ``ParamTraits`` interface than 628 non-reference-counted types. 629 ``[ManualDealloc]`` Indicates that the IPDL interface uses the legacy 630 manual allocation/deallocation interface, rather 631 than modern reference counting. 632 ``[MoveOnly]`` Indicates that an imported C++ data type should 633 not be copied. IPDL code will move it instead. 634 ``namespace`` Specifies the namespace for IPDL generated code. 635 ``union`` An IPDL union definition. 636 ``struct`` An IPDL struct definition. 637 ``[Comparable]`` Indicates that IPDL should generate 638 ``operator==`` and ``operator!=`` for the given 639 IPDL struct/union. 640 ``nullable`` Indicates that an actor reference in an IPDL type 641 may be null when sent over IPC. 642 ``protocol`` An IPDL protocol (actor) definition. 643 ``sync/async`` These are used in two cases: (1) to indicate 644 whether a message blocks as it waits for a result 645 and (2) because an actor that contains ``sync`` 646 messages must itself be labeled ``sync``. 647 ``[NestedUpTo=inside_sync]`` Indicates that an actor contains 648 [Nested=inside_sync] messages, in addition to 649 normal messages. 650 ``[NestedUpTo=inside_cpow]`` Indicates that an actor contains 651 [Nested=inside_cpow] messages, in addition to 652 normal messages. 653 ``[Nested=inside_sync]`` Indicates that the message can be handled while 654 waiting for lower-priority, or in-message-thread, 655 sync responses. 656 ``[Nested=inside_cpow]`` Indicates that the message can be handled while 657 waiting for lower-priority, or in-message-thread, 658 sync responses. Cannot be sent by the parent 659 actor. 660 ``manager`` Used in a protocol definition to indicate that 661 this actor manages another one. 662 ``manages`` Used in a protocol definition to indicate that 663 this actor is managed by another one. 664 ``or`` Used in a ``manager`` clause for actors that have 665 multiple potential managers. 666 ``parent: / child: / both:`` Indicates direction of subsequent actor messages. 667 As a mnemonic to remember which direction they 668 indicate, put the word "to" in front of them. 669 ``returns`` Defines return values for messages. All types 670 of message, including ``async``, support 671 returning values. 672 ``__delete__`` A special message that destroys the related 673 actors at both endpoints when sent. 674 ``Recv__delete__`` and ``ActorDestroy`` are 675 called before destroying the actor at the other 676 endpoint, to allow for cleanup. 677 ``int32_t``,... The standard primitive types are included. 678 ``String`` Translated into ``nsString`` in C++. 679 ``?`` When following a type T in an IPDL data structure 680 or message parameter, 681 the parameter is translated into ``Maybe<T>`` in 682 C++. 683 ``[]`` When following a type T in an IPDL data structure 684 or message parameter, 685 the parameter is translated into ``nsTArray<T>`` 686 in C++. 687 ``[Tainted]`` Used to indicate that a message's handler should 688 receive parameters that it is required to 689 manually validate. Parameters of type ``T`` 690 become ``Tainted<T>`` in C++. 691 ``[Compress]`` Indicates repeated messages of this type will 692 consolidate. When two messages of this type are 693 sent and end up side-by-side in the message queue 694 then the older message is discarded (not sent). 695 ``[Compress=all]`` Like ``[Compress]`` but discards the older 696 message regardless of whether they are adjacent 697 in the message queue. 698 ``[Priority=Foo]`` Priority of ``MessageTask`` that runs the C++ 699 message handler. ``Foo`` is one of: ``low``, 700 ``normal``, ``mediumhigh``, ``input``, ``vsync``, 701 or ``control``. 702 ``[LazySend]`` Messages with this annotation will be queued up to 703 be sent together immediately before a non-LazySend 704 message, or from a direct task. 705 ``[ChildImpl="RemoteFoo"]`` Indicates that the child side implementation of 706 the actor is a class named ``RemoteFoo``, and the 707 definition is included by one of the 708 ``include "...";`` statements in the file. 709 *New uses of this attribute are discouraged.* 710 ``[ParentImpl="FooImpl"]`` Indicates that the parent side implementation of 711 the actor is a class named ``FooImpl``, and the 712 definition is included by one of the 713 ``include "...";`` statements in the file. 714 *New uses of this attribute are discouraged.* 715 ``[ChildImpl=virtual]`` Indicates that the child side implementation of 716 the actor is not exported by a header, so virtual 717 ``Recv`` methods should be used instead of direct 718 function calls. *New uses of this attribute are 719 discouraged.* 720 ``[ParentImpl=virtual]`` Indicates that the parent side implementation of 721 the actor is not exported by a header, so virtual 722 ``Recv`` methods should be used instead of direct 723 function calls. *New uses of this attribute are 724 discouraged.* 725 ``[ChildProc=...]`` Indicates which process the child side of the actor 726 is expected to be bound in. This will be release 727 asserted when creating the actor. Required for 728 top-level actors. See `Process Type Attributes`_ 729 for possible values. 730 ``[ParentProc=...]`` Indicates which process the parent side of the 731 actor is expected to be bound in. This will be 732 release asserted when creating the actor. 733 Defaults to ``Parent`` for top-level actors. See 734 `Process Type Attributes`_ for possible values. 735 ============================= ================================================= 736 737 .. _Process Type Attributes: 738 739 Process Type Attributes 740 ^^^^^^^^^^^^^^^^^^^^^^^ 741 742 The following are valid values for the ``[ChildProc=...]`` and 743 ``[ParentProc=...]`` attributes on protocols, each corresponding to a specific 744 process type: 745 746 ============================= ================================================= 747 ``Parent`` The primary "parent" or "main" process 748 ``Content`` A content process, such as those used to host web 749 pages, workers, and extensions 750 ``IPDLUnitTest`` Test-only process used in IPDL gtests 751 ``GMPlugin`` Gecko Media Plugin (GMP) process 752 ``GPU`` GPU process 753 ``VR`` VR process 754 ``RDD`` Remote Data Decoder (RDD) process 755 ``Socket`` Socket/Networking process 756 ``ForkServer`` Fork Server process 757 ``Utility`` Utility process 758 ============================= ================================================= 759 760 The attributes also support some wildcard values, which can be used when an 761 actor can be bound in multiple processes. If you are adding an actor which 762 needs a new wildcard value, please reach out to the IPC team, and we can add one 763 for your use-case. They are as follows: 764 765 ============================= ================================================= 766 ``any`` Any process. If a more specific value is 767 applicable, it should be preferred where possible. 768 ``anychild`` Any process other than ``Parent``. Often used for 769 utility actors which are bound on a per-process 770 basis, such as profiling. 771 ``compositor`` Either the ``GPU`` or ``Parent`` process. Often 772 used for actors bound to the compositor thread. 773 ``anydom`` Either the ``Parent`` or a ``Content`` process. 774 Often used for actors used to implement DOM APIs. 775 ============================= ================================================= 776 777 Note that these assertions do not provide security guarantees, and are primarily 778 intended for use when auditing and as documentation for how actors are being 779 used. 780 781 782 The C++ Interface 783 ----------------- 784 785 ParamTraits 786 ~~~~~~~~~~~ 787 788 Before discussing how C++ represents actors and messages, we look at how IPDL 789 connects to the imported C++ data types. In order for any C++ type to be 790 (de)serialized, it needs an implementation of the ``ParamTraits`` C++ type 791 class. ``ParamTraits`` is how your code tells IPDL what bytes to write to 792 serialize your objects for sending, and how to convert those bytes back to 793 objects at the other endpoint. Since ``ParamTraits`` need to be reachable by 794 IPDL code, they need to be declared in a C++ header and imported by your 795 protocol file. Failure to do so will result in a build error. 796 797 Most basic types and many essential Mozilla types are always available for use 798 without inclusion. An incomplete list includes: C++ primitives, strings 799 (``std`` and ``mozilla``), vectors (``std`` and ``mozilla``), ``RefPtr<T>`` 800 (for serializable ``T``), ``UniquePtr<T>``, ``nsCOMPtr<T>``, ``nsTArray<T>``, 801 ``std::unordered_map<T>``, ``nsresult``, etc. See `builtin.py 802 <https://searchfox.org/mozilla-central/source/ipc/ipdl/ipdl/builtin.py>`_, 803 `ipc_message_utils.h 804 <https://searchfox.org/mozilla-central/source/ipc/chromium/src/chrome/common/ipc_message_utils.h>`_ 805 and `IPCMessageUtilsSpecializations.h 806 <https://searchfox.org/mozilla-central/source/ipc/glue/IPCMessageUtilsSpecializations.h>`_. 807 808 ``ParamTraits`` typically bootstrap with the ``ParamTraits`` of more basic 809 types, until they hit bedrock (e.g. one of the basic types above). In the most 810 extreme cases, a ``ParamTraits`` author may have to resort to designing a 811 binary data format for a type. Both options are available. 812 813 We haven't seen any of this C++ yet. Let's look at the data types included 814 from ``MyDataTypes.h``: 815 816 .. code-block:: cpp 817 818 // MyDataTypes.h 819 namespace mozilla::myns { 820 struct MyData { 821 nsCString s; 822 uint8_t bytes[17]; 823 MyData(); // IPDL requires the default constructor to be public 824 }; 825 826 struct MoveonlyData { 827 MoveonlyData(); 828 MoveonlyData& operator=(const MoveonlyData&) = delete; 829 830 MoveonlyData(MoveonlyData&& m); 831 MoveonlyData& operator=(MoveonlyData&& m); 832 }; 833 834 typedef MoveonlyData MyOtherData; 835 836 class MyUnusedData { 837 public: 838 NS_INLINE_DECL_REFCOUNTING(MyUnusedData) 839 int x; 840 }; 841 }; 842 843 namespace IPC { 844 // Basic type 845 template<> 846 struct ParamTraits<mozilla::myns::MyData> { 847 typedef mozilla::myns::MyData paramType; 848 static void Write(MessageWriter* m, const paramType& in); 849 static bool Read(MessageReader* m, paramType* out); 850 }; 851 852 // [MoveOnly] type 853 template<> 854 struct ParamTraits<mozilla::myns::MyOtherData> { 855 typedef mozilla::myns::MyOtherData paramType; 856 static void Write(MessageWriter* m, const paramType& in); 857 static bool Read(MessageReader* m, paramType* out); 858 }; 859 860 // [RefCounted] type 861 template<> 862 struct ParamTraits<mozilla::myns::MyUnusedData*> { 863 typedef mozilla::myns::MyUnusedData paramType; 864 static void Write(MessageWriter* m, paramType* in); 865 static bool Read(MessageReader* m, RefPtr<paramType>* out); 866 }; 867 } 868 869 MyData is a struct and MyOtherData is a typedef. IPDL is fine with both. 870 Additionally, MyOtherData is not copyable, matching its IPDL ``[MoveOnly]`` 871 annotation. 872 873 ``ParamTraits`` are required to be defined in the ``IPC`` namespace. They must 874 contain a ``Write`` method with the proper signature that is used for 875 serialization and a ``Read`` method, again with the correct signature, for 876 deserialization. 877 878 Here we have three examples of declarations: one for an unannotated type, one 879 for ``[MoveOnly]`` and a ``[RefCounted]`` one. Notice the difference in the 880 ``[RefCounted]`` type's method signatures. The only difference that may not be 881 clear from the function types is that, in the non-reference-counted case, a 882 default-constructed object is supplied to ``Read`` but, in the 883 reference-counted case, ``Read`` is given an empty ``RefPtr<MyUnusedData>`` and 884 should only allocate a ``MyUnusedData`` to return if it so desires. 885 886 These are straight-forward implementations of the ``ParamTraits`` methods for 887 ``MyData``: 888 889 .. code-block:: cpp 890 891 /* static */ void IPC::ParamTraits<MyData>::Write(MessageWriter* m, const paramType& in) { 892 WriteParam(m, in.s); 893 m->WriteBytes(in.bytes, sizeof(in.bytes)); 894 } 895 /* static */ bool IPC::ParamTraits<MyData>::Read(MessageReader* m, paramType* out) { 896 return ReadParam(m, &out->s) && 897 m->ReadBytesInto(out->bytes, sizeof(out->bytes)); 898 } 899 900 ``WriteParam`` and ``ReadParam`` call the ``ParamTraits`` for the data you pass 901 them, determined using the type of the object as supplied. ``WriteBytes`` and 902 ``ReadBytesInto`` work on raw, contiguous bytes as expected. ``MessageWriter`` 903 and ``MessageReader`` are IPDL internal objects which hold the incoming/outgoing 904 message as a stream of bytes and the current spot in the stream. It is *very* 905 rare for client code to need to create or manipulate these objects. Their 906 advanced use is beyond the scope of this document. 907 908 .. important:: 909 Potential failures in ``Read`` include everyday C++ failures like 910 out-of-memory conditions, which can be handled as usual. But ``Read`` can 911 also fail due to things like data validation errors. ``ParamTraits`` read 912 data that is considered insecure. It is important that they catch 913 corruption and properly handle it. Returning false from ``Read`` will 914 usually result in crashing the process (everywhere except in the main 915 process). This is the right behavior as the browser would be in an 916 unexpected state, even if the serialization failure was not malicious 917 (since it cannot process the message). Other responses, such as failing 918 with a crashing assertion, are inferior. IPDL fuzzing relies on 919 ``ParamTraits`` not crashing due to corruption failures. 920 Occasionally, validation will require access to state that ``ParamTraits`` 921 can't easily reach. (Only) in those cases, validation can be reasonably 922 done in the message handler. Such cases are a good use of the ``Tainted`` 923 annotation. See `Actors and Messages in C++`_ for more. 924 925 .. note:: 926 If you need to access the actor object during serialization, use 927 ``IPC::Message{Reader,Writer}::GetActor()``. This should be null-checked, as 928 it may not be set if an actor is unavailable. 929 930 A special case worth mentioning is that of enums. Enums are a common source of 931 security holes since code is rarely safe with enum values that are not valid. 932 Since data obtained through IPDL messages should be considered tainted, enums 933 are of principal concern. ``ContiguousEnumSerializer`` and 934 ``ContiguousEnumSerializerInclusive`` safely implement ``ParamTraits`` for 935 enums that are only valid for a contiguous set of values, which is most of 936 them. The generated ``ParamTraits`` confirm that the enum is in valid range; 937 ``Read`` will return false otherwise. As an example, here is the 938 ``MyActorEnum`` included from ``MyActorUtils.h``: 939 940 .. code-block:: cpp 941 942 enum MyActorEnum { e1, e2, e3, e4, e5 }; 943 944 template<> 945 struct ParamTraits<MyActorEnum> 946 : public ContiguousEnumSerializerInclusive<MyActorEnum, MyActorEnum::e1, MyActorEnum::e5> {}; 947 948 IPDL Structs and Unions in C++ 949 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 950 951 IPDL structs and unions become C++ classes that provide interfaces that are 952 fairly self-explanatory. Recalling ``MyUnion`` and ``MyActorPair`` from 953 `IPDL Structs and Unions`_ : 954 955 .. code-block:: cpp 956 957 union MyUnion { 958 float; 959 MyOtherData; 960 }; 961 962 struct MyActorPair { 963 PMyManaged actor1; 964 nullable PMyManaged actor2; 965 }; 966 967 These compile to: 968 969 .. code-block:: cpp 970 971 class MyUnion { 972 enum Type { Tfloat, TMyOtherData }; 973 Type type(); 974 MyUnion(float f); 975 MyUnion(MyOtherData&& aOD); 976 MyUnion& operator=(float f); 977 MyUnion& operator=(MyOtherData&& aOD); 978 operator float&(); 979 operator MyOtherData&(); 980 }; 981 982 class MyActorPair { 983 MyActorPair(PMyManagedParent* actor1Parent, PMyManagedChild* actor1Child, 984 PMyManagedParent* actor2Parent, PMyManagedChild* actor2Child); 985 // Exactly one of { actor1Parent(), actor1Child() } must be non-null. 986 PMyManagedParent*& actor1Parent(); 987 PMyManagedChild*& actor1Child(); 988 // As nullable, zero or one of { actor2Parent(), actor2Child() } will be non-null. 989 PMyManagedParent*& actor2Parent(); 990 PMyManagedChild*& actor2Child(); 991 } 992 993 The generated ``ParamTraits`` use the ``ParamTraits`` for the types referenced 994 by the IPDL struct or union. Fields respect any annotations for their type 995 (see `IPDL Includes`_). For example, a ``[RefCounted]`` type ``T`` generates 996 ``RefPtr<T>`` fields. 997 998 Note that actor members result in members of both the parent and child actor 999 types, as seen in ``MyActorPair``. When actors are used to bridge processes, 1000 only one of those could ever be used at a given endpoint. IPDL makes sure 1001 that, when you send one type (say, ``PMyManagedChild``), the adjacent actor of 1002 the other type (``PMyManagedParent``) is received. This is not only true for 1003 message parameters and IPDL structs/unions but also for custom ``ParamTraits`` 1004 implementations. If you ``Write`` a ``PFooParent*`` then you must ``Read`` a 1005 ``PFooChild*``. This is hard to confuse in message handlers since they are 1006 members of a class named for the side they operate on, but this cannot be 1007 enforced by the compiler. If you are writing 1008 ``MyManagerParent::RecvSomeMsg(Maybe<MyActorPair>&& aActors, nsTArray<MyData>&& aMyData)`` 1009 then the ``actor1Child`` and ``actor2Child`` fields cannot be valid since the 1010 child (usually) exists in another process. 1011 1012 .. _IPDL Structs and Unions: `Generating IPDL-Aware C++ Data Types: IPDL Structs and Unions`_ 1013 .. _IPDL Includes: `Referencing Externally Defined Data Types: IPDL Includes`_ 1014 1015 Actors and Messages in C++ 1016 ~~~~~~~~~~~~~~~~~~~~~~~~~~ 1017 1018 As mentioned in `Using The IPDL compiler`_, the IPDL compiler generates two 1019 header files for the protocol ``PMyManager``: ``PMyManagerParent.h`` and 1020 ``PMyManagerChild.h``, which declare the actor's base classes. There, we 1021 discussed how the headers are visible to C++ components that include 1022 ``chromium-config.mozbuild``. We, in turn, always need to define two files 1023 that declare our actor implementation subclasses (``MyManagerParent.h`` and 1024 ``MyManagerChild.h``). The IPDL file looked like this: 1025 1026 .. literalinclude:: _static/PMyManager.ipdl 1027 :language: c++ 1028 :name: PMyManager.ipdl 1029 1030 So ``MyManagerParent.h`` looks like this: 1031 1032 .. code-block:: cpp 1033 1034 #include "PMyManagerParent.h" 1035 1036 namespace mozilla { 1037 namespace myns { 1038 1039 class MyManagerParent : public PMyManagerParent { 1040 NS_INLINE_DECL_REFCOUNTING(MyManagerParent, override) 1041 protected: 1042 IPCResult Recv__delete__(const nsString& aNote); 1043 IPCResult RecvSomeMsg(const Maybe<MyActorPair>& aActors, const nsTArray<MyData>& aMyData, 1044 int32_t* x, int32_t* y, MyUnion* aUnion); 1045 IPCResult RecvAnotherMsg(const Tainted<MyActorEnum>& aEnum, const Tainted<int32_t>& a number, 1046 AnotherMsgResolver&& aResolver); 1047 1048 already_AddRefed<PMyManagerParent> AllocPMyManagedParent(); 1049 IPCResult RecvPMyManagedConstructor(PMyManagedConstructor* aActor); 1050 1051 // ... etc ... 1052 }; 1053 1054 } // namespace myns 1055 } // namespace mozilla 1056 1057 All messages that can be sent to the actor must be handled by ``Recv`` methods 1058 in the proper actor subclass. They should return ``IPC_OK()`` on success and 1059 ``IPC_FAIL(actor, reason)`` if an error occurred (where ``actor`` is ``this`` 1060 and ``reason`` is a human text explanation) that should be considered a failure 1061 to process the message. The handling of such a failure is specific to the 1062 process type. 1063 1064 ``Recv`` methods are called by IPDL by enqueueing a task to run them on the 1065 ``MessageLoop`` for the thread on which they are bound. This thread is the 1066 actor's *worker thread*. All actors in a managed actor tree have the same 1067 worker thread -- in other words, actors inherit the worker thread from their 1068 managers. Top level actors establish their worker thread when they are 1069 *bound*. More information on threads can be found in `Top Level Actors`_. For 1070 the most part, client code will never engage with an IPDL actor outside of its 1071 worker thread. 1072 1073 Received parameters become stack variables that are ``std::move``-d into the 1074 ``Recv`` method. They can be received as a const l-value reference, 1075 rvalue-reference, or by value (type-permitting). ``[MoveOnly]`` types should 1076 not be received as const l-values. Return values for sync messages are 1077 assigned by writing to non-const (pointer) parameters. Return values for async 1078 messages are handled differently -- they are passed to a resolver function. In 1079 our example, ``AnotherMsgResolver`` would be a ``std::function<>`` and 1080 ``aResolver`` would be given the value to return by passing it a reference to a 1081 ``MyOtherData`` object. 1082 1083 ``MyManagerParent`` is also capable of ``sending`` an async message that 1084 returns a value: ``AnotherMsg``. This is done with ``SendAnotherMsg``, which 1085 is defined automatically by IPDL in the base class ``PMyManagerParent``. There 1086 are two signatures for ``Send`` and they look like this: 1087 1088 .. code-block:: cpp 1089 1090 // Return a Promise that IPDL will resolve with the response or reject. 1091 RefPtr<MozPromise<MyOtherData, ResponseRejectReason, true>> 1092 SendAnotherMsg(const MyActorEnum& aEnum, int32_t a number); 1093 1094 // Provide callbacks to process response / reject. The callbacks are just 1095 // std::functions. 1096 void SendAnotherMsg(const MyActorEnum& aEnum, int32_t a number, 1097 ResolveCallback<MyOtherData>&& aResolve, RejectCallback&& aReject); 1098 1099 The response is usually handled by lambda functions defined at the site of the 1100 ``Send`` call, either by attaching them to the returned promise with e.g. 1101 ``MozPromise::Then``, or by passing them as callback parameters. See docs on 1102 ``MozPromise`` for more on its use. The promise itself is either resolved or 1103 rejected by IPDL when a valid reply is received or when the endpoint determines 1104 that the communication failed. ``ResponseRejectReason`` is an enum IPDL 1105 provides to explain failures. 1106 1107 Additionally, the ``AnotherMsg`` handler has ``Tainted`` parameters, as a 1108 result of the [Tainted] annotation in the protocol file. Recall that 1109 ``Tainted`` is used to force explicit validation of parameters in the message 1110 handler before their values can be used (as opposed to validation in 1111 ``ParamTraits``). They therefore have access to any state that the message 1112 handler does. Their APIs, along with a list of macros that are used to 1113 validate them, are detailed `here 1114 <https://searchfox.org/mozilla-central/source/mfbt/Tainting.h>`__. 1115 1116 Send methods that are not for async messages with return values follow a 1117 simpler form; they return a ``bool`` indicating success or failure and return 1118 response values in non-const parameters, as the ``Recv`` methods do. For 1119 example, ``PMyManagerChild`` defines this to send the sync message ``SomeMsg``: 1120 1121 .. code-block:: cpp 1122 1123 // generated in PMyManagerChild 1124 bool SendSomeMsg(const Maybe<MyActorPair>& aActors, const nsTArray<MyData>& aMyData, 1125 int32_t& x, int32_t& y, MyUnion& aUnion); 1126 1127 Since it is sync, this method will not return to its caller until the response 1128 is received or an error is detected. 1129 1130 All calls to ``Send`` methods, like all messages handler ``Recv`` methods, must 1131 only be called on the worker thread for the actor. 1132 1133 Constructors, like the one for ``MyManaged``, are clearly an exception to these 1134 rules. They are discussed in the next section. 1135 1136 .. _Actor Lifetimes in C++: 1137 1138 Actor Lifetimes in C++ 1139 ~~~~~~~~~~~~~~~~~~~~~~ 1140 1141 The constructor message for ``MyManaged`` becomes *two* methods at the 1142 receiving end. ``AllocPMyManagedParent`` constructs the managed actor, then 1143 ``RecvPMyManagedConstructor`` is called to update the new actor. The following 1144 diagram shows the construction of the ``MyManaged`` actor pair: 1145 1146 .. mermaid:: 1147 :align: center 1148 :caption: A ``MyManaged`` actor pair being created by some ``Driver`` 1149 object. Internal IPC objects in the parent and child processes 1150 are combined for compactness. Connected **par** blocks run 1151 concurrently. This shows that messages can be safely sent while 1152 the parent is still being constructed. 1153 1154 %%{init: {'sequence': {'boxMargin': 4, 'actorMargin': 10} }}%% 1155 sequenceDiagram 1156 participant d as Driver 1157 participant mgdc as MyManagedChild 1158 participant mgrc as MyManagerChild 1159 participant ipc as IPC Child/Parent 1160 participant mgrp as MyManagerParent 1161 participant mgdp as MyManagedParent 1162 d->>mgdc: new 1163 mgdc->>d: [mgd_child] 1164 d->>mgrc: SendPMyManagedConstructor<br/>[mgd_child, params] 1165 mgrc->>ipc: Form actor pair<br/>[mgd_child, params] 1166 par 1167 mgdc->>ipc: early PMyManaged messages 1168 and 1169 ipc->>mgrp: AllocPMyManagedParent<br/>[params] 1170 mgrp->>mgdp: new 1171 mgdp->>mgrp: [mgd_parent] 1172 ipc->>mgrp: RecvPMyManagedConstructor<br/>[mgd_parent, params] 1173 mgrp->>mgdp: initialization 1174 ipc->>mgdp: early PMyManaged messages 1175 end 1176 Note over mgdc,mgdp: Bi-directional sending and receiving will now happen concurrently. 1177 1178 The next diagram shows the destruction of the ``MyManaged`` actor pair, as 1179 initiated by a call to ``Send__delete__``. ``__delete__`` is sent from the 1180 child process because that is the only side that can call it, as declared in 1181 the IPDL protocol file. 1182 1183 .. mermaid:: 1184 :align: center 1185 :caption: A ``MyManaged`` actor pair being disconnected due to some 1186 ``Driver`` object in the child process sending ``__delete__``. 1187 1188 %%{init: {'sequence': {'boxMargin': 4, 'actorMargin': 10} }}%% 1189 sequenceDiagram 1190 participant d as Driver 1191 participant mgdc as MyManagedChild 1192 participant ipc as IPC Child/Parent 1193 participant mgdp as MyManagedParent 1194 d->>mgdc: Send__delete__ 1195 mgdc->>ipc: Disconnect<br/>actor pair 1196 par 1197 ipc->>mgdc: ActorDestroy 1198 ipc->>mgdc: Release 1199 and 1200 ipc->>mgdp: Recv__delete__ 1201 ipc->>mgdp: ActorDestroy 1202 ipc->>mgdp: Release 1203 end 1204 1205 Finally, let's take a look at the behavior of an actor whose peer has been lost 1206 (e.g. due to a crashed process). 1207 1208 .. mermaid:: 1209 :align: center 1210 :caption: A ``MyManaged`` actor pair being disconnected when its peer is 1211 lost due to a fatal error. Note that ``Recv__delete__`` is not 1212 called. 1213 1214 %%{init: {'sequence': {'boxMargin': 4, 'actorMargin': 10} }}%% 1215 sequenceDiagram 1216 participant mgdc as MyManagedChild 1217 participant ipc as IPC Child/Parent 1218 participant mgdp as MyManagedParent 1219 Note over mgdc: CRASH!!! 1220 ipc->>ipc: Notice fatal error. 1221 ipc->>mgdp: ActorDestroy 1222 ipc->>mgdp: Release 1223 1224 The ``Alloc`` and ``Recv...Constructor`` methods are somewhat mirrored by 1225 ``Recv__delete__`` and ``ActorDestroy`` but there are a few differences. 1226 First, the ``Alloc`` method really does create the actor but the 1227 ``ActorDestroy`` method does not delete it. Additionally, ``ActorDestroy`` 1228 is run at *both* endpoints, during ``Send__delete__`` or after 1229 ``Recv__delete__``. Finally and most importantly, ``Recv__delete__`` is only 1230 called if the ``__delete__`` message is received but it may not be if, for 1231 example, the remote process crashes. ``ActorDestroy``, on the other hand, is 1232 guaranteed to run for *every* actor unless the process terminates uncleanly. 1233 For this reason, ``ActorDestroy`` is the right place for most actor shutdown 1234 code. ``Recv__delete__`` is rarely useful, although it is occasionally 1235 beneficial to have it receive some final data. 1236 1237 The relevant part of the parent class looks like this: 1238 1239 .. code-block:: cpp 1240 1241 class MyManagerParent : public PMyManagerParent { 1242 already_AddRefed<PMyManagedParent> AllocPMyManagedParent(); 1243 IPCResult RecvPMyManagedConstructor(PMyManagedParent* aActor); 1244 1245 IPCResult Recv__delete__(const nsString& aNote); 1246 void ActorDestroy(ActorDestroyReason why); 1247 1248 // ... etc ... 1249 }; 1250 1251 The ``Alloc`` method is required for managed actors that are constructed by 1252 IPDL receiving a ``Send`` message. It is not required for the actor at the 1253 endpoint that calls ``Send``. The ``Recv...Constructor`` message is not 1254 required -- it has a base implementation that does nothing. 1255 1256 If the constructor message has parameters, they are sent to both methods. 1257 Parameters are given to the ``Alloc`` method by const reference but are moved 1258 into the ``Recv`` method. They differ in that messages can be sent from the 1259 ``Recv`` method but, in ``Alloc``, the newly created actor is not yet 1260 operational. 1261 1262 The ``Send`` method for a constructor is similarly different from other 1263 ``Send`` methods. In the child actor, ours looks like this: 1264 1265 .. code-block:: cpp 1266 1267 IPCResult SendPMyManagedConstructor(PMyManagedChild* aActor); 1268 1269 The method expects a ``PMyManagedChild`` that the caller will have constructed, 1270 presumably using ``new`` (this is why it does not require an ``Alloc`` method). 1271 Once ``Send...Constructor`` is called, the actor can be used to send and 1272 receive messages. It does not matter that the remote actor may not have been 1273 created yet due to asynchronicity. 1274 1275 The destruction of actors is as unusual as their construction. Unlike 1276 construction, it is the same for managed and top-level actors. Avoiding 1277 ``[ManualDealloc]`` actors removes a lot of the complexity but there is still 1278 a process to understand. Actor destruction begins when an ``__delete__`` 1279 message is sent. In ``PMyManager``, this message is declared from child to 1280 parent. The actor calling ``Send__delete__`` is no longer connected to 1281 anything when the method returns. Future calls to ``Send`` return an error 1282 and no future messages will be received. This is also the case for an actor 1283 that has run ``Recv__delete__``; it is no longer connected to the other 1284 endpoint. 1285 1286 .. note:: 1287 Since ``Send__delete__`` may release the final reference to itself, it 1288 cannot safely be a class instance method. Instead, unlike other ``Send`` 1289 methods, it's a ``static`` class method and takes the actor as a parameter: 1290 1291 .. code-block:: cpp 1292 1293 static IPCResult Send__delete__(PMyManagerChild* aToDelete); 1294 1295 Additionally, the ``__delete__`` message tells IPDL to disconnect both the 1296 given actor *and all of its managed actors*. So it is really deleting the 1297 actor subtree, although ``Recv__delete__`` is only called for the actor it 1298 was sent to. 1299 1300 During the call to ``Send__delete__``, or after the call to ``Recv__delete__``, 1301 the actor's ``ActorDestroy`` method is called. This method gives client code a 1302 chance to do any teardown that must happen in `all` circumstances were it is 1303 possible -- both expected and unexpected. This means that ``ActorDestroy`` 1304 will also be called when, for example, IPDL detects that the other endpoint has 1305 terminated unexpectedly, so it is releasing its reference to the actor, or 1306 because an ancestral manager (manager or manager's manager...) received a 1307 ``__delete__``. The only way for an actor to avoid ``ActorDestroy`` is for its 1308 process to crash first. ``ActorDestroy`` is always run after its actor is 1309 disconnected so it is pointless to try to send messages from it. 1310 1311 Why use ``ActorDestroy`` instead of the actor's destructor? ``ActorDestroy`` 1312 gives a chance to clean up things that are only used for communication and 1313 therefore don't need to live for however long the actor's (reference-counted) 1314 object does. For example, you might have references to shared memory (Shmems) 1315 that are no longer valid. Or perhaps the actor can now release a cache of data 1316 that was only needed for processing messages. It is cleaner to deal with 1317 communication-related objects in ``ActorDestroy``, where they become invalid, 1318 than to leave them in limbo until the destructor is run. 1319 1320 Consider actors to be like normal reference-counted objects, but where IPDL 1321 holds a reference while the connection will or does exist. One common 1322 architecture has IPDL holding the `only` reference to an actor. This is common 1323 with actors created by sending constructor messages but the idea is available to 1324 any actor. That only reference is then released when the ``__delete__`` 1325 message is sent or received. 1326 1327 The dual of IPDL holding the only reference is to have client code hold the 1328 only reference. A common pattern to achieve this has been to override the 1329 actor's ``AddRef`` to have it send ``__delete__`` only when it's count is down 1330 to one reference (which must be IPDL if ``actor.CanSend()`` is true). A better 1331 approach would be to create a reference-counted delegate for your actor that 1332 can send ``__delete__`` from its destructor. IPDL does not guarantee that it 1333 will not hold more than one reference to your actor. 1334 1335 .. _Top Level Actors: 1336 1337 Top Level Actors 1338 ---------------- 1339 1340 Recall that top level actors are actors that have no manager. They are at the 1341 root of every actor tree. There are two settings in which we use top-level 1342 actors that differ pretty dramatically. The first type are top-level actors 1343 that are created and maintained in a way that resembles managed actors, but 1344 with some important differences we will cover in this section. The second type 1345 of top-level actors are the very first actors in a new process -- these actors 1346 are created through different means and closing them (usually) terminates the 1347 process. The `new process example 1348 <https://phabricator.services.mozilla.com/D119038>`_ demonstrates both of 1349 these. It is discussed in detail in :ref:`Adding a New Type of Process`. 1350 1351 Value of Top Level Actors 1352 ~~~~~~~~~~~~~~~~~~~~~~~~~ 1353 1354 Top-level actors are harder to create and destroy than normal actors. They 1355 used to be more heavyweight than managed actors but this has recently been 1356 dramatically reduced. 1357 1358 .. note:: 1359 Top-level actors previously required a dedicated *message channel*, which 1360 are limited OS resources. This is no longer the case -- message channels 1361 are now shared by actors that connect the same two processes. This 1362 *message interleaving* can affect message delivery latency but profiling 1363 suggests that the change was basically inconsequential. 1364 1365 So why use a new top level actor? 1366 1367 * The most dramatic property distinguishing top-level actors is the ability to 1368 *bind* to whatever ``EventTarget`` they choose. This means that any thread 1369 that runs a ``MessageLoop`` can use the event target for that loop as the 1370 place to send incoming messages. In other words, ``Recv`` methods would be 1371 run by that message loop, on that thread. The IPDL apparatus will 1372 asynchronously dispatch messages to these event targets, meaning that 1373 multiple threads can be handling incoming messages at the same time. The 1374 `PBackground`_ approach was born of a desire to make it easier to exploit 1375 this, although it has some complications, detailed in that section, that 1376 limit its value. 1377 * Top level actors suggest modularity. Actor protocols are tough to debug, as 1378 is just about anything that spans process boundaries. Modularity can give 1379 other developers a clue as to what they need to know (and what they don't) 1380 when reading an actor's code. The alternative is proverbial *dumpster 1381 classes* that are as critical to operations (because they do so much) as they 1382 are difficult to learn (because they do so much). 1383 * Top level actors are required to connect two processes, regardless of whether 1384 the actors are the first in the process or not. As said above, the first 1385 actor is created through special means but other actors are created through 1386 messages. In Gecko, apart from the launcher and main processes, all new 1387 processes X are created with their first actor being between X and the main 1388 process. To create a connection between X and, say, a content process, the 1389 main process has to send connected ``Endpoints`` to X and to the content 1390 process, which in turn use those endpoints to create new top level actors 1391 that form an actor pair. This is discussed at length in :ref:`Connecting 1392 With Other Processes`. 1393 1394 Top-level actors are not as frictionless as desired but they are probably 1395 under-utilized relative to their power. In cases where it is supported, 1396 ``PBackground`` is sometimes a simpler alternative to achieve the same goals. 1397 1398 Creating Top Level Actors From Other Actors 1399 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1400 1401 The most common way to create new top level actors is by creating a pair of 1402 connected Endpoints and sending one to the other actor. This is done exactly 1403 the way it sounds. For example: 1404 1405 .. code-block:: cpp 1406 1407 bool MyPreexistingActorParent::MakeMyActor() { 1408 Endpoint<PMyActorParent> parentEnd; 1409 Endpoint<PMyActorChild> childEnd; 1410 if (NS_WARN_IF(NS_FAILED(PMyActor::CreateEndpoints(&parentEnd, &childEnd)))) { 1411 // ... handle failure ... 1412 return false; 1413 } 1414 RefPtr<MyActorParent> parent = new MyActorParent; 1415 if (!parentEnd.Bind(parent)) { 1416 // ... handle failure ... 1417 delete parent; 1418 return false; 1419 } 1420 // Do this second so we skip child if parent failed to connect properly. 1421 if (!SendCreateMyActorChild(std::move(childEnd))) { 1422 // ... assume an IPDL error will destroy parent. Handle failure beyond that ... 1423 return false; 1424 } 1425 return true; 1426 } 1427 1428 Here ``MyPreexistingActorParent`` is used to send a child endpoint for the new 1429 top level actor to ``MyPreexistingActorChild``, after it hooks up the parent 1430 end. In this example, we bind our new actor to the same thread we are running 1431 on -- which must be the same thread ``MyPreexistingActorParent`` is bound to 1432 since we are sending ``CreateMyActorChild`` from it. We could have bound on a 1433 different thread. 1434 1435 At this point, messages can be sent on the parent. Eventually, it will start 1436 receiving them as well. 1437 1438 ``MyPreexistingActorChild`` still has to receive the create message. The code 1439 for that handler is pretty similar: 1440 1441 .. code-block:: cpp 1442 1443 IPCResult MyPreexistingActorChild::RecvCreateMyActorChild(Endpoint<PMyActorChild>&& childEnd) { 1444 RefPtr<MyActorChild> child = new MyActorChild; 1445 if (!childEnd.Bind(child)) { 1446 // ... handle failure and return ok, assuming a related IPDL error will alert the other side to failure ... 1447 return IPC_OK(); 1448 } 1449 return IPC_OK(); 1450 } 1451 1452 Like the parent, the child is ready to send as soon as ``Bind`` is complete. 1453 It will start receiving messages soon afterward on the event target for the 1454 thread on which it is bound. 1455 1456 Creating First Top Level Actors 1457 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1458 1459 The first actor in a process is an advanced topic that is covered in 1460 :ref:`the documentation for adding a new process<Adding a New Type of Process>`. 1461 1462 PBackground 1463 ----------- 1464 1465 Developed as a convenient alternative to top level actors, ``PBackground`` is 1466 an IPDL protocol whose managees choose their worker threads in the child 1467 process and share a thread dedicated solely to them in the parent process. 1468 When an actor (parent or child) should run without hogging the main thread, 1469 making that actor a managee of ``PBackground`` (aka a *background actor*) is an 1470 option. 1471 1472 .. warning:: 1473 Background actors can be difficult to use correctly, as spelled out in this 1474 section. It is recommended that other options -- namely, top-level actors 1475 -- be adopted instead. 1476 1477 Background actors can only be used in limited circumstances: 1478 1479 * ``PBackground`` only supports the following process connections (where 1480 ordering is parent <-> child): main <-> main, main <-> content, 1481 main <-> socket and socket <-> content. 1482 1483 .. important:: 1484 1485 Socket process ``PBackground`` actor support was added after the other 1486 options. It has some rough edges that aren't easy to anticipate. In the 1487 future, their support may be broken out into a different actor or removed 1488 altogether. You are strongly encouraged to use new `Top Level Actors`_ 1489 instead of ``PBackground`` actor when communicating with socket process 1490 worker threads. 1491 1492 * Background actor creation is always initiated by the child. Of course, a 1493 request to create one can be sent to the child by any other means. 1494 * All parent background actors run in the same thread. This thread is 1495 dedicated to serving as the worker for parent background actors. While it 1496 has no other functions, it should remain responsive to all connected 1497 background actors. For this reason, it is a bad idea to conduct long 1498 operations in parent background actors. For such cases, create a top level 1499 actor and an independent thread on the parent side instead. 1500 * Background actors are currently *not* reference-counted. IPDL's ownership 1501 has to be carefully respected and the (de-)allocators for the new actors have 1502 to be defined. See `The Old Ways`_ for details. 1503 1504 A hypothetical layout of ``PBackground`` threads, demonstrating some of the 1505 process-type limitations, is shown in the diagram below. 1506 1507 .. mermaid:: 1508 :align: center 1509 :caption: Hypothetical ``PBackground`` thread setup. Arrow direction 1510 indicates child-to-parent ``PBackground``-managee relationships. 1511 Parents always share a thread and may be connected to multiple 1512 processes. Child threads can be any thread, including main. 1513 1514 flowchart LR 1515 subgraph content #1 1516 direction TB 1517 c1tm[main] 1518 c1t1[worker #1] 1519 c1t2[worker #2] 1520 c1t3[worker #3] 1521 end 1522 subgraph content #2 1523 direction TB 1524 c2tm[main] 1525 c2t1[worker #1] 1526 c2t2[worker #2] 1527 end 1528 subgraph socket 1529 direction TB 1530 stm[main] 1531 st1[background parent /\nworker #1] 1532 st2[worker #2] 1533 end 1534 subgraph main 1535 direction TB 1536 mtm[main] 1537 mt1[background parent] 1538 end 1539 1540 %% PBackground connections 1541 c1tm --> mt1 1542 c1t1 --> mt1 1543 c1t2 --> mt1 1544 1545 c1t3 --> mt1 1546 c1t3 --> st1 1547 1548 c2t1 --> st1 1549 c2t1 --> mt1 1550 1551 c2t2 --> mt1 1552 1553 c2tm --> st1 1554 1555 stm --> mt1 1556 st1 --> mt1 1557 st2 --> mt1 1558 1559 Creating background actors is done a bit differently than normal managees. The 1560 new managed type and constructor are still added to ``PBackground.ipdl`` as 1561 with normal managees but, instead of ``new``-ing the child actor and then 1562 passing it in a ``SendFooConstructor`` call, background actors issue the send 1563 call to the ``BackgroundChild`` manager, which returns the new child: 1564 1565 .. code-block:: cpp 1566 1567 // Bind our new PMyBackgroundActorChild to the current thread. 1568 PBackgroundChild* bc = BackgroundChild::GetOrCreateForCurrentThread(); 1569 if (!bc) { 1570 return false; 1571 } 1572 PMyBackgroundActorChild* pmyBac = bac->SendMyBackgroundActor(constructorParameters); 1573 if (!pmyBac) { 1574 return false; 1575 } 1576 auto myBac = static_cast<MyBackgroundActorChild*>(pmyBac); 1577 1578 .. note:: 1579 ``PBackgroundParent`` still needs a ``RecvMyBackgroundActorConstructor`` 1580 handler, as usual. This must be done in the ``ParentImpl`` class. 1581 ``ParentImpl`` is the non-standard name used for the implementation of 1582 ``PBackgroundParent``. 1583 1584 To summarize, ``PBackground`` attempts to simplify a common desire in Gecko: 1585 to run tasks that communicate between the main and content processes but avoid 1586 having much to do with the main thread of either. Unfortunately, it can be 1587 complicated to use correctly and has missed on some favorable IPDL 1588 improvements, like reference counting. While top level actors are always a 1589 complete option for independent jobs that need a lot of resources, 1590 ``PBackground`` offers a compromise for some cases. 1591 1592 IPDL Best Practices 1593 ------------------- 1594 1595 IPC performance is affected by a lot of factors. Many of them are out of our 1596 control, like the influence of the system thread scheduler on latency or 1597 messages whose travel internally requires multiple legs for security reasons. 1598 On the other hand, some things we can and should control for: 1599 1600 * Messages incur inherent performance overhead for a number of reasons: IPDL 1601 internal thread latency (e.g. the I/O thread), parameter (de-)serialization, 1602 etc. While not usually dramatic, this cost can add up. What's more, each 1603 message generates a fair amount of C++ code. For these reasons, it is wise 1604 to reduce the number of messages being sent as far as is reasonable. This 1605 can be as simple as consolidating two asynchronous messages that are always 1606 in succession. Or it can be more complex, like consolidating two 1607 somewhat-overlapping messages by merging their parameter lists and marking 1608 parameters that may not be needed as optional. It is easy to go too far down 1609 this path but careful message optimization can show big gains. 1610 * Even ``[moveonly]`` parameters are "copied" in the sense that they are 1611 serialized. The pipes that transmit the data are limited in size and require 1612 allocation. So understand that the performance of your transmission will be 1613 inversely proportional to the size of your content. Filter out data you 1614 won't need. For complex reasons related to Linux pipe write atomicity, it is 1615 highly desirable to keep message sizes below 4K (including a small amount for 1616 message metadata). 1617 * On the flip side, very large messages are not permitted by IPDL and will 1618 result in a runtime error. The limit is currently 256M but message failures 1619 frequently arise even with slightly smaller messages. 1620 * Parameters to messages are C++ types and therefore can be very complex in the 1621 sense that they generally represent a tree (or graph) of objects. If this 1622 tree has a lot of objects in it, and each of them is serialized by 1623 ``ParamTraits``, then we will find that serialization is allocating and 1624 constructing a lot of objects, which will stress the allocator and cause 1625 memory fragmentation. Avoid this by using larger objects or by sharing this 1626 kind of data through careful use of shared memory. 1627 * As it is with everything, concurrency is critical to the performance of IPDL. 1628 For actors, this mostly manifests in the choice of bound thread. While 1629 adding a managed actor to an existing actor tree may be a quick 1630 implementation, this new actor will be bound to the same thread as the old 1631 one. This contention may be undesirable. Other times it may be necessary 1632 since message handlers may need to use data that isn't thread safe or may 1633 need a guarantee that the two actors' messages are received in order. Plan 1634 up front for your actor hierarchy and its thread model. Recognize when you 1635 are better off with a new top level actor or ``PBackground`` managee that 1636 facilitates processing messages simultaneously. 1637 * Remember that latency will slow your entire thread, including any other 1638 actors/messages on that thread. If you have messages that will need a long 1639 time to be processed but can run concurrently then they should use actors 1640 that run on a separate thread. 1641 * Top-level actors decide a lot of properties for their managees. Probably the 1642 most important are the process layout of the actor (including which process 1643 is "Parent" and which is "Child") and the thread. Every top-level actor 1644 should clearly document this, ideally in their .ipdl file. 1645 1646 The Old Ways 1647 ------------ 1648 1649 TODO: 1650 1651 The FUD 1652 ------- 1653 1654 TODO: 1655 1656 The Rest 1657 -------- 1658 1659 Nested messages 1660 ~~~~~~~~~~~~~~~ 1661 1662 The ``Nested`` message annotations indicate the nesting type of the message. 1663 They attempt to process messages in the nested order of the "conversation 1664 thread", as found in e.g. a mailing-list client. This is an advanced concept 1665 that should be considered to be discouraged, legacy functionality. 1666 Essentially, ``Nested`` messages can make other ``sync`` messages break the 1667 policy of blocking their thread -- nested messages are allowed to be received 1668 while a sync messagee is waiting for a response. The rules for when a nested 1669 message can be handled are somewhat complex but they try to safely allow a 1670 ``sync`` message ``M`` to handle and respond to some special (nested) messages 1671 that may be needed for the other endpoint to finish processing ``M``. There is 1672 a `comment in MessageChannel`_ with info on how the decision to handle nested 1673 messages is made. For sync nested messages, note that this implies a relay 1674 between the endpoints, which could dramatically affect their throughput. 1675 1676 Declaring messages to nest requires an annotation on the actor and one on the 1677 message itself. The nesting annotations were listed in `Defining Actors`_ and 1678 `Declaring IPDL Messages`_. We repeat them here. The actor annotations 1679 specify the maximum priority level of messages in the actor. It is validated 1680 by the IPDL compiler. The annotations are: 1681 1682 ============================== ================================================ 1683 ``[NestedUpTo=inside_sync]`` Indicates that an actor contains messages of 1684 priority [Nested=inside_sync] or lower. 1685 ``[NestedUpTo=inside_cpow]`` Indicates that an actor contains messages of 1686 priority [Nested=inside_cpow] or lower. 1687 ============================== ================================================ 1688 1689 .. note:: 1690 1691 The order of the nesting priorities is: 1692 (no nesting priority) < ``inside_sync`` < ``inside_cpow``. 1693 1694 The message annotations are: 1695 1696 ========================== ==================================================== 1697 ``[Nested=inside_sync]`` Indicates that the message can be handled while 1698 waiting for lower-priority, or in-message-thread, 1699 sync responses. 1700 ``[Nested=inside_cpow]`` Indicates that the message can be handled while 1701 waiting for lower-priority, or in-message-thread, 1702 sync responses. Cannot be sent by the parent actor. 1703 ========================== ==================================================== 1704 1705 .. note:: 1706 1707 ``[Nested=inside_sync]`` messages must be sync (this is enforced by the 1708 IPDL compiler) but ``[Nested=inside_cpow]`` may be async. 1709 1710 Nested messages are obviously only interesting when sent to an actor that is 1711 performing a synchronous wait. Therefore, we will assume we are in such a 1712 state. Say ``actorX`` is waiting for a sync reply from ``actorY`` for message 1713 ``m1`` when ``actorY`` sends ``actorX`` a message ``m2``. We distinguish two 1714 cases here: (1) when ``m2`` is sent while processing ``m1`` (so ``m2`` is sent 1715 by the ``RecvM1()`` method -- this is what we mean when we say "nested") and 1716 (2) when ``m2`` is unrelated to ``m1``. Case (2) is easy; ``m2`` is only 1717 dispatched while ``m1`` waits if 1718 ``priority(m2) > priority(m1) > (no priority)`` and the message is being 1719 received by the parent, or if ``priority(m2) >= priority(m1) > (no priority)`` 1720 and the message is being received by the child. Case (1) is less 1721 straightforward. 1722 1723 To analyze case (1), we again distinguish the two possible ways we can end up 1724 in the nested case: (A) ``m1`` is sent by the parent to the child and ``m2`` 1725 is sent by the child to the parent, or (B) where the directions are reversed. 1726 The following tables explain what happens in all cases: 1727 1728 .. |strike| raw:: html 1729 1730 <strike> 1731 1732 .. |endstrike| raw:: html 1733 1734 </strike> 1735 1736 .. |br| raw:: html 1737 1738 <br/> 1739 1740 .. table :: Case (A): Child sends message to a parent that is awaiting a sync response 1741 :align: center 1742 1743 ============================== ======================== ======================================================== 1744 sync ``m1`` type (from parent) ``m2`` type (from child) ``m2`` handled or rejected 1745 ============================== ======================== ======================================================== 1746 sync (no priority) \* IPDL compiler error: parent cannot send sync (no priority) 1747 sync inside_sync async (no priority) |strike| ``m2`` delayed until after ``m1`` completes |endstrike| |br| 1748 Currently ``m2`` is handled during the sync wait (bug?) 1749 sync inside_sync sync (no priority) |strike| ``m2`` send fails: lower priority than ``m1`` |endstrike| |br| 1750 Currently ``m2`` is handled during the sync wait (bug?) 1751 sync inside_sync sync inside_sync ``m2`` handled during ``m1`` sync wait: same message thread and same priority 1752 sync inside_sync async inside_cpow ``m2`` handled during ``m1`` sync wait: higher priority 1753 sync inside_sync sync inside_cpow ``m2`` handled during ``m1`` sync wait: higher priority 1754 sync inside_cpow \* IPDL compiler error: parent cannot use inside_cpow priority 1755 ============================== ======================== ======================================================== 1756 1757 .. table :: Case (B): Parent sends message to a child that is awaiting a sync response 1758 :align: center 1759 1760 ============================= ========================= ======================================================== 1761 sync ``m1`` type (from child) ``m2`` type (from parent) ``m2`` handled or rejected 1762 ============================= ========================= ======================================================== 1763 \* async (no priority) ``m2`` delayed until after ``m1`` completes 1764 \* sync (no priority) IPDL compiler error: parent cannot send sync (no priority) 1765 sync (no priority) sync inside_sync ``m2`` send fails: no-priority sync messages cannot handle 1766 incoming messages during wait 1767 sync inside_sync sync inside_sync ``m2`` handled during ``m1`` sync wait: same message thread and same priority 1768 sync inside_cpow sync inside_sync ``m2`` send fails: lower priority than ``m1`` 1769 \* async inside_cpow IPDL compiler error: parent cannot use inside_cpow priority 1770 \* sync inside_cpow IPDL compiler error: parent cannot use inside_cpow priority 1771 ============================= ========================= ======================================================== 1772 1773 We haven't seen rule #2 from the `comment in MessageChannel`_ in action but, as 1774 the comment mentions, it is needed to break deadlocks in cases where both the 1775 parent and child are initiating message-threads simultaneously. It 1776 accomplishes this by favoring the parent's sent messages over the child's when 1777 deciding which message-thread to pursue first (and blocks the other until the 1778 first completes). Since this distinction is entirely thread-timing based, 1779 client code needs only to be aware that IPDL internals will not deadlock 1780 because of this type of race, and that this protection is limited to a single 1781 actor tree -- the parent/child messages are only well-ordered when under the 1782 same top-level actor so simultaneous sync messages across trees are still 1783 capable of deadlock. 1784 1785 Clearly, tight control over these types of protocols is required to predict how 1786 they will coordinate within themselves and with the rest of the application 1787 objects. Control flow, and hence state, can be very difficult to predict and 1788 are just as hard to maintain. This is one of the key reasons why we have 1789 stressed that message priorities should be avoided whenever possible. 1790 1791 .. _comment in MessageChannel: https://searchfox.org/mozilla-central/rev/077501b34cca91763ae04f4633a42fddd919fdbd/ipc/glue/MessageChannel.cpp#54-118 1792 1793 .. _Message Logging: 1794 1795 Message Logging 1796 ~~~~~~~~~~~~~~~ 1797 1798 The environment variable ``MOZ_IPC_MESSAGE_LOG`` controls the logging of IPC 1799 messages. It logs details about the transmission and reception of messages. 1800 This isn't controlled by ``MOZ_LOG`` -- it is a separate system. Set this 1801 variable to ``1`` to log information on all IPDL messages, or specify a 1802 comma-separated list of protocols to log. 1803 If the ``Child`` or ``Parent`` suffix is given, then only activity on the given 1804 side is logged; otherwise, both sides are logged. All protocol names must 1805 include the ``P`` prefix. 1806 1807 For example: 1808 1809 .. code-block:: 1810 1811 MOZ_IPC_MESSAGE_LOG="PMyManagerChild,PMyManaged" 1812 1813 This requests logging of child-side activity on ``PMyManager``, and both 1814 parent- and child-side activity on ``PMyManaged``. 1815 1816 :ref:`Debugging with IPDL Logging` has an example where IPDL logging is useful 1817 in tracking down a bug.