engines.rst (5939B)
1 ============================ 2 The Sync engines in the tree 3 ============================ 4 5 Unless otherwise specified, the engine implementations can be found 6 `here <https://searchfox.org/mozilla-central/source/services/sync/modules/engines>`_ 7 8 Please read the :doc:`overview`. 9 10 Clients 11 ======= 12 13 The ``clients`` engine is a special engine in that it's invisible to the 14 user and can not be disabled - think of it as a "meta" engine. As such, it 15 doesn't really have a sensible concept of ``store`` or ``tracker``. 16 17 The engine is mainly responsible for keeping its own record current in the 18 ``clients`` collection. Some parts of Sync use this collection to know what 19 other clients exist and when they last synced (although alot of this is moving 20 to using the Firefox Accounts devices). 21 22 Clients also has the ability to handle ``commands`` - in short, some other 23 client can write to this client's ``commands``, and when this client notices, 24 it will execute the command. Commands aren't arbitrary, so commands must be 25 understood by both sides for them to work. There are commands to "wipe" 26 collections etc. In practice, this is used only by ``bookmarks`` when a device 27 restores bookmarks - in that case, the restoring device will send a ``wipe`` 28 command to all other clients so that they take the new bookmarks instead of 29 merging them. 30 31 If not for this somewhat limited ``commands`` functionality, this engine could 32 be considered deprecated and subsumed by FxA devices - but because we 33 can't just remove support for commands and also do not have a plan for 34 replacing them, the clients engine remains important. 35 36 Bookmarks 37 ========= 38 39 The ``bookmarks`` engine has changed so that it's tightly integrated with the 40 ``places`` database. Instead of an external ``tracker``, the tracking is 41 integrated into Places. Each bookmark has a `syncStatus` and a 42 `syncChangeCounter` and these are managed internally by places. Sync then just 43 queries for changed bookmarks by looking for these fields. 44 45 Bookmarks is somewhat unique in that it needs to maintain a tree structure, 46 which makes merging a challenge. The `dogear <https://github.com/mozilla/dogear>`_ 47 component (written in Rust and also used by the 48 `application-services bookmarks component <https://github.com/mozilla/application-services/tree/main/components/places>`_) 49 performs this merging. 50 51 Bookmarks also pioneered the concept of a "mirror" - this is a database table 52 which tracks exactly what is on the server. Because each sync only fetches 53 changes from the server since the last sync, each sync does not supply every 54 record on the server. However, the merging code does need to know what's on 55 the server - so the mirror tracks this. 56 57 History 58 ======= 59 60 History is similar to bookmarks described above - it's closely integrated with 61 places - but is less complex because there's no tree structure involved. 62 63 One unique characteristic of history is that the engine takes steps to *not* 64 upload everything - old profiles tend to have too much history to reasonably 65 store and upload, so typically uploads are limited to the last 5000 visits. 66 67 Logins 68 ====== 69 70 Logins has also been upgraded to be closely integrated with `Services.logins` - 71 the logins component itself manages the metadata. 72 73 Tabs 74 ==== 75 76 Tabs is a special engine in that there's no underlying storage at all - it 77 both saves the currently open tabs from this device (which are enumerated 78 every time it's updated) and also lets other parts of Firefox know which tabs 79 are open on other devices. There's no database - if we haven't synced yet we 80 don't know what other tabs are open, and when we do know, the list is just 81 stored in memory. 82 83 The `SyncedTabs module <https://searchfox.org/mozilla-central/source/services/sync/modules/SyncedTabs.sys.mjs>`_ 84 is the main interface the browser uses to get the list of tabs from other 85 devices. 86 87 Add-ons 88 ======= 89 90 Addons is still an "old school" engine, with a tracker and store which aren't 91 closely integrated with the addon manager. As a result it's fairly complex and 92 error prone - eg, it persists the "last known" state so it can know what to 93 sync, where a better model would be for the addon manager to track the changes 94 on Sync's behalf. 95 96 It also attempts to sync themes etc. The future of this engine isn't clear given 97 it doesn't work on mobile platforms. 98 99 Addresses / Credit-Cards 100 ======================== 101 102 Addresses and Credit-cards have Sync functionality tightly bound with the 103 store. Unlike other engines above, this engine has always been tightly bound, 104 because it was written after we realized this tight-binding was a feature and 105 not a bug. 106 107 Technically these are 2 separate engines and collections. However, because the 108 underlying storage uses a shared implementation, the syncing also uses a 109 shared implementation - ie, the same logic is used for both - so we tend to 110 treat them as a single engine in practice. 111 112 As a result, only a shim is in the `services/sync/modules/engines/` directory, 113 while the actual logic is 114 `next to the storage implementation <https://searchfox.org/mozilla-central/source/toolkit/components/formautofill/FormAutofillSync.sys.mjs>`_. 115 116 This engine has a unique twist on the "mirror" concept described above - 117 whenever a change is made to a fields, the original value of the field is 118 stored directly in the storage. This means that on the next sync, the value 119 of the record on the server can be deduced, meaning a "3-way" merge can be 120 done, so it can better tell the difference between local only, remote only, or 121 conflicting changes. 122 123 WebExt-Storage 124 ============== 125 126 webext-storage is implemented in Rust and lives in 127 `application services <https://github.com/mozilla/application-services/tree/main/components/webext-storage>`_ 128 and is vendored into the addons code - note that this includes the storage 129 *and* Sync code. The Sync engine itself is a shim in the sync directory. 130 131 See the :doc:`rust-engines` document for more about how rust engines are 132 integrated.