Robert Widmann 859b87fd8c Move The Last Pieces for Cross-Module Incremental Builds ...

We're going to play a dirty, dirty trick - but it'll make our users'
lives better in the end so stick with me here.

In order to build up an incremental compilation, we need two sources of
dependency information:

1) "Priors" - Swiftdeps with dependency information from the past
   build(s)
2) "Posteriors" - Swiftdeps with dependencies from after we rebuild the
   file or module or whatever

With normal swift files built in incremental mode, the priors are given by the
swiftdeps files which are generated parallel to a swift file and usually
placed in the build directory alongside the object files. Because we
have entries in the output file map, we can always know where these
swiftdeps files are. The priors are integrated by the driver and then
the build is scheduled. As the build runs and jobs complete, their
swiftdeps are reloaded and re-integrated. The resulting changes are then
traversed and more jobs are scheduled if necessary. These give us the
posteriors we desire.

A module flips this on its head. The swiftdeps information serialized
in a module functions as the *posterior* since the driver consuming the
module has no way of knowing how to rebuild the module, and because its
dependencies are, for all intents and purposes, fixed in time. The
missing piece of the puzzle is the priors. That is, we need some way of
knowing what the "past" interface of the module looked like so we can
compare it to the "present" interface. Moreover, we need to always know
where to look for these priors.

We solve this problem by serializing a file alongside the build record:
the "external" build record. This is given by a... creative encoding
of multiple source file dependency graphs into a single source file
dependency graph. The rough structure of this is:

   SourceFile => interface <BUILD_RECORD>.external
   | - Incremental External Dependency => interface <MODULE_1>.swiftmodule
   | | - <dependency> ...
   | | - <dependency> ...
   | | - <dependency> ...
   | - Incremental External Dependency => interface <MODULE_2>.swiftmodule
   | | - <dependency> ...
   | | - <dependency> ...
   | - Incremental External Dependency => interface <MODULE_3>.swiftmodule
   | - ...

Sorta, `cat`'ing a bunch of source file dependency graphs together but
with incremental external dependency nodes acting as glue.

Now for the trick:

We have to unpack this structure and integrate it to get our priors.
This is easy. The tricky bit comes in integrate itself. Because the
top-level source file node points directly at the external build record,
not the original swift modules that defined these dependency nodes, we
swap the key it wants to use (the external build record) for the
incremental external dependency acting as the "parent" of the dependency
node. We do this by following the arc we carefully laid down in the
structure above.

For rdar://69595010
Goes a long way towards rdar://48955139, rdar://64238133

2020-12-10 18:45:21 -08:00

30 KiB

Raw Blame History

View Raw