Keep track of information that led the module interface loader to reject loading a compiled module, if it needed to fall back to compiling an interface.
rdar://47792754
Although the default cache location separates by platform, a
/combined/ cache is still an interesting idea, and in that case we
need to use the full platform names.
- Fix a typo in the debug output for finding a prebuilt module
- Don't add the prebuilt module path to the dependency tracker
- Ensure we're registering the _forwarding module_'s dependencies in
cached modules, not the prebuilt module's dependencies.
- Check for the existence of a prebuilt module before doing the
is-up-to-date check (which will have failed anyway if it didn't exist).
- Test that forwarding module dependency collection works
rdar://48659199
We previously added dependencies to the tracker inline while validating a cached
module's dependencies were up to date. If one of its dependencies ended up being
out of date though, we shouldn't have added the previous dependencies, as that
means the dependency list itself was also out of date.
This patch changes the behavior to only add the module's dependencies once we've
verified they're all up to date.
This patch modifies ParseableInterfaceBuilder::CollectDepsForSerialization to
avoid serializing dependencies from the runtime resource path into the
swiftmodules generated from .swiftinterface files. This means the module cache
should now be relocatable across machines.
It also modifies ParseableInterfaceModuleLoader to never add any dependencies
from the module cache and prebuilt cache to the dependency tracker (in addition
to the existing behaviour of not serializing them in the generated
swiftmodules). As a result, CollectDepsForSerialization no longer checks if the
dependencies it is given come from the cache as they are provided by the
dependency tracker. It now asserts that's the case instead.
*Their* dependencies are already being serialized out, so this shouldn't affect
up-to-date-checking except by alowing the regular and prebuilt module caches to
be relocated without invalidating their contents. In the case of the prebuilt
module cache, this gets us closer to supporting relocation across machines.
One interesting thing here is that the decision of whether or not to
track system dependencies ends up going into the cache key for a
swiftmodule built from a parseable interface, because it affects that
module's up-to-date check. If we didn't include
-track-system-dependencies in the cache key, we could end up tracking
system dependencies for some modules but not others in the same build.
There's a bit of a bug here where they're /not/ honored if the
top-level invocation isn't tracking /any/ dependencies, but given
how uncommon this flag is in practice that's probably okay for now.
Still TODO: honor this for -build-swiftmodule-from-parseable-interface
as well. That's currently not tracking dependencies at all and it
probably should.
Extract a Unix-path specific test into its own file to allow the rest of
the tests to be covered on Windows. This makes the last
ParseableInterface test pass.
Rewrite the shell to be more clever so it is amenable to porting to
Windows. Use env not rather than not env to permit the lit environment
handling to take over for Windows.
...and remove the option. This is ~technically~ CLI-breaking because
Swift 5 shipped this as a hidden driver option, but it wouldn't have
/done/ anything in Swift 5, so I think it's okay to remove.
Note that if a parseable interface (.swiftinterface) and a binary
interface (.swiftmodule) are both present, the binary one will still
be preferred. This just /allows/ parseable interfaces to be used.
rdar://problem/36885834
And fix the error message printing while I'm at it.
I don't have a public test for this at the moment, but an Apple-
internal tool will use it. I hope to upstream that tool in the future.
A ‘forwarding module’ is a YAML file that’s meant to stand in for a .swiftmodule file and provide an up-to-date description of its dependencies, always using modification times.
When a ‘prebuilt module’ is first loaded, we verify that it’s up-to-date by hashing all of its dependencies. Since this is orders of magnitude slower than reading mtimes, we’ll install a `forwarding module` containing the mtimes of the now-validated dependencies.
When loading a module supporting multiple targets, the module loader now looks for a file named with a normalized version of the target triple first, and only falls back to the architecture name if the normalized triple is not found.
This changes the Swift resource directory from looking like
lib/
swift/
macosx/
libswiftCore.dylib
libswiftDarwin.dylib
x86_64/
Swift.swiftmodule
Swift.swiftdoc
Darwin.swiftmodule
Darwin.swiftdoc
to
lib/
swift/
macosx/
libswiftCore.dylib
libswiftDarwin.dylib
Swift.swiftmodule/
x86_64.swiftmodule
x86_64.swiftdoc
Darwin.swiftmodule/
x86_64.swiftmodule
x86_64.swiftdoc
matching the layout we use for multi-architecture swiftmodules
everywhere else (particularly frameworks).
There's no change in this commit to how Linux swiftmodules are
packaged. There's been past interest in going the /opposite/ direction
for Linux, since there's not standard support for fat
(multi-architecture) .so libraries. Moving the .so search path /down/
to an architecture-specific directory on Linux would allow the same
resource directory to be used for both host-compiling and
cross-compiling.
rdar://problem/43545560
We were checking the parent invocation's DiagnosticEnginer rather than the
subinstance's to determine if there were any errors building the module, which
meant we would fail to load the module if there were errors prior to the import
statement in the importing file.
This also meant code completion would fail to load the module, because it always
emits a bogus error in order to mark the AST as erroneous so that different
parts of the compiler (e.g. the verifier) have less strict assumptions.
rdar://problem/43906499
We were checking the parent invocation's DiagnosticEnginer rather than the
subinstance's to determine if there were any errors building the module, which
meant we would fail to load the module if there were errors prior to the import
statement in the importing file.
This also meant code completion would fail to load the module, because it always
emits a bogus error in order to mark the AST as erroneous so that different
parts of the compiler (e.g. the verifier) have less strict assumptions.
rdar://problem/43906499
Adds a test that errors reported in the inlinable function bodies of
.swiftinterface files are propagated correctly when generating the corresponding
.swiftmodule files.
rdar://problem/44400260
Adds a test to make sure that there's only one version of a dependent
module, regardless how many -swift-versions are present in the
dependency hierarchy.
These tests made sure that changing the hash, while keeping mtime constant, causes dependent modules to be rebuilt. Flip this around so changing the mtime, while keeping the hash constant, causes the same rebuilding behavior.
Otherwise, the top-level compilation gets the benefit of the prebuilt
cache path, but the sub-invocations for swiftinterfaces that /do/
need to be compiled do not.
The previous 'openModuleFiles' interface in SerializedModuleLoaderBase
still assumed that swiftmodule files and swiftdoc files would be found
next to each other, but that's not true anymore with
swiftinterfaces-built-to-modules. Give up on this assumption (and on
the minor optimization of passing down a scratch buffer) and split out
the interface into the customization point
'findModuleFilesInDirectory' and the implementation 'openModuleFiles'.
The latter now takes two full paths: one for the swiftmodule, one for
the swiftdoc.
Ensure that we explicitly provide the path to the interpreter when
running the python scripts during the tests. This is needed to have the
tests work on Windows which does not honour the shebang in the file. It
also ensures that we use the correct interpreter for the tests.
Deinitializers are always @objc. (Arguably, this makes it unnecessary
to print, but we don't want to do any @objc inference at all in a
swiftinterface.)
- Use the name for the cached module, so that we don't end up with a
zillion "x86_64-XXXXXXXX.swiftmodule" files in the cache when we're
working with architecture-specific swiftmodules.
- Diagnose if the expected name is different from the name specified
in the swiftinterface.
- Emit all diagnostics at the location of the import, instead of
without any location at all.
