Add '-validate-clang-modules-once' and '-clang-build-session-file' corresponding to Clang's '-fmodules-validate-once-per-build-session' and '-fbuild-session-file='. Ensure they are propagated to module interface build sub-invocations.
We require these to be first-class Swift options in order to ensure they are propagated to both: ClangImporter and implicit interface build compiler sub-invocations.
Compiler portion of rdar://105982120
Since https://github.com/apple/swift/pull/63178 added support for Clang modules in the explicit module map, it is possible for there to be multiple modules with the same name: a Swift module and a Clang module. The current parsing logic just overwrites the corresponding entry module in a hashmap so we always only preserved the module that comes last, with the same name.
This change separates the parsing of the modulemap JSON file to produce a separate Swift module map and Clang module map. The Swift one is used by the 'ExplicitSwiftModuleLoader', as before, and the Clang one is only used to populate the ClangArgs with the requried -fmodule-... flags.
If a module was first read using the adjacent swiftmodule and then
reloaded using the swiftinterface, we would do an up to date check on
the adjacent module but write out the unit using the swiftinterface.
This would cause the same modules to be indexed repeatedly for the first
invocation using a new SDK. On the next run we would instead raad the
swiftmodule from the cache and thus the out of date check would match
up.
The impact of this varies depending on the size of the module graph in
the initial compilation and the number of jobs started at the same time.
Each SDK dependency is re-indexed *and* reloaded, which is a drain on
both CPU and memory. Thus, if many jobs are initially started and
they're all going down this path, it can cause the system to run out of
memory very quickly.
Resolves rdar://103119964.
Do this by computing a transitive closure on the computed dependency graph, relying on the fact that it is a DAG.
The used algorithm is:
```
for each v ∈ V {
T(v) = { v }
}
for v ∈ V in reverse topological order {
for each (v, w) ∈ E {
T(v) = T(v) ∪ T(w)
}
}
```
This lets users of `-explicit-swift-module-map-file` use a single mapping
for all module dependencies, regardless of whether they're Swift or Clang
modules, instead of manually splitting them among this file and command
line flags.
If the json file doesn't contain a value for this, this was never set,
which results in UB.
Unfortunately clang doesn't warn about this but gcc does https://godbolt.org/z/M3sdE73zs
Intro ASTContext::setIgnoreAdjacentModules to change module loading to
accept load only resilient modules from their swiftinterfaces, ignoring
the adjacent module and any silencing swiftinterfaces errors.
Previously, when evaluating a `#if canImport(Module, _version: 42)` directive the compiler could diagnose and ignore the directive under the following conditions:
- The associated binary module is corrupt/bogus.
- The .tbd for an underlying Clang module is missing a current-version field.
This behavior is surprising when there is a valid `.swiftinterface` available and it only becomes apparent when building against an SDK with an old enough version of the module that the version in the `.swiftinterface` is too low, making this failure easy to miss. Some modules have different versioning systems for their Swift and Clang modules and it can also be intentional for a distributed binary `.swiftmodule` to contain bogus data (to force the compiler to recompile the `.swiftinterface`) so we need to handle both of these cases gracefully and predictably.
Now the compiler will enumerate all module loaders, ask each of them to attempt to parse the module version and then consistently use the parsed version from a single source. The `.swiftinterface` is preferred if present, then the binary module if present, and then finally the `.tbd`. The `.tbd` is still always used exclusively for the `_underlyingVersion` variant of `canImport()`.
Resolves rdar://88723492
Instead of checking that the stdlib can be loaded in a variety of places, check it when setting up the compiler instance. This required a couple more checks to avoid loading the stdlib in cases where it’s not needed.
To be able to differentiate stdlib loading failures from other setup errors, make `CompilerInstance::setup` return an error message on failure via an inout parameter. Consume that error on the call side, replacing a previous, more generic error message, adding error handling where appropriate or ignoring the error message, depending on the context.
Ideally, module interface verification should fail the build when fatal error occurs when
type checking emitted module interfaces. However, we found it's hard to stage this phase in
because the ideal case requires all Swift adopters to have valid interfaces. This new front-end flag allows
driver to downgrade all interface verification errors to warnings as an intermediate step.
We should hold off actually building the binary module file until it is imported.
`canImport` queries can happen, for example, during dependency scanning, when we do not wish to have the scanner tool execute any module builds.
Resolves rdar://82603098
We have implemented a libSwiftDriver-based tool to generate prebuilt module cache for
entire SDKs. Anchored on the same infrastructure, we could also generate ABI baselines
for entire SDKs.
This mechanism allows the compiler to use a backup interface file to build into a binary module when
a corresponding interface file from the SDK is failing for whatever reasons. This mechansim should be entirely opaque
to end users except several diagnostic messages communicating backup interfaces are used.
Part of rdar://77676064
canImport should be able to take an additional parameter labeled by either version or
underlyingVersion. We need underlyingVersion for clang modules with Swift overlays because they
have separate version numbers. The library users are usually interested in checking the importability
of the underlying clang module instead of its Swift overlay.
Part of rdar://73992299
This will enable users to try out the '-enable-ossa-modules' flag if their
compiler supports it and get OSSA code on all inlinable code that they use. The
idea is that this is a nice way to stage this in and get more testing.
The specific implementation is that the module interface loader:
1. Knows if enable ossa modules is enabled not to search for any compiled
modules. We always rebuild from the interface file on the system.
2. Knows that if enable ossa modules is enabled to mixin a bit into the module
interface loader cache hash to ensure that we consider the specialized ossa
compiled modules to be different than the modules in that cache from the system.
This ensures that when said flag is enabled, the user transparently gets all
their code in OSSA form from transparent libraries.
This refactoring allows us to drop ModuleInterfaceLoader when explicit modules
are enabled. Before this change, the dependencies scanner needs the loader to be
present to access functionalities like collecting prebuilt module candidates.
In the fast dependency scanner, depending on whether a module intrface was found via the import search path or framework search path, encode into the dependency graph Swift module details, whether a given module is a framework.
Some implicitly imported modules aren't printed in the textual interface file as explicit import,
e.g. SwiftOnoneSupport. We should check implicit imports and add them to the dependency graph.
-compile-module-from-interface action now takes arguments of -candidate-module-file.
If one of the candidate module files is up-to-date, the action emits a forwarding
module pointing to the candidate module instead of building a binary module.
Instead of replacing an interface file with its up-to-date compile module,
the dep-scanner should report potentially up-to-date module candidates either adjacent to
the interface file or in the prebuilt module cache. swift-driver should later pass down
these candidates to -compile-module-from-interface invocation and the front-end job
will check if one of the candidates is ready to use. The front-end job then either emits a forwarding
module to an up-to-date candidate or a binary module.
