Introduce a new source file kind to describe source files for macro
expansions, and include the macro expression that they expand. This
establishes a "parent" relationship
Also track every kind of auxiliary source file---whether for macro
expansions or other reasons---that is introduced into a module, adding
an operation that allows us to find the source file that contains a
given source location.
Previously, when evaluating a `#if canImport(Module, _version: 42)` directive the compiler could diagnose and ignore the directive under the following conditions:
- The associated binary module is corrupt/bogus.
- The .tbd for an underlying Clang module is missing a current-version field.
This behavior is surprising when there is a valid `.swiftinterface` available and it only becomes apparent when building against an SDK with an old enough version of the module that the version in the `.swiftinterface` is too low, making this failure easy to miss. Some modules have different versioning systems for their Swift and Clang modules and it can also be intentional for a distributed binary `.swiftmodule` to contain bogus data (to force the compiler to recompile the `.swiftinterface`) so we need to handle both of these cases gracefully and predictably.
Now the compiler will enumerate all module loaders, ask each of them to attempt to parse the module version and then consistently use the parsed version from a single source. The `.swiftinterface` is preferred if present, then the binary module if present, and then finally the `.tbd`. The `.tbd` is still always used exclusively for the `_underlyingVersion` variant of `canImport()`.
Resolves rdar://88723492
When looking for a Swift module on disk, we were scanning all module search paths if they contain the module we are searching for. In a setup where each module is contained in its own framework search path, this scaled quadratically with the number of modules being imported. E.g. a setup with 100 modules being imported form 100 module search paths could cause on the order of 10,000 checks of `FileSystem::exists`. While these checks are fairly fast (~10µs), they add up to ~100ms.
To improve this, perform a first scan of all module search paths and list the files they contain. From this, create a lookup map that maps filenames to the search paths they can be found in. E.g. for
```
searchPath1/
Module1.framework
searchPath2/
Module1.framework
Module2.swiftmodule
```
we create the following lookup table
```
Module1.framework -> [searchPath1, searchPath2]
Module2.swiftmodule -> [searchPath2]
```
These kinds of modules differ from `SwiftTextual` modules in that they do not have an interface and have source-files.
It is cleaner to enforce this distinction with types, instead of checking for interface optionality everywhere.
We should hold off actually building the binary module file until it is imported.
`canImport` queries can happen, for example, during dependency scanning, when we do not wish to have the scanner tool execute any module builds.
Resolves rdar://82603098
The dependency scanner's cache persists across different queries and answering a subsequent query's module lookup with a module not in the query's search path is not correct.
For example, suppose we are looking for a Swift module `Foo` with a set of search paths `SP`.
And dependency scanner cache already contains a module `Foo`, for which we found an interface file at location `L`. If `L`∉`SP`, then we cannot re-use the cached entry because we’d be resolving the scanning query to a filesystem location that the current scanning context is not aware of.
Resolves rdar://81175942
This mechanism allows the compiler to use a backup interface file to build into a binary module when
a corresponding interface file from the SDK is failing for whatever reasons. This mechansim should be entirely opaque
to end users except several diagnostic messages communicating backup interfaces are used.
Part of rdar://77676064
canImport should be able to take an additional parameter labeled by either version or
underlyingVersion. We need underlyingVersion for clang modules with Swift overlays because they
have separate version numbers. The library users are usually interested in checking the importability
of the underlying clang module instead of its Swift overlay.
Part of rdar://73992299
This matches the behavior of the current client (`swift-driver`) and reduces ambiguity in how the nodes in the graph are to be treated. Swift dependencies with a textual interface, for example, must be built into a binary module by clients. Swift dependencies without a textual interface, with only a binary module, are to be used directly, without any up-to-date checks.
Note, this is distinct from Swift dependencies that have a textual interface, for which we also detect potential pre-build binary module candidates. Those are still reported in the `details` field of textual Swift dependencies as `prebuiltModuleCandidates`.
This refactoring allows us to drop ModuleInterfaceLoader when explicit modules
are enabled. Before this change, the dependencies scanner needs the loader to be
present to access functionalities like collecting prebuilt module candidates.
This is meant to address a problem that arises on incremental package builds:
A re-scan on an already-built package instead of a placeholder dependency produces a graph that contains a dependency consisting solely of the previously-built swiftmodule.
This is not the behaviour we would like. Instead, we should respect that this is a placeholder dependency and ensure that the dependency graph for that dependency itself captures the fact that a previously-built module exists using compiledModuleCandidates field of the dependency graph.
In the fast dependency scanner, depending on whether a module intrface was found via the import search path or framework search path, encode into the dependency graph Swift module details, whether a given module is a framework.
Some implicitly imported modules aren't printed in the textual interface file as explicit import,
e.g. SwiftOnoneSupport. We should check implicit imports and add them to the dependency graph.
Instead of replacing an interface file with its up-to-date compile module,
the dep-scanner should report potentially up-to-date module candidates either adjacent to
the interface file or in the prebuilt module cache. swift-driver should later pass down
these candidates to -compile-module-from-interface invocation and the front-end job
will check if one of the candidates is ready to use. The front-end job then either emits a forwarding
module to an up-to-date candidate or a binary module.
For the explicit module mode, swift-driver uses -compile-module-from-interface to
generate modules from interfaces found by the dependency scanner. However, we don't
need to build the binary module if up-to-date modules are available, either adjacent
to the interface file or in the prebuilt module cache directory. This patch teaches
dependencies scanner to report these ready-to-use binary modules.
Building each Swift module explicitly requires dependency PCMs to be built
with the exactly same deployment target version. This means we may need to
build a Clang module multiple times with different target triples.
This patch removes the -target arguments from the reported PCM build
arguments and inserts extraPcmArgs fields to each Swift module.
swift-driver can combine the generic PCM arguments with these extra arguments
to get the command suitable for building a PCM specifically for
that loading Swift module.
Module interface builder used to maintain a separate compiler instance for
building Swift modules. The configuration of this compiler instance is also
useful for dependencies scanner because it needs to emit front-end compiler invocation
for building Swift modules explicitly.
This patch refactor the configuration out to a delegate class, and the
delegate class is also used by the dependency scanner.
Additional flags in interface files may change parsing behavior like #if
statements. We should use a fresh ASTContext with these additional
flags when parsing interface files to collect imports.
rdar://62612027
