This change refactors the module loaders to explicitly take a parameter indicating whether or not the loader is handling a 'canImport' query, in order to avoid emitting an error when finding a dependency Swift binary module with only imcompatible architecture variants present.
Resolves rdar://161175498
A swiftmodule can only be correctly ingested by a compiler
that has a matching state of using or not-using
NoncopyableGenerics.
The reason for this is fundamental: the absence of a Copyable
conformance in the swiftmodule indicates that a type is
noncopyable. Thus, if a compiler with NoncopyableGenerics
reads a swiftmodule that was not compiled with that feature,
it will think every type in that module is noncopyable.
Similarly, if a compiler with NoncopyableGenerics produces a
swiftmodule, there will be Copyable requirements on each
generic parameter that the compiler without the feature will
become confused about.
The solution here is to trigger a module mismatch, so that
the compiler re-generates the swiftmodule file using the
swiftinterface, which has been kept compatible with the compiler
regardless of whether the feature is enabled.
If a module was first read using the adjacent swiftmodule and then
reloaded using the swiftinterface, we would do an up to date check on
the adjacent module but write out the unit using the swiftinterface.
This would cause the same modules to be indexed repeatedly for the first
invocation using a new SDK. On the next run we would instead raad the
swiftmodule from the cache and thus the out of date check would match
up.
The impact of this varies depending on the size of the module graph in
the initial compilation and the number of jobs started at the same time.
Each SDK dependency is re-indexed *and* reloaded, which is a drain on
both CPU and memory. Thus, if many jobs are initially started and
they're all going down this path, it can cause the system to run out of
memory very quickly.
Resolves rdar://103119964.
Change the way swiftmodules built against a different SDK than their
clients are rejected. This makes them silently ignored when the module
can be rebuilt from their swiftinterface, instead of reporting a hard
error.
rdar://93257769
* Fix unnecessary one-time recompile of stdlib with -enable-ossa-flag
This includes a bit in the module format to represent if the module was
compiled with -enable-ossa-modules flag. When compiling a client module
with -enable-ossa-modules flag, all dependent modules are checked for this bit,
if not on, recompilation is triggered with -enable-ossa-modules.
* Updated tests
We should hold off actually building the binary module file until it is imported.
`canImport` queries can happen, for example, during dependency scanning, when we do not wish to have the scanner tool execute any module builds.
Resolves rdar://82603098
This will enable users to try out the '-enable-ossa-modules' flag if their
compiler supports it and get OSSA code on all inlinable code that they use. The
idea is that this is a nice way to stage this in and get more testing.
The specific implementation is that the module interface loader:
1. Knows if enable ossa modules is enabled not to search for any compiled
modules. We always rebuild from the interface file on the system.
2. Knows that if enable ossa modules is enabled to mixin a bit into the module
interface loader cache hash to ensure that we consider the specialized ossa
compiled modules to be different than the modules in that cache from the system.
This ensures that when said flag is enabled, the user transparently gets all
their code in OSSA form from transparent libraries.
This refactoring allows us to drop ModuleInterfaceLoader when explicit modules
are enabled. Before this change, the dependencies scanner needs the loader to be
present to access functionalities like collecting prebuilt module candidates.
We need ClangImporterOptions to be persistent for several scenarios: (1)
when creating a sub-ASTContext to build Swift modules from interfaces; and
(2) when creating a new Clang instance to invoke Clang dependencies scanner.
This change is NFC.
In the fast dependency scanner, depending on whether a module intrface was found via the import search path or framework search path, encode into the dependency graph Swift module details, whether a given module is a framework.
* Remove dead ModuleSourceInfoFilename parameters
These were never actually used; we might find a way to bring them back later.
* Introduce SerializedModuleBaseName
This is intended to replace the _n_ filename parameters that tend to get passed around in the SerializedModuleLoader classes.
* Manipulate currPath in SerializedModuleLoader less often
* Don’t pass raw paths around SerializedModuleLoader
Only pass base names.
* Regularize module file opening functions
✔ More informative error messages in case of crashes.
✔ Handling and documenting different cases.
✔ Test cases for different cases.
✔ Make SDKDependencies.swift pass again.
After setting up the .swiftsourceinfo file, this patch starts to actually serialize
and de-serialize source locations for declaration. The binary format of .swiftsourceinfo
currently contains these three records:
BasicDeclLocs: a hash table mapping from a USR ID to a list of basic source locations. The USR id
could be retrieved from the following DeclUSRs record using an actual decl USR. The basic source locations
include a file ID and the results from Decl::getLoc(), ValueDecl::getNameLoc(), Decl::getStartLoc() and Decl::getEndLoc().
The file ID could be used to retrieve the actual file name from the following SourceFilePaths record.
Each location is encoded as a line:column pair.
DeclUSRS: a hash table mapping from USR to a USR ID used by location records.
SourceFilePaths: a hash table mapping from a file ID to actual file name.
BasicDeclLocs should be sufficient for most diagnostic cases. If additional source locations
are needed, we could always add new source location records without breaking the backward compatibility.
When de-serializing the source location from a module-imported decl, we calculate its USR, retrieve the USR ID
from the DeclUSRS record, and use the USR ID to look up the basic location list in the BasicDeclLocs record.
For more details about .swiftsourceinfo file: https://forums.swift.org/t/proposal-emitting-source-information-file-during-compilation
The clang importer has to deal with two virtual file systems, one coming
from clang, and one coming from swift. Currently, if both are set, we
emit a diagnostic that we'll pick the swift one.
This commit changes that, by merging the two virtual file systems into a
single overlay file system, and using that. To make this possible, we
always initialize the file manager with an overlay file system. In the
clang importer, we then create a new overlay file system, starting with
the one coming from clang, and adding overlays from swift on top.
The motivation for this change is the reproducer infrastructure in LLDB,
which adds a third virtual file system to the mix.
The dependency file that is being generated should not escape : and # in
the filename. This makes the behaviour of the filename escaping similar
to clang and GCC and fixes incorrect quoting of paths on Windows.
