SILOptions::EnableSerializePackage info is lost.
SILVerifier needs this info to determine whether resilience
can be bypassed for decls serialized in a resiliently
built module when Package CMO optimization enabled.
This PR adds SerializePackageEnabled bit to Module format
and uses that in SILVerifier.
Resolves rdar://126157356
If a testable module is loaded from a non-testable import, ignore its
optional dependencies because the consumer should not use them. This
matches the behavior of the implicit build or the behavior how
forwarding module is created.
Follow-up adjustment for binary module selection in dependency scanning
time. If a testable binary module doesn't have an interface file, it
should be used even it might pull in more dependencies.
Teach scanner to pick and choose binary modules correctly based on if it
is testable import or not. Some situations that scanner need to be
careful when testable is involved:
* When it is a regular import, it should not import binary modules that
are built with -enable-testing, it should prefer interfaces if that is
available.
* When testable import, it should only load binary module and it should
make sure the internal imports from binary modules are actually
required for testable import to work.
If a testable import only find a regular binary module, dependency
scanner currently will just preceed with such module and leave the
diagnostics to swift-frontend, because the alternative (failed to find
module) can be confusing to users.
rdar://125914165
Improve swift dependency scanner by validating and selecting dependency
module into scanner. This provides benefits that:
* Build system does not need to schedule interface compilation task if
the candidate module is picked, it can just use the candidate module
directly.
* There is no need for forwarding module in the explicit module build.
Since the build system is coordinating the build, there is no need for
the forwarding module in the module cache to avoid duplicated work,
* This also correctly supports all the module loading modes in the
dependency scanner.
This is achieved by only adding validate and up-to-date binary module as
the candidate module for swift interface module dependency. This allows
caching build to construct the correct dependency in the CAS. If there
is a candidate module for the interface module, dependency scanner will
return a binary module dependency in the dependency graph.
The legacy behavior is mostly preserved with a hidden frontend flag
`-no-scanner-module-validation`, while the scanner output is mostly
interchangeable with new scanner behavior with `prefer-interface` module
loading mode except the candidate module will not be returned.
rdar://123711823
Add support for cross import modules by ingesting swiftoverlay files for
the cross import into CAS file system.
The long-term better fix will be just passing the cross import
information from scanner to swift-frontend so frontend doesn't need to
read overlay files again to figure out the cross import module.
rdar://123839248
In certain cases (e.g. using arm64e interface to build arm64 target),
the target needs to be updated when building swiftinterface. Push the
target overwrite as early as possible to swiftinterface parsing by
providing a preferred target to relevant functions. In such cases, the
wrong target is never observed by other functions to avoid errors like
the sub-invocation was partially setup for the wrong target.
Otherwise they may have module dependencies of their own which will not be detected by the scanner and included in the list of explicit inputs for compilation.
we only check if the loaded module is built from a package interface. This is
not enough as a binary module could just contain exportable decls if built with
experimental-skip-non-exportable-decls, essentially resulting in content equivalent
to interface content. This might be made a default behavior so this PR requires
a module to opt in to allow non-resilient access by a participating client in the
same package.
Since it affects module format, SWIFTMODULE_VERSION_MINOR is updated.
rdar://123651270
There are scenarios where different compilers are distributed with
compatible serialization format versions and the same tag. Distinguish
swiftmodules in such a case by assigning them to different distribution
channels. A compiler expecting a specific channel will only read
swiftmodules from the same channel. The channels should be defined by
downstream code as it is by definition vendor specific.
For development, a no-channel compiler loads or defining the env var
SWIFT_IGNORE_SWIFTMODULE_REVISION skips this new check.
rdar://123731777
When scanning finds a dependency in the same package, do not load
public/private swiftinterface since they do not have the package level
decl to compile the current module. Always prefer package module (if
enabled), or use binary module, unless it is building a public/private
swiftinterface file in which case the interface file is preferred.
This also does some clean up to sync up the code path between implicit
and explicit module finding path.
rdar://122356964
When `-enable-lazy-typecheck` is specified, serialization may be expected to
run on an AST containing invalid declarations since type checking may happen
on-demand, during serialization, in this mode. If the declarations that are
invalid are not skipped, then the compiler is likely to crash when attempting
to serialize them. Now, invalid declarations are skipped and an error is
emitted at the end of serialization to note that serialization failed.
Additionally, a new `-Rmodule-serialization` flag can be specified to request
more detailed information about module serialization failures. This would be
useful in a situation where lazy typechecking does not produce any diagnostic
for some reason, but module serialization fails and more information is
therefore required to debug.
Resolves rdar://123260476
Always prefer binary module when using @testable imports because the
swiftmodule rebuilt from interface cannot be imported as testable.
rdar://123120159
When a NoncopyableGenericsMismatch happens between the compiler and
stdlib, allow the compiler to rebuild the stdlib from its interface
instead of exiting with an error.
A swiftmodule can only be correctly ingested by a compiler
that has a matching state of using or not-using
NoncopyableGenerics.
The reason for this is fundamental: the absence of a Copyable
conformance in the swiftmodule indicates that a type is
noncopyable. Thus, if a compiler with NoncopyableGenerics
reads a swiftmodule that was not compiled with that feature,
it will think every type in that module is noncopyable.
Similarly, if a compiler with NoncopyableGenerics produces a
swiftmodule, there will be Copyable requirements on each
generic parameter that the compiler without the feature will
become confused about.
The solution here is to trigger a module mismatch, so that
the compiler re-generates the swiftmodule file using the
swiftinterface, which has been kept compatible with the compiler
regardless of whether the feature is enabled.
This reverts commit 3cc2831608.
The compiler's revision check has been relaxed since the feature was introduced
and so it's nos better to reduce the number of special code paths for LLDB in
the compiler to facilitate reasoning about it.
rdar://117824367
Add a new flag to enable package interface loading.
Use the last value of package-name in case of dupes.
Rename PrintInterfaceContentMode as InterfaceMode.
Update diagnostics.
Test package interface loading with various scenarios.
Test duplicate package-name.
It has an extension .package.swiftinterface and contains package decls
as well as SPIs and public/inlinable decls. When a module is loaded
from interface, it now looks up the package-name in the interface
and checks if the importer is in the same package. If so, it uses
that package interface found to load the module. If not, uses the existing
logic to load modules.
Resolves rdar://104617854
We only record these dependencies in CAS mode, because we require explicit PCH tasks to be produced for imported header of binary module dependencies. In the meantime, in non-CAS mode loading clients will consume the `.h` files encoded in the `.swiftmodules` directly.
Followup changes to SwiftDriver will enable explicit PCH compilation of such dependenceis, but for the time being restore prior behavior for non-CAS explicit module builds.
Resolves rdar://116006619
Rename -experimental-serialize-external-decls only to
-experimental-skip-non-exportable-decls in preparation for the flag being used
to influence more than just serialization.
Resolves rdar://116771543
Allow DependencyScanner to canonicalize path using a prefix map. When
option `-scanner-prefix-map` option is used, dependency scanner will
remap all the input paths in following:
* all the paths in the CAS file system or clang include tree
* all the paths related to input on the command-line returned by scanner
This allows all the input paths to be canonicalized so cache key can be
computed reguardless of the exact on disk path.
The sourceFile field is not remapped so build system can track the exact
file as on the local file system.
'ModuleDependencyScanner' maintains a Thread Pool along with a pool of workers
which are capable of executing a filesystem lookup of a named module dependency.
When resolving imports of a given Swift module, each import's resolution
operation can be issued asunchronously.
From being a scattered collection of 'static' methods in ScanDependencies.cpp
and member methods of ASTContext. This makes 'ScanDependencies.cpp' much easier
to read, and abstracts the actual scanning logic away to a place with common
state which will make it easier to reason about in the future.
- Add a flag to the serialized module (IsEmbeddedSwiftModule)
- Check on import that the mode matches (don't allow importing non-embedded module in embedded mode and vice versa)
- Drop TBD support, it's not expected to work in embedded Swift for now
- Drop auto-linking backdeploy libraries, it's not expected to backdeploy embedded Swift for now
- Drop prespecializations, not expected to work in embedded Swift for now
- Use CMO to serialize everything when emitting an embedded Swift module
- Change SILLinker to deserialize/import everything when importing an embedded Swift module
- Add an IR test for importing modules
- Add a deserialization validation test
This option is designed to be used in conjunction with
`-experimental-lazy-typecheck` and `-experimental-skip-all-function-bodies`
when emitting a resilient module. The emitted binary module should contain only
the decls needed by clients and should contain roughly the same contents as it
would if the corresponding swiftinterface were emitted instead and then built.
This functionality is a work in progress. Some parts of the AST may still get
typechecked unnecessarily. Additionally, serialization does not trigger the
appropriate typechecking requests for some ASTs and then fails due to missing
types.
Resolves rdar://114230586
Instead of the code querying the compiler's built-in Clang instance, refactor the
dependency scanner to explicitly keep track of module output path. It is still
set according to '-module-cache-path' as it has been prior to this change, but
now the scanner can use a different module cache for scanning PCMs, as specified
with '-clang-scanner-module-cache-path', without affecting module output path.
Resolves rdar://113222853
Reformatting everything now that we have `llvm` namespaces. I've
separated this from the main commit to help manage merge-conflicts and
for making it a bit easier to read the mega-patch.
This is phase-1 of switching from llvm::Optional to std::optional in the
next rebranch. llvm::Optional was removed from upstream LLVM, so we need
to migrate off rather soon. On Darwin, std::optional, and llvm::Optional
have the same layout, so we don't need to be as concerned about ABI
beyond the name mangling. `llvm::Optional` is only returned from one
function in
```
getStandardTypeSubst(StringRef TypeName,
bool allowConcurrencyManglings);
```
It's the return value, so it should not impact the mangling of the
function, and the layout is the same as `std::optional`, so it should be
mostly okay. This function doesn't appear to have users, and the ABI was
already broken 2 years ago for concurrency and no one seemed to notice
so this should be "okay".
I'm doing the migration incrementally so that folks working on main can
cherry-pick back to the release/5.9 branch. Once 5.9 is done and locked
away, then we can go through and finish the replacement. Since `None`
and `Optional` show up in contexts where they are not `llvm::None` and
`llvm::Optional`, I'm preparing the work now by going through and
removing the namespace unwrapping and making the `llvm` namespace
explicit. This should make it fairly mechanical to go through and
replace llvm::Optional with std::optional, and llvm::None with
std::nullopt. It's also a change that can be brought onto the
release/5.9 with minimal impact. This should be an NFC change.
Teach swift dependency scanner to use CAS to capture the full dependencies for a build and construct build commands with immutable inputs from CAS.
This allows swift compilation caching using CAS.
