Always use the swiftinterfaces of modules under the public Frameworks
folder in the SDK. Distributed swiftmodules are unreliable, they should
only be used as a local cache. Let's avoid them when possible.
rdar://92037292
While implicitly building .swiftinterface, the interface may import other binary modules.
These binary modules may contain serialized search paths that have been obfuscated. To help
interface building commands recover these search paths, we need to pass down the obfuscators
to the module building commands.
rdar://87840268
When looking for a Swift module on disk, we were scanning all module search paths if they contain the module we are searching for. In a setup where each module is contained in its own framework search path, this scaled quadratically with the number of modules being imported. E.g. a setup with 100 modules being imported form 100 module search paths could cause on the order of 10,000 checks of `FileSystem::exists`. While these checks are fairly fast (~10µs), they add up to ~100ms.
To improve this, perform a first scan of all module search paths and list the files they contain. From this, create a lookup map that maps filenames to the search paths they can be found in. E.g. for
```
searchPath1/
Module1.framework
searchPath2/
Module1.framework
Module2.swiftmodule
```
we create the following lookup table
```
Module1.framework -> [searchPath1, searchPath2]
Module2.swiftmodule -> [searchPath2]
```
Instead of checking that the stdlib can be loaded in a variety of places, check it when setting up the compiler instance. This required a couple more checks to avoid loading the stdlib in cases where it’s not needed.
To be able to differentiate stdlib loading failures from other setup errors, make `CompilerInstance::setup` return an error message on failure via an inout parameter. Consume that error on the call side, replacing a previous, more generic error message, adding error handling where appropriate or ignoring the error message, depending on the context.
Ideally, module interface verification should fail the build when fatal error occurs when
type checking emitted module interfaces. However, we found it's hard to stage this phase in
because the ideal case requires all Swift adopters to have valid interfaces. This new front-end flag allows
driver to downgrade all interface verification errors to warnings as an intermediate step.
* Fix unnecessary one-time recompile of stdlib with -enable-ossa-flag
This includes a bit in the module format to represent if the module was
compiled with -enable-ossa-modules flag. When compiling a client module
with -enable-ossa-modules flag, all dependent modules are checked for this bit,
if not on, recompilation is triggered with -enable-ossa-modules.
* Updated tests
When we fall back to loading an arm64e module interface during an arm64 build, we want to compile it for the arm64 target so that it is fully compatible with the module that will load it, even though the flags in the file specify the arm64e target. Rewrite the sub-invocation's TargetTriple property in this specific situation. If the two targets differ by more than just the sub-architecture, we will continue to respect the -target flag in the file.
Fixes <rdar://83056545>.
We should hold off actually building the binary module file until it is imported.
`canImport` queries can happen, for example, during dependency scanning, when we do not wish to have the scanner tool execute any module builds.
Resolves rdar://82603098
We have implemented a libSwiftDriver-based tool to generate prebuilt module cache for
entire SDKs. Anchored on the same infrastructure, we could also generate ABI baselines
for entire SDKs.
Swift module building commands need -sdk because dependencies sometimes use
sdk-relative paths (prebuilt modules for example). Without -sdk, the command
will not be able to local these dependencies, leading to unnecessary
rebuilding from textual interfaces.
rdar://81177968
https://github.com/apple/swift/pull/37774 (related to rdar://72480261) has made it so that the target of built clang modules (even downstream from Swift interface dependencies) can be consistent with that of the main module. This means that when building transitive Clang dependencies of the main module, they will always have a matching triple to the main module itself (ensured with `-clang-target`). This means we no longer have to report the target triple in the set of `extraPCMArgs` which encode an interface-specific requirement for building its dependencies.
It's a known issue that we are using arm64e interfaces contents for the arm64 target,
meaning the encoded module flags are specifying -target arm64e-x-x instead of
-target arm64-x-x. Fortunately, we can tell the target arch from the interface file
name, so we could sanitize the target to use by inferring arch from the file name.
Before this change, we always use the Swift target triple to instantiate the internal
Clang instance. When loading a Swift module from the textual interface, we may pick up
a lower target triple to use to build the Swift module because the target is hard-coded
in the textual interface file. This implies we may end up building multiple versions of the
same Clang module, one for each target triple of the loading Swift module.
This change adds a new frontend flag -clang-target to allow clients to specify a
consistent clang target to use across the Swift module boundaries. This value won't change
because it's not part of .swiftinterface files.
swift-driver should pass down -clang-target for each frontend invocation, and its value should be
identical to -target.
Related to: rdar://72480261
Although users should usually use a prebuilt standard library, in those rare configurations where one needs to be built, the compiler appears to hang for several minutes, even on a trivial compilation. This commit adds a remark that's emitted when this happens, explaining that the standard library is being rebuilt and it will take a few minutes.
This mechanism allows the compiler to use a backup interface file to build into a binary module when
a corresponding interface file from the SDK is failing for whatever reasons. This mechansim should be entirely opaque
to end users except several diagnostic messages communicating backup interfaces are used.
Part of rdar://77676064
canImport should be able to take an additional parameter labeled by either version or
underlyingVersion. We need underlyingVersion for clang modules with Swift overlays because they
have separate version numbers. The library users are usually interested in checking the importability
of the underlying clang module instead of its Swift overlay.
Part of rdar://73992299
This will enable users to try out the '-enable-ossa-modules' flag if their
compiler supports it and get OSSA code on all inlinable code that they use. The
idea is that this is a nice way to stage this in and get more testing.
The specific implementation is that the module interface loader:
1. Knows if enable ossa modules is enabled not to search for any compiled
modules. We always rebuild from the interface file on the system.
2. Knows that if enable ossa modules is enabled to mixin a bit into the module
interface loader cache hash to ensure that we consider the specialized ossa
compiled modules to be different than the modules in that cache from the system.
This ensures that when said flag is enabled, the user transparently gets all
their code in OSSA form from transparent libraries.
`InterfaceSubContextDelegateImpl` causes sub-instances to inherit `-fmodule-map-file=` options.
Those Module Maps become file dependencies of all downstream PCMs and their depending Swift modules, even though they really aren't.
This causes frequent re-builds of the Module Cache contents when seemingly-unrelated files are touched.
Explicit Module Builds rely on these options for building Swift Interface files, so for now we just disable inheritance of these options in Implicit Module builds.
This refactoring allows us to drop ModuleInterfaceLoader when explicit modules
are enabled. Before this change, the dependencies scanner needs the loader to be
present to access functionalities like collecting prebuilt module candidates.
Tying InputFile to this option meant that every input that was not one of the explictly-blessed kinds was modeled as a Swift file.
With the new InputFile that infers file kinds, we no longer need CompilerInvocation::setInputKind
Prebuilt-module directory now contains a SystemVersion.plist file copied from the SDK
it's built from. This patch teaches the compiler to remark this version and the SDK version
when -Rmodule-interface-rebuild is specified. The difference between these versions could
help us debug unusable prebuilt modules.
