Push the top level logic for writing out swiftmodules and associated files into the frontend library which has access to all the necessary dependencies.
For release-management purposes during development, LLDB's embedded Swift
compiler's version number can sometimes be off-by-one in the last digit
compared to the Swift compiler.
This patch restores the old behavior from before 17183629e4.
rdar://101299168
Specifically, we get an additional table like thing called sil_moveonlydeinit. It looks as follows:
sil_moveonlydeinit TYPE {
@FUNC_NAME
}
It always has a single entry.
Previously, when evaluating a `#if canImport(Module, _version: 42)` directive the compiler could diagnose and ignore the directive under the following conditions:
- The associated binary module is corrupt/bogus.
- The .tbd for an underlying Clang module is missing a current-version field.
This behavior is surprising when there is a valid `.swiftinterface` available and it only becomes apparent when building against an SDK with an old enough version of the module that the version in the `.swiftinterface` is too low, making this failure easy to miss. Some modules have different versioning systems for their Swift and Clang modules and it can also be intentional for a distributed binary `.swiftmodule` to contain bogus data (to force the compiler to recompile the `.swiftinterface`) so we need to handle both of these cases gracefully and predictably.
Now the compiler will enumerate all module loaders, ask each of them to attempt to parse the module version and then consistently use the parsed version from a single source. The `.swiftinterface` is preferred if present, then the binary module if present, and then finally the `.tbd`. The `.tbd` is still always used exclusively for the `_underlyingVersion` variant of `canImport()`.
Resolves rdar://88723492
The ObjCMethodLookupTable for protocols was not being serialized and rebuilt on load, so NominalTypeDecl::lookupDirect() on selectors was not working correctly for deserialized types. Correct this oversight.
Change the way swiftmodules built against a different SDK than their
clients are rejected. This makes them silently ignored when the module
can be rebuilt from their swiftinterface, instead of reporting a hard
error.
rdar://93257769
llvm/llvm-project d0262c2394f46bb7da2a75529413d625c70908e5 added a new
default bool param to the two constructors in `SmallVectorMemoryBuffer`.
Since `options.OutputPath` is a `const char *` and that can be promoted
to a `bool`, the constructor being called was changed to the first
constructor (with a default buffer name) - promotion is preferred over
conversion.
Convert the various output paths to a `StringRef` - all their uses
converted to `StringRef` anyway. Also specify the default parameter in
order to maintain the old behaviour, which didn't require a null
terminator.
The main point of this change is to make sure that a shared function always has a body: both, in the optimizer pipeline and in the swiftmodule file.
This is important because the compiler always needs to emit code for a shared function. Shared functions cannot be referenced from outside the module.
In several corner cases we missed to maintain this invariant which resulted in unresolved-symbol linker errors.
As side-effect of this change we can drop the shared_external SIL linkage and the IsSerializable flag, which simplifies the serialization and linkage concept.
ABI descriptors should always be emitted as sidecars for library-evolution-enabled modules.
However, generating these files requires traversing the entire module (like indexing), which may
hit additional deserialization issues. To unblock builds, this patch introduces a flag to skip
the traversing logic so that we emit an empty ABI descriptor file. The empty file serves as
a placeholder so that build system doesn't need to know the details.
When looking for a Swift module on disk, we were scanning all module search paths if they contain the module we are searching for. In a setup where each module is contained in its own framework search path, this scaled quadratically with the number of modules being imported. E.g. a setup with 100 modules being imported form 100 module search paths could cause on the order of 10,000 checks of `FileSystem::exists`. While these checks are fairly fast (~10µs), they add up to ~100ms.
To improve this, perform a first scan of all module search paths and list the files they contain. From this, create a lookup map that maps filenames to the search paths they can be found in. E.g. for
```
searchPath1/
Module1.framework
searchPath2/
Module1.framework
Module2.swiftmodule
```
we create the following lookup table
```
Module1.framework -> [searchPath1, searchPath2]
Module2.swiftmodule -> [searchPath2]
```
We noticed some Swift clients rely on the serialized search paths in the module to
find dependencies and droping these paths altogether can lead to build failures like
rdar://85840921.
This change teaches the serialization to obfuscate the search paths and the deserialization
to recover them. This allows clients to keep accessing these paths without exposing
them when shipping the module to other users.
We've recently added the -experimental-hermetic-seal-at-link compiler flag,
which turns on aggressive dead-stripping optimizations and assumes that library
code can be optimized against client code because all users of the library
code/types are present at link/LTO time. This means that any module that's
built with -experimental-hermetic-seal-at-link requires all clients of this
module to also use -experimental-hermetic-seal-at-link. This PR enforces that
by storing a bit in the serialized module, and checking the bit when importing
modules.
This commit adds a new frontend flag that applies debug path prefixing to the
paths serialized in swiftmodule files. This makes it possible to use swiftmodule
files that have been built on different machines by applying the inverse map
when debugging, in a similar fashion to source path prefixing.
The inverse mapping in LLDB will be handled in a follow up PR.
Second pass at #39138
Tests updated to handle windows path separators.
This reverts commit f5aa95b381.
* Fix unnecessary one-time recompile of stdlib with -enable-ossa-flag
This includes a bit in the module format to represent if the module was
compiled with -enable-ossa-modules flag. When compiling a client module
with -enable-ossa-modules flag, all dependent modules are checked for this bit,
if not on, recompilation is triggered with -enable-ossa-modules.
* Updated tests
This commit adds the `-prefix-serialized-debugging-options` flag,
which is used to apply the debug prefix map to serialized debugging
options embedded in the swiftmodule files.
Introduce a new loading restriction that is more strict than the serialization
version check on swiftmodules. Tagged compilers will only load
library-evolution enabled swiftmodules that are produced by a compiler with the
exact same revision id. This will be more reliable in production
environments than using the serialization version which we forgot to
update from time to time. This shouldn't affect development compilers that
will still load any module with a compatible serialization version.
rdar://83105234
Serialize the canonical name of the SDK used when building a swiftmodule
file and use it to ensure that the swiftmodule file is loaded only with
the same SDK. The SDK name must be passed down from the frontend.
This will report unsupported configurations like:
- Installing roots between incompatible SDKs without deleting the
swiftmodule files.
- Having multiple targets in the same project using different SDKs.
- Loading a swiftmodule created with a newer SDK (and stdlib) with an
older SDK.
All of these lead to hard to investigate deserialization failures and
this change should detect them early, before reaching a deserialization
failure.
rdar://78048939
We should hold off actually building the binary module file until it is imported.
`canImport` queries can happen, for example, during dependency scanning, when we do not wish to have the scanner tool execute any module builds.
Resolves rdar://82603098
The MemoryBuffer loader is used by LLDB during debugging to import binary Swift
modules from .swift_ast sections. Modules imported from .swift_ast sections are
never produced from textual interfaces. By disabling resilience the expression
evaluator in the debugger can directly access private members.
rdar://79462915
Rework Sendable checking to be completely based on "missing"
conformances, so that we can individually diagnose missing Sendable
conformances based on both the module in which the conformance check
happened as well as where the type was declared. The basic rules here
are to only diagnose if either the module where the non-Sendable type
was declared or the module where it was checked was compiled with a
mode that consistently diagnoses `Sendable`, either by virtue of
being Swift 6 or because `-warn-concurrency` was provided on the
command line. And have that diagnostic be an error in Swift 6 or
warning in Swift 5.x.
There is much tuning to be done here.
It's a known issue that we are using arm64e interfaces contents for the arm64 target,
meaning the encoded module flags are specifying -target arm64e-x-x instead of
-target arm64-x-x. Fortunately, we can tell the target arch from the interface file
name, so we could sanitize the target to use by inferring arch from the file name.
Foundation imports CoreFoundation with `@_implementationOnly`,
so CoreFoundation's modulemap won't be read, and the dependent libraries
of CoreFoundation will not be automatically linked when using static
linking.
For example, CoreFoundation depends on libicui18n and it's modulemap has
`link "icui18n"` statement. If Foundation imports CoreFoundation with
`@_implementationOnly` as a private dependency, the toolchain doesn't have
CoreFoundation's modulemap and Foundation's swiftmodule doesn't import
CoreFoundation. So the swiftc can't know that libicui18n is required.
This new option will add LINK_LIBRARY entry in swiftmodule to
specify dependent libraries (in the example case, Foundation's
swiftmodule should have LINK_LIBRARY entry of libicui18n)
See also: [Autolinking behavior of @_implementationOnly with static linking](https://forums.swift.org/t/autolinking-behavior-of-implementationonly-with-static-linking/44393)
