For clients, such as the debugger, who do not have access the full
output of the dependency scanner, it is a huger performance and
correctness improvement if each explicitly built Swift module not just
serialized all its Clang .pcm dependencies (via the serialized Clang
compiler invocation) but also its direct Swift module dependencies.
This patch changes the Swift module format to store the absolute path
or cas cache key for each dependency in the INPUT block, and makes
sure the deserialization makes these available to the ESML.
rdar://150969755
To allow prefix mapping of the bridging header to achieve cache hit when
source files are located in different location, the generated chained
bridging header should not include absolute paths of the headers. Fix
the problem by concat the chained bridging header together.
Fixes: https://github.com/swiftlang/swift/issues/84088
Introduce an experimental feature DeferredCodeGen, that defers the
generation of LLVM IR (and therefore object code) for all entities
within an Embedded Swift module unless they have explicitly requested
to not be emitted into the client (e.g., with
`@_neverEmitIntoClient`).
This feature is meant to generalize and subsume
-emit-empty-object-file, relying on lazy emission of entities rather
than abruptly ending the compilation pipeline before emitting any IR.
Part of rdar://158363967.
In addition to skipping it on textual Swift module dependencies which were built without C++ interop enabled, also skip it over similarly on binary Swift dependencies
Unlike with implicitly-built modules (prior to Swift 6 mode), explicitly-built modules require that all search paths be specified explicitly and no longer inherit search paths serialized into discovered Swift binary modules. This behavior was never intentional and is considered a bug. This change adds a diagnostic note to a scan failure: for each binary Swift module dependency, the scanner will attempt to execute a dependency scanning query for each serialized search path inside that module. If such diagnostic query returns a result, a diagnostic will be emitted to inform the user that the dependency may be found in the search path configuration of another Swift binary module dependency, specifying which search path contains the "missing" module, and stating that such search paths are not automatically inherited by the current compilation.
For now the semantics provided by `@extensible` keyword on per-enum
basis. We might return this as an upcoming feature in the future with
a way to opt-out.
An "abstract" ProtocolConformanceRef is a conformance of a type
parameter or archetype to a given protocol. Previously, we would only
store the protocol requirement itself---but not track the actual
conforming type, requiring clients of ProtocolConformanceRef to keep
track of this information separately.
Record the conforming type as part of an abstract ProtocolConformanceRef,
so that clients will be able to recover it later. This is handled by a uniqued
AbstractConformance structure, so that ProtocolConformanceRef itself stays one
pointer.
There remain a small number of places where we create an abstract
ProtocolConformanceRef with a null type. We'll want to chip away at
those and establish some stronger invariants on the abstract conformance
in the future.
When `ExtensibleEnums` flag is set, it's going to be reflected in
the module file produced by the compiler to make sure that consumers
know that non-`@frozen` enumerations can gain new cases in the
future and switching cannot be exhaustive.
Add ability to automatically chaining the bridging headers discovered from all
dependencies module when doing swift caching build. This will eliminate all
implicit bridging header imports from the build and make the bridging header
importing behavior much more reliable, while keep the compatibility at maximum.
For example, if the current module A depends on module B and C, and both B and
C are binary modules that uses bridging header, when building module A,
dependency scanner will construct a new header that chains three bridging
headers together with the option to build a PCH from it. This will make all
importing errors more obvious while improving the performance.
Extend the module trace format with a field indicating whether a given
module, or any module it depends on, was compiled with strict memory
safety enabled. This separate output from the compiler can be used as
part of an audit to determine what parts of Swift programs are built
with strict memory safety checking enabled.
When serializing the module interface path of an interface that
is part of the SDK, we serialize relative to the SDK path. During
deserialization we need to know if a path was serialized relative
to the SDK or not. The existing logic assumes any relative path
has been serialized relative to the SDK, which makes it impossible
to compile modules from relative swiftinterface paths that are not
part of the SDK.
Update the swiftmodule file to include an attribute to show if the
path was serialized relative to the SDK or not, which is used
during deserialization to correctly reconstruct the interface path.
This change ensures that when loading some module dependency 'Bar' which has a package-only dependency on 'Foo', only the following clients attempt to resolve/load 'Foo':
- Source compilation with package-name equal to that of 'Bar'.
- Textual interface compilation of a *'package'* interface with package-name equal to that of 'Bar'.
Ensuring that the following kinds of clients do not attempt to resolve/load 'Foo':
- Source compilation with package-name different to that of 'Bar'
- Textual interface compilation of a public or private interface, regardless of package name.
This fixes the behavior where previously compilation of a Swift textual interface dependency 'X' from its public or private interface, with an interface-specified package-name, from a client without a matching package-name, resulted in a lookup of package-only dependencies of modules loaded into 'X'. This behavior is invalid if we are not building from the package textual interface, becuase the module dependency graph is defined by the package name of the source client, not individual module dependency package name. i.e. In-package module dependencies are resolved/loaded only if the parent source compile matches the package name.
Resolves rdar://139979180
It might be unexpected to future users that `-swift-compiler-version`
would produce a version aligned to .swiftinterface instead of one used
to build the .swiftmodule file. To avoid this possible confusion, let's
scope down the version to `-interface-compiler-version` flag and
`SWIFT_INTERFACE_COMPILER_VERSION` option in the module.
Add function to handle all macro dependencies kinds in the scanner,
including taking care of the macro definitions in the module interface
for its client to use. The change involves:
* Encode the macro definition inside the binary module
* Resolve macro modules in the dependencies scanners, including those
declared inside the dependency modules.
* Propagate the macro defined from the direct dependencies to track
all the potentially available modules inside a module compilation.
This makes sure that Swift respects `-Xcc -stdlib=libc++` flags.
Clang already has existing logic to discover the system-wide libc++ installation on Linux. We rely on that logic here.
Importing a Swift module that was built with a different C++ stdlib is not supported and emits an error.
The Cxx module can be imported when compiling with any C++ stdlib. The synthesized conformances, e.g. to CxxRandomAccessCollection also work. However, CxxStdlib currently cannot be imported when compiling with libc++, since on Linux it refers to symbols from libstdc++ which have different mangled names in libc++.
rdar://118357548 / https://github.com/swiftlang/swift/issues/69825
This patch allows controlling the automatic import of private dependencies
separately from the DebuggerSupport option, which currently also triggers this
behavior. With explicit modules + precise compiler invocations LLDB is moving
towards no longer needing this behavior.
rdar://133088201
(cherry picked from commit a1ba7159e3)
When the dependency scanner picks a pre-built binary module candidate for a given dependency, it needs to be able to attempt to resolve its cross-import overlays relative to the textual interface that the binary module was built from. For example, if a collection of binary modules are located in, and resolved as dependencies from, a pre-built module directory, the scanner must lookup their corresponding cross-import overlays relative to the defining interface as read out from the binary module's MODULE_INTERFACE_PATH. https://github.com/swiftlang/swift/pull/70817 ensures that binary modules serialize the path to their defining textual interface.
Resolves rdar://130778577
SILOptions::EnableSerializePackage info is lost.
SILVerifier needs this info to determine whether resilience
can be bypassed for decls serialized in a resiliently
built module when Package CMO optimization enabled.
This PR adds SerializePackageEnabled bit to Module format
and uses that in SILVerifier.
Resolves rdar://126157356
we only check if the loaded module is built from a package interface. This is
not enough as a binary module could just contain exportable decls if built with
experimental-skip-non-exportable-decls, essentially resulting in content equivalent
to interface content. This might be made a default behavior so this PR requires
a module to opt in to allow non-resilient access by a participating client in the
same package.
Since it affects module format, SWIFTMODULE_VERSION_MINOR is updated.
rdar://123651270
A swiftmodule can only be correctly ingested by a compiler
that has a matching state of using or not-using
NoncopyableGenerics.
The reason for this is fundamental: the absence of a Copyable
conformance in the swiftmodule indicates that a type is
noncopyable. Thus, if a compiler with NoncopyableGenerics
reads a swiftmodule that was not compiled with that feature,
it will think every type in that module is noncopyable.
Similarly, if a compiler with NoncopyableGenerics produces a
swiftmodule, there will be Copyable requirements on each
generic parameter that the compiler without the feature will
become confused about.
The solution here is to trigger a module mismatch, so that
the compiler re-generates the swiftmodule file using the
swiftinterface, which has been kept compatible with the compiler
regardless of whether the feature is enabled.
From being a scattered collection of 'static' methods in ScanDependencies.cpp
and member methods of ASTContext. This makes 'ScanDependencies.cpp' much easier
to read, and abstracts the actual scanning logic away to a place with common
state which will make it easier to reason about in the future.
- Add a flag to the serialized module (IsEmbeddedSwiftModule)
- Check on import that the mode matches (don't allow importing non-embedded module in embedded mode and vice versa)
- Drop TBD support, it's not expected to work in embedded Swift for now
- Drop auto-linking backdeploy libraries, it's not expected to backdeploy embedded Swift for now
- Drop prespecializations, not expected to work in embedded Swift for now
- Use CMO to serialize everything when emitting an embedded Swift module
- Change SILLinker to deserialize/import everything when importing an embedded Swift module
- Add an IR test for importing modules
- Add a deserialization validation test
A @testable import allows a client to call internal decls which may
refer to non-public dependencies. To support such a use case, load
non-public transitive dependencies of a module when it's imported
@testable from the main module.
This replaces the previous behavior where we loaded those dependencies
for any modules built for testing. This was risky as we would load more
module for any debug build, opening the door to a different behavior
between debug and release builds. In contrast, applying this logic to
@testable clients will only change the behavior of test targets.
rdar://107329303
Differentiate `internal` and `fileprivate` imports from
implementation-only imports at the module-wide level to offer a
different module loading strategy. The main difference is for non-public
imports from a module with testing enabled to be loaded by transitive
clients.
Ideally, we would only load transitive non-public dependencies on
testable imports of the middle module. The current module loading logic
doesn't allow for this behavior easily as a module may be first loaded
for a normal import and extra dependencies would have to be loaded on
later imports. We may want to refactor the module loading logic to allow
this if needed.
rdar://106514965
When loading a swiftmodule A, read its package information to tell if
the current client should load A's dependencies imports by a package
import. Only clients belonging to the same package as A should load
those dependencies, clients outside of the package likely don't have
access to those dependencies.
This is specific to swiftmodules as swiftinterfaces never display a
package-only import. Clients are unaware of package dependencies when
building against a swiftinterface.
rdar://106164813
If a module was first read using the adjacent swiftmodule and then
reloaded using the swiftinterface, we would do an up to date check on
the adjacent module but write out the unit using the swiftinterface.
This would cause the same modules to be indexed repeatedly for the first
invocation using a new SDK. On the next run we would instead raad the
swiftmodule from the cache and thus the out of date check would match
up.
The impact of this varies depending on the size of the module graph in
the initial compilation and the number of jobs started at the same time.
Each SDK dependency is re-indexed *and* reloaded, which is a drain on
both CPU and memory. Thus, if many jobs are initially started and
they're all going down this path, it can cause the system to run out of
memory very quickly.
Resolves rdar://103119964.
Introduce a new flag `-export-as` to specify a name used to identify the
target module in swiftinterfaces. This provides an analoguous feature
for Swift module as Clang's `export_as` feature.
In practice it should be used when a lower level module `MyKitCore` is
desired to be shown publicly as a downstream module `MyKit`. This should
be used in conjunction with `@_exported import MyKitCore` from `MyKit`
that allows clients to refer to all services as being part of `MyKit`,
while the new `-export-as MyKit` from `MyKitCore` will ensure that the
clients swiftinterfaces also use the `MyKit` name for all services.
In the current implementation, the export-as name is used in the
module's clients and not in the declarer's swiftinterface (e.g.
`MyKitCore`'s swiftinterface still uses the `MyKitCore` module name).
This way the module swiftinterface can be verified. In the future, we
may want a similar behavior for other modules in between `MyKitCore` and
`MyKit` as verifying a swiftinterface referencing `MyKit` without it
being imported would fail.
rdar://103888618
