Introduce a last resort check reporting references to
implementation-only dependencies that would appear in the generated
swiftmodule. This check is applied at serialization, long after
exportability checking applied at typechecking. It should act as a back
stop to references missed by typechecking or @_implementationOnly decls
that should have been skipped.
This check is gated behind CheckImplementationOnlyStrict and should be
used with embedded only.
rdar://160697599
This PR is another attempt at landing #76903. The changes compared to
the original PR:
* Instead of increasing the size of SILDeclRef, store the necessary type
information in a side channel using withClosureTypeInfo.
* Rely on SGFContext to get the right ClangType
* Extend BridgingConversion with an AbstractionPattern to store the
original clang type.
* The PR above introduced a crash during indexing system modules that
references foreign types coming from modules imported as
implementation only. These entities are implementation details so they
do not need to be included during serialization. This PR adds a test
and adds logic to exclude such clang types in the serialization
process.
rdar://131321096&141786724
An "abstract" ProtocolConformanceRef is a conformance of a type
parameter or archetype to a given protocol. Previously, we would only
store the protocol requirement itself---but not track the actual
conforming type, requiring clients of ProtocolConformanceRef to keep
track of this information separately.
Record the conforming type as part of an abstract ProtocolConformanceRef,
so that clients will be able to recover it later. This is handled by a uniqued
AbstractConformance structure, so that ProtocolConformanceRef itself stays one
pointer.
There remain a small number of places where we create an abstract
ProtocolConformanceRef with a null type. We'll want to chip away at
those and establish some stronger invariants on the abstract conformance
in the future.
This patch adds support for serialization and deserialization of
debug scopes.
Debug scopes are serialized in post order and enablement is
controlled through the experimental-serialize-debug-info flag which
is turned off by default. Functions only referred to by these debug
scopes are deserialized as zombie functions directly.
When `-enable-lazy-typecheck` is specified, serialization may be expected to
run on an AST containing invalid declarations since type checking may happen
on-demand, during serialization, in this mode. If the declarations that are
invalid are not skipped, then the compiler is likely to crash when attempting
to serialize them. Now, invalid declarations are skipped and an error is
emitted at the end of serialization to note that serialization failed.
Additionally, a new `-Rmodule-serialization` flag can be specified to request
more detailed information about module serialization failures. This would be
useful in a situation where lazy typechecking does not produce any diagnostic
for some reason, but module serialization fails and more information is
therefore required to debug.
Resolves rdar://123260476
This option is designed to be used in conjunction with
`-experimental-lazy-typecheck` and `-experimental-skip-all-function-bodies`
when emitting a resilient module. The emitted binary module should contain only
the decls needed by clients and should contain roughly the same contents as it
would if the corresponding swiftinterface were emitted instead and then built.
This functionality is a work in progress. Some parts of the AST may still get
typechecked unnecessarily. Additionally, serialization does not trigger the
appropriate typechecking requests for some ASTs and then fails due to missing
types.
Resolves rdar://114230586
llvm::SmallSetVector changed semantics
(https://reviews.llvm.org/D152497) resulting in build failures in Swift.
The old semantics allowed usage of types that did not have an
`operator==` because `SmallDenseSet` uses `DenseSetInfo<T>::isEqual` to
determine equality. The new implementation switched to using
`std::find`, which internally uses `operator==`. This type is used
pretty frequently with `swift::Type`, which intentionally deletes
`operator==` as it is not the canonical type and therefore cannot be
compared in normal circumstances.
This patch adds a new type-alias to the Swift namespace that provides
the old semantic behavior for `SmallSetVector`. I've also gone through
and replaced usages of `llvm::SmallSetVector` with the
`Swift::SmallSetVector` in places where we're storing a type that
doesn't implement or explicitly deletes `operator==`. The changes to
`llvm::SmallSetVector` should improve compile-time performance, so I
left the `llvm::SmallSetVector` where possible.
This is phase-1 of switching from llvm::Optional to std::optional in the
next rebranch. llvm::Optional was removed from upstream LLVM, so we need
to migrate off rather soon. On Darwin, std::optional, and llvm::Optional
have the same layout, so we don't need to be as concerned about ABI
beyond the name mangling. `llvm::Optional` is only returned from one
function in
```
getStandardTypeSubst(StringRef TypeName,
bool allowConcurrencyManglings);
```
It's the return value, so it should not impact the mangling of the
function, and the layout is the same as `std::optional`, so it should be
mostly okay. This function doesn't appear to have users, and the ABI was
already broken 2 years ago for concurrency and no one seemed to notice
so this should be "okay".
I'm doing the migration incrementally so that folks working on main can
cherry-pick back to the release/5.9 branch. Once 5.9 is done and locked
away, then we can go through and finish the replacement. Since `None`
and `Optional` show up in contexts where they are not `llvm::None` and
`llvm::Optional`, I'm preparing the work now by going through and
removing the namespace unwrapping and making the `llvm` namespace
explicit. This should make it fairly mechanical to go through and
replace llvm::Optional with std::optional, and llvm::None with
std::nullopt. It's also a change that can be brought onto the
release/5.9 with minimal impact. This should be an NFC change.
Push the top level logic for writing out swiftmodules and associated files into the frontend library which has access to all the necessary dependencies.
Write the real module name for XRefs in swiftmodule files instead of the
exported module name, from `export_as` declarations in module maps.
Swiftmodule files are internal details now, they should represent the
truth.
We keep using the exported module name for the extensions lookup table
as clients should still use the exported name. However we may need to
write both alternatives in the lookup table in the future if extensions
can't be found from clients not importing the exported as module.
rdar://90272035
We now schedule conformance emissions in basically the same way
we do for types and declarations, which means that we'll emit them
uniquely in the module file instead of redundantly at every use.
This should produce substantially smaller module files overall,
especially for modules that heavily use generics. It also means
that we can remove all the unfortunate code to support using
different abbrev codes for them in different bitcode blocks.
Requirement lists are now emitted inline in the records that need
them instead of as trailing records. I think this will improve
space usage, but mostly it assists in eliminating the problem
where abbrev codes are shared between blocks.
```
@_specialize(exported: true, spi: SPIGroupName, where T == Int)
public func myFunc() { }
```
The specialized entry point is only visible for modules that import
using `_spi(SPIGroupName) import ModuleDefiningMyFunc `.
rdar://64993425
Take advantage of the binary swiftdeps serialization utliities built during #32131. Add a new optional information block to swiftdeps files. For now, don't actually serialize swiftdeps information.
Frontends will use this information to determine whether to write incremental dependencies across modules into their swiftdeps files. We will then teach the driver to deserialize the data from this section and integrate it into its incremental decision making.
Serialize derivative function configurations per module.
`@differentiable` and `@derivative` attributes register derivatives for
`AbstractFunctionDecl`s for a particular "derivative function configuration":
parameter indices and dervative generic signature.
To find `@derivative` functions registered in other Swift modules, derivative
function configurations must be serialized per module. When configurations for
a `AbstractFunctionDecl` are requested, all configurations from imported
modules are deserialized. This module serialization technique has precedent: it
is used for protocol conformances (e.g. extension declarations for a nominal
type) and Obj-C members for a class type.
Add `AbstractFunctionDecl::getDerivativeFunctionConfigurations` entry point
for accessing derivative function configurations.
In the differentiation transform: use
`AbstractFunctionDecl::getDerivativeFunctionConfigurations` to implement
`findMinimalDerivativeConfiguration` for canonical derivative function
configuration lookup, replacing `getMinimalASTDifferentiableAttr`.
Resolves TF-1100.
Remove the option to switch off nested types tables. In a world where
re-entrant direct lookup will cause deserialization to fail (or worse),
disabling these tables will only lead to further instability in the
compiler.
As part of this, we have to change the type export rules to
prevent `@convention(c)` function types from being used in
exported interfaces if they aren't serializable. This is a
more conservative version of the original rule I had, which
was to import such function-pointer types as opaque pointers.
That rule would've completely prevented importing function-pointer
types defined in bridging headers and so simply doesn't work,
so we're left trying to catch the unsupportable cases
retroactively. This has the unfortunate consequence that we
can't necessarily serialize the internal state of the compiler,
but that was already true due to normal type uses of aggregate
types from bridging headers; if we can teach the compiler to
reliably serialize such types, we should be able to use the
same mechanisms for function types.
This PR doesn't flip the switch to use Clang function types
by default, so many of the clang-function-type-serialization
FIXMEs are still in place.
Structurally prevent a number of common anti-patterns involving generic
signatures by separating the interface into GenericSignature and the
implementation into GenericSignatureBase. In particular, this allows
the comparison operators to be deleted which forces callers to
canonicalize the signature or ask to compare pointers explicitly.
Now that GenericSignatures store their single unique GenericEnvironment,
we can remove similar logic from deserialization to preserve identity
of GenericEnvironments.
A generic environment is always serialized as a GenericSignature with
a lazily-recreated environment, though sometimes it has to include
extra info specifically for generic environments used by SIL. The code
that was doing this claimed a bit for disambiguating between the two,
shrinking the permitted size of a compiled module from 2^31 bits to
2^30. (The code isn't just needlessly complicated; GenericEnvironments
used to be serialized with more information.)
Rather than have two representations for GenericEnvironmentID, this
commit just drops it altogether in favor of referencing
GenericSignatures directly. This causes a negligible file size
shrinkage for swiftmodules in addition to eliminating the problematic
disambiguation bit.
For now, the Deserialization logic will continue to cache
GenericEnvironments that are used directly by Deserialization, but
really that should probably be done at the AST level. Then we can
simplify further to ModuleFile tracking a plain list of
GenericSignatures.
...by making it a tagged union of either a DeclID or a
LocalDeclContextID. This should lead to smaller module files and be
slightly more efficient to deserialize, and also means that every
AST entity kind is serialized in exactly one way, which allows for
the following commit's refactoring.
The AST block in a compiled module represents an object graph, which
is essentially serialized in four steps:
- An entity (such as a Decl or Type) is encountered, given an ID, and
added to a worklist.
- The next entity is popped from the worklist and its offset in the
output stream is recorded.
- During the course of writing that entity, more entities will be
referenced and added to the worklist.
- Once the entire worklist is drained, the offsets get written to a
table in the Index block.
The implementation of this was duplicated for each kind of entity in
the AST block; this commit factors that out into a reusable helper.
No intended high-level functionality change, but the order in which
Decls and Types get emitted might change a little now that they're not
in the same queue.
