Using a SetVector fixes an issue where many source files imported the
same SPI group from the same module, the emitted private textual
interfaces superfluously repeated the `@_spi` attribute on the import.
rdar://problem/63681845
Start fixing SR-12526: `@derivative` attribute cross-module deserialization
crash. Remove original `AbstractFunctionDecl *` from `DerivativeAttr` and store
`DeclID` instead, mimicking `DynamicReplacementAttr`.
Type erasure requires a circular construction by its very nature:
@_typeEraser(AnyProto)
protocol Proto { /**/ }
public struct AnyProto : Proto {}
If we eagerly resolve AnyProto, the chain of resolution steps that
deserialization must make goes a little something like this:
Lookup(Proto)
-> Deserialize(@_typeEraser(AnyProto))
-> Lookup(AnyProto)
-> DeserializeInheritedStuff(AnyProto)
-> Lookup(Proto)
This cycle could be broken if the order of incremental inputs was
such that we had already cached the lookup of Proto.
Resolve this cycle in any case by suspending the deserialization of the
type eraser until the point it's demanded by adding
ResolveTypeEraserTypeRequest.
rdar://61270195
Query the SourceLookupCache for the operator decls,
and use ModuleDecl::getOperatorDecls for both
frontend stats and to clean up some code
completion logic.
In addition, this commit switches getPrecedenceGroups
over to querying SourceLookupCache.
Serialize derivative function configurations per module.
`@differentiable` and `@derivative` attributes register derivatives for
`AbstractFunctionDecl`s for a particular "derivative function configuration":
parameter indices and dervative generic signature.
To find `@derivative` functions registered in other Swift modules, derivative
function configurations must be serialized per module. When configurations for
a `AbstractFunctionDecl` are requested, all configurations from imported
modules are deserialized. This module serialization technique has precedent: it
is used for protocol conformances (e.g. extension declarations for a nominal
type) and Obj-C members for a class type.
Add `AbstractFunctionDecl::getDerivativeFunctionConfigurations` entry point
for accessing derivative function configurations.
In the differentiation transform: use
`AbstractFunctionDecl::getDerivativeFunctionConfigurations` to implement
`findMinimalDerivativeConfiguration` for canonical derivative function
configuration lookup, replacing `getMinimalASTDifferentiableAttr`.
Resolves TF-1100.
Components of a requirement may be hidden behind an implementation-only
import. Attempts at deserializing them would fail on a 'module not
loaded' error. We only see failures in non-compilation paths, either in
indexing or with tools like ide-test as they try to deserialize
things that are private.
- Add DocRangesLayout to the `.swiftsourceinfo`.
This is a blob containing an array of `SingleRawComment`
source locations.
- Add DocLocWriter for serializing `SingleRawComment` locs into the
`DocLocsLayout` buffer.
Serialize start line, start column, and length of `SingleRawComment`
pieces in `.swiftsourceinfo`
- Read doc locs when loading basic declaration locs from a ModuleFile.
- Load `DOC_LOCS` blob into ModuleFile::DocLocsData
- Reconstitute RawComment ranges when available from .swiftsourceinfo
- Allow requesting serialized raw comment if available
rdar://problem/58339492
When a module extends a type from another module, serialize those symbols into
separated files dedicated to those extended modules. This makes it easier to
ingest and categorize those symbols under the extended module if desired.
rdar://58941718
As part of this, we have to change the type export rules to
prevent `@convention(c)` function types from being used in
exported interfaces if they aren't serializable. This is a
more conservative version of the original rule I had, which
was to import such function-pointer types as opaque pointers.
That rule would've completely prevented importing function-pointer
types defined in bridging headers and so simply doesn't work,
so we're left trying to catch the unsupportable cases
retroactively. This has the unfortunate consequence that we
can't necessarily serialize the internal state of the compiler,
but that was already true due to normal type uses of aggregate
types from bridging headers; if we can teach the compiler to
reliably serialize such types, we should be able to use the
same mechanisms for function types.
This PR doesn't flip the switch to use Clang function types
by default, so many of the clang-function-type-serialization
FIXMEs are still in place.
To support lazy resolution of the cross-referenced function in a serialized @_dynamicReplacement(for: ...) attribute, add a utility to the LazyMemberLoader and plumb it through. This is a more general utility than the current resolver, which relies on the type checker to strip the attribute off of VarDecls and fan it back out onto accessors, which means serialization has only ever seen AbstractFunctionDecls.
Add an alternative to getTopLevelDecls and getDeclChecked to limit which
decls are deserialized by first looking at their attributes. If the
attributes are accepted by a function passed as argument the decl is
fully deserialized, otherwise it is ignored.
The filter is included in the signature of existing functions in the
Serilalization services, but I’ve added new methods for it in FileUnit
and its subclasses to leave existing implementations untouched.
The Bitstream part of Bitcode moved to llvm/Bitstream in LLVM. This
updates the uses in swift.
See r365091 [Bitcode] Move Bitstream to a separate library.
(cherry picked from commit 1cd8e19357)
* Fix Swift following bitstream reader API update
Upstream change in rL364464 broke downstream Swift.
(cherry picked from commit 50de105bf1)
Conflicts:
lib/Serialization/Deserialization.cpp
lib/Serialization/ModuleFile.cpp
tools/driver/modulewrap_main.cpp
✔ More informative error messages in case of crashes.
✔ Handling and documenting different cases.
✔ Test cases for different cases.
✔ Make SDKDependencies.swift pass again.
After this change, we only use one single hash table for USR to USR id
mapping. The basic source locations are an array of fixed length
records that could be retrieved by using the USR id since each
USR id is guaranteed to be associated with one basic location entry.
The source file paths are refactored to a blob of 0-terminated strings.
Decl locations use offset in this blob to refer to the source file path
where the decl was defined.
After setting up the .swiftsourceinfo file, this patch starts to actually serialize
and de-serialize source locations for declaration. The binary format of .swiftsourceinfo
currently contains these three records:
BasicDeclLocs: a hash table mapping from a USR ID to a list of basic source locations. The USR id
could be retrieved from the following DeclUSRs record using an actual decl USR. The basic source locations
include a file ID and the results from Decl::getLoc(), ValueDecl::getNameLoc(), Decl::getStartLoc() and Decl::getEndLoc().
The file ID could be used to retrieve the actual file name from the following SourceFilePaths record.
Each location is encoded as a line:column pair.
DeclUSRS: a hash table mapping from USR to a USR ID used by location records.
SourceFilePaths: a hash table mapping from a file ID to actual file name.
BasicDeclLocs should be sufficient for most diagnostic cases. If additional source locations
are needed, we could always add new source location records without breaking the backward compatibility.
When de-serializing the source location from a module-imported decl, we calculate its USR, retrieve the USR ID
from the DeclUSRS record, and use the USR ID to look up the basic location list in the BasicDeclLocs record.
For more details about .swiftsourceinfo file: https://forums.swift.org/t/proposal-emitting-source-information-file-during-compilation
Structurally prevent a number of common anti-patterns involving generic
signatures by separating the interface into GenericSignature and the
implementation into GenericSignatureBase. In particular, this allows
the comparison operators to be deleted which forces callers to
canonicalize the signature or ask to compare pointers explicitly.
Most of AST, Parse, and Sema deal with FileUnits regularly, but SIL
and IRGen certainly don't. Split FileUnit out into its own header to
cut down on recompilation times when something changes.
No functionality change.
Harden more of the serialization functions to propagate errors for
the caller to handle these errors gracefully. This fixes a crash in
finishNormalConformance when indexing a system module with an
implementation-only import.
rdar://problem/52837313
This eliminates the entire 'lazy generic environment' concept;
essentially, all generic environments are now lazy, and since
each signature has exactly one environment, their construction
no longer needs to be co-ordinated with deserialization.
Now that GenericSignatures store their single unique GenericEnvironment,
we can remove similar logic from deserialization to preserve identity
of GenericEnvironments.
- Parameterize maybeReadGenericParams' BitstreamCursor so that we can read from
the correct cursor when trying to read the generic params of a SILFunction.
- Only serialize the context generic params for SILFunctions for which we're
serializing a complete definition. This fixes issues with us getting the
wrong archetypes forward-declared from references in other modules.
In this version of the patch, we adjust the deserialization point for the
generic param list to correctly come before we check if the SILFunction block
is empty, and we add a kludge to keep the JIT from crapping itself when it sees
the same transparent definition in multiple REPL lines' modules
<rdar://problem/16094902>.
The previous commit solves a problem this exposed at r14050 in inout deshadowing
that caused memory corruption when transparent functions were imported. This
should now be safe to commit.
Swift SVN r14109
- Parameterize maybeReadGenericParams' BitstreamCursor so that we can read from the correct cursor when trying to read the generic params of a SILFunction.
- Only serialize the context generic params for SILFunctions for which we're serializing a complete definition. This fixes issues with us getting the wrong archetypes forward-declared from references in other modules.
In this version of the patch, we adjust the deserialization point for the generic param list to correctly come before we check if the SILFunction block is empty, and we add a kludge to keep the JIT from crapping itself when it sees the same transparent definition in multiple REPL lines' modules <rdar://problem/16094902>.
Swift SVN r14030
- Parameterize maybeReadGenericParams' BitstreamCursor so that we can read from the correct cursor when trying to read the generic params of a SILFunction.
- Only serialize the context generic params for SILFunctions for which we're serializing a complete definition. This fixes issues with us getting the wrong archetypes forward-declared from references in other modules.
This gets me a clean build when applied against r13984.
Swift SVN r14005
Edge SILFunction one step closer to independence from SILFunctionType context by taking the generic param list as a separate constructor parameter, and serializing those params alongside the function record. For now we still pass in the context params from the SILFunctionType in most cases, because the logic for finding the generic params tends to be entangled in type lowering, but this pushes the problem up a step.
Thanks Jordan for helping work out the serialization changes needed.
Compared to r13036, this version of the patch includes the decls_block RecordKind enumerators for the GENERIC_PARAM_LIST layouts in the sil_block RecordKind enumerator, as Jordan had suggested before. r13036 caused buildbot failures when building for iOS, but I am unable to reproduce those failures locally now.
Swift SVN r13485
Edge SILFunction one step closer to independence from SILFunctionType context by taking the generic param list as a separate constructor parameter, and serializing those params alongside the function record. For now we still pass in the context params from the SILFunctionType in most cases, because the logic for finding the generic params tends to be entangled in type lowering, but this pushes the problem up a step.
Thanks Jordan for helping work out the serialization changes needed.
Swift SVN r13036
In general, this forces SILGen and IRGen code that's grabbing
a declaration to state whether it's doing so to define it.
Change SIL serialization to serialize the linkage of functions
and global variables, which means also serializing declarations.
Change the deserializer to use this stored linkage, even when
only deserializing a declaration, and to call a callback to
inform the client that it has deserialized a new entity.
Take advantage of that callback in the linking pass to alter
the deserialized linkage as appropriate for the fact that we
imported the declaration. This computation should really take
advantage of the relationship between modules, but currently
it does not.
Swift SVN r12090
Previously, cross-references just carried a chain of identifiers and a
top-level module, plus a type to validate against, a generic parameter index,
or an operator fixity. However, referencing "the first generic parameter
of the prefix function ++ that takes a ForwardIndex" requires /all three/
of these filters to unambiguously select the right declaration.
Now, cross-references consist of a chain of trailing records, one for each
link in the path. There are (currently) five kinds of links:
Type: a declaration that cannot have overloads
Value: a declaration that can have overloads (filtered by type)
Extension: filter to decls within extensions on another module
Operator:
- as the first path piece, an operator declaration
- as a later path piece, a fixity filter for operator functions
Generic Param: an indexed generic parameter of the previous result
This should allow us to uniquely cross-reference any Swift declaration we
need to.
Swift SVN r11399
NominalTypeDecls and ExtensionDecls have a list of members. Rather than
eagerly populating that list when the nominal or extension is just referenced,
just include a pointer back to the ModuleFile, so that they can be
deserialized when we actually look into the decl. The design here is
general enough that we could do something similar with imported Clang decls.
Clang is even more lazy here: the on-disk representation is a hash table,
so lookup only forces deserialization of members with the same name. We
probably want that some day, but this might be enough to get by for now.
This is groundwork for loading partial ASTs, where eagerly deserializing
members leads to circular references we're not able to handle.
Swift SVN r11219
Since we don't have soft-failure yet from deserialization, it's helpful to
at least know where to start looking when something crashes. There are some
rough edges here but it should be much better than nothing.
This also pulls the list of record nodes out into a separate file, so that
we can avoid repeating it.
Example crash:
1. While reading from ./CTypes.swiftmodule
2. While deserializing 'CBool' (StructDecl)
3. While deserializing decl #26 (XREF)
4. Cross-reference to 'LogicValue' in swift
(don't worry, this is an example where I'm tweaking things)
<rdar://problem/14838332>
Swift SVN r11057
