Query the SourceLookupCache for the operator decls,
and use ModuleDecl::getOperatorDecls for both
frontend stats and to clean up some code
completion logic.
In addition, this commit switches getPrecedenceGroups
over to querying SourceLookupCache.
Serialize derivative function configurations per module.
`@differentiable` and `@derivative` attributes register derivatives for
`AbstractFunctionDecl`s for a particular "derivative function configuration":
parameter indices and dervative generic signature.
To find `@derivative` functions registered in other Swift modules, derivative
function configurations must be serialized per module. When configurations for
a `AbstractFunctionDecl` are requested, all configurations from imported
modules are deserialized. This module serialization technique has precedent: it
is used for protocol conformances (e.g. extension declarations for a nominal
type) and Obj-C members for a class type.
Add `AbstractFunctionDecl::getDerivativeFunctionConfigurations` entry point
for accessing derivative function configurations.
In the differentiation transform: use
`AbstractFunctionDecl::getDerivativeFunctionConfigurations` to implement
`findMinimalDerivativeConfiguration` for canonical derivative function
configuration lookup, replacing `getMinimalASTDifferentiableAttr`.
Resolves TF-1100.
- Add DocRangesLayout to the `.swiftsourceinfo`.
This is a blob containing an array of `SingleRawComment`
source locations.
- Add DocLocWriter for serializing `SingleRawComment` locs into the
`DocLocsLayout` buffer.
Serialize start line, start column, and length of `SingleRawComment`
pieces in `.swiftsourceinfo`
- Read doc locs when loading basic declaration locs from a ModuleFile.
- Load `DOC_LOCS` blob into ModuleFile::DocLocsData
- Reconstitute RawComment ranges when available from .swiftsourceinfo
- Allow requesting serialized raw comment if available
rdar://problem/58339492
This cleanup exposed a problem with deserialization recovery and
property wrappers. If deserializing a property backed by a wrapper
failed, the lazy member lookup would fail, but subsequently a
loadAllMembers() call would still load the property.
This behavior is actually incorrect, because silently dropping a
stored property of a @frozen struct can result in miscompiles.
I've filed rdar://59403542 and rdar://59403617 to track fixing this.
In the meantime, I've tweaked the logic a bit to preserve the old
behavior.
As part of this, we have to change the type export rules to
prevent `@convention(c)` function types from being used in
exported interfaces if they aren't serializable. This is a
more conservative version of the original rule I had, which
was to import such function-pointer types as opaque pointers.
That rule would've completely prevented importing function-pointer
types defined in bridging headers and so simply doesn't work,
so we're left trying to catch the unsupportable cases
retroactively. This has the unfortunate consequence that we
can't necessarily serialize the internal state of the compiler,
but that was already true due to normal type uses of aggregate
types from bridging headers; if we can teach the compiler to
reliably serialize such types, we should be able to use the
same mechanisms for function types.
This PR doesn't flip the switch to use Clang function types
by default, so many of the clang-function-type-serialization
FIXMEs are still in place.
This fixes a bug that made it impossible to read any subsequent Swift modules
out of a .swift_ast section a previous section had a size divisible by 4.
rdar://problem/57110020
Add an alternative to getTopLevelDecls and getDeclChecked to limit which
decls are deserialized by first looking at their attributes. If the
attributes are accepted by a function passed as argument the decl is
fully deserialized, otherwise it is ignored.
The filter is included in the signature of existing functions in the
Serilalization services, but I’ve added new methods for it in FileUnit
and its subclasses to leave existing implementations untouched.
* Fix Swift following bitstream reader API update
Upstream change in rL364464 broke downstream Swift.
(cherry picked from commit 50de105bf1)
Conflicts:
lib/Serialization/Deserialization.cpp
lib/Serialization/ModuleFile.cpp
tools/driver/modulewrap_main.cpp
After this change, we only use one single hash table for USR to USR id
mapping. The basic source locations are an array of fixed length
records that could be retrieved by using the USR id since each
USR id is guaranteed to be associated with one basic location entry.
The source file paths are refactored to a blob of 0-terminated strings.
Decl locations use offset in this blob to refer to the source file path
where the decl was defined.
After setting up the .swiftsourceinfo file, this patch starts to actually serialize
and de-serialize source locations for declaration. The binary format of .swiftsourceinfo
currently contains these three records:
BasicDeclLocs: a hash table mapping from a USR ID to a list of basic source locations. The USR id
could be retrieved from the following DeclUSRs record using an actual decl USR. The basic source locations
include a file ID and the results from Decl::getLoc(), ValueDecl::getNameLoc(), Decl::getStartLoc() and Decl::getEndLoc().
The file ID could be used to retrieve the actual file name from the following SourceFilePaths record.
Each location is encoded as a line:column pair.
DeclUSRS: a hash table mapping from USR to a USR ID used by location records.
SourceFilePaths: a hash table mapping from a file ID to actual file name.
BasicDeclLocs should be sufficient for most diagnostic cases. If additional source locations
are needed, we could always add new source location records without breaking the backward compatibility.
When de-serializing the source location from a module-imported decl, we calculate its USR, retrieve the USR ID
from the DeclUSRS record, and use the USR ID to look up the basic location list in the BasicDeclLocs record.
For more details about .swiftsourceinfo file: https://forums.swift.org/t/proposal-emitting-source-information-file-during-compilation
Structurally prevent a number of common anti-patterns involving generic
signatures by separating the interface into GenericSignature and the
implementation into GenericSignatureBase. In particular, this allows
the comparison operators to be deleted which forces callers to
canonicalize the signature or ask to compare pointers explicitly.
Now that GenericSignatures store their single unique GenericEnvironment,
we can remove similar logic from deserialization to preserve identity
of GenericEnvironments.
...fulfilling the promised audit from 0747d9a339. No intended
functionality change /other/ than the order of already-unsorted lists.
This affected a number of SIL tests that relied on deserialization
order matching the original source order; I have no idea why the old
hash logic would make that the case. If we think that's a valuable
property, we should serialize a list of functions in addition to the
iterable table. (Maybe just in SIB mode?)
Unlike compiled modules, swiftdoc files are considered a stable
format, so we can't change how information is stored in them. If we
add any more string-hashed tables to swiftdoc files, we should
consider using a new hash seed for those.
A generic environment is always serialized as a GenericSignature with
a lazily-recreated environment, though sometimes it has to include
extra info specifically for generic environments used by SIL. The code
that was doing this claimed a bit for disambiguating between the two,
shrinking the permitted size of a compiled module from 2^31 bits to
2^30. (The code isn't just needlessly complicated; GenericEnvironments
used to be serialized with more information.)
Rather than have two representations for GenericEnvironmentID, this
commit just drops it altogether in favor of referencing
GenericSignatures directly. This causes a negligible file size
shrinkage for swiftmodules in addition to eliminating the problematic
disambiguation bit.
For now, the Deserialization logic will continue to cache
GenericEnvironments that are used directly by Deserialization, but
really that should probably be done at the AST level. Then we can
simplify further to ModuleFile tracking a plain list of
GenericSignatures.
...by making it a tagged union of either a DeclID or a
LocalDeclContextID. This should lead to smaller module files and be
slightly more efficient to deserialize, and also means that every
AST entity kind is serialized in exactly one way, which allows for
the following commit's refactoring.
We already do this for other ASTContext-allocated types (see
Decl.cpp). This will prevent the sort of mistakes in the previous two
commits.
Note that if any particular subclass of FileUnit wants to have its
destructor run, it can opt into that manually using
ASTContext::addDestructorCleanup. SourceFile and BuiltinUnit both do
this. But we generally don't /want/ to do this if we can avoid it
because it adds to compiler teardown time.
...rather than the buffer, for a compiled module that came from a
module interface.
This was already happening at a higher level
(ModuleDecl::getModuleFilename) so pushing it down to the low-level
ModuleFile::getModuleFilename doesn't really change things much. The
important fix that goes with this is that SerializedASTFile no longer
leaks this name by storing it outside of ModuleFile.
https://bugs.swift.org/browse/SR-11365
The code tried extra hard to make sure this stayed in sync with the
returned ValidationInfo, but there's no real reason for that. Just
keep a single "HasError" bit for checking that the module's not being
obviously misused. (I snuck some groundwork for this into the previous
commit.)
`ModuleFile::error` was being used both for errors of initial parse
and configuration (non-fatal) and format errors during actual
deserialization (fatal, indicating a corrupted module). Split out the
latter to `ModuleFile::fatal()` (to go with the existing
`ModuleFile::fatal(llvm::Error)`) and be more consistent about
explicitly setting statuses for the former.
Since 'fatal()' is always fatal, this also allows deleting dummy
recovery code that would never be used in practice.
Similar to 517f5d6b6a, the "shadowed" terminology didn't end up
describing the most common use of the feature; there is pretty much no
intended case where a Swift module shadows a Clang module without also
re-exporting it. Switch to "underlying", which was already in use in a
few places, and which better parallels "overlay".
No intended functionality change.
This commit adds a new type DynamicLookupInfo that provides information
about how a dynamic member lookup found a particular Decl. This is
needed to correctly handle KeyPath dynamic member lookups, but for now
just plumb it through everywhere.
Dependency tracking for cached compiled modules (compiled from
swiftinterfaces) can lead to a high percentage of the module being
SDK-relative paths when -track-system-dependencies is on. Cut down on
this by storing directory names in a separate record that gets
referenced from each file dependency. (Since a lot of per-file
dependencies are header files in a common directory, this is a win.)
We can do something more clever in the future, but this is a
reasonable start for, say, the overlays.
rdar://problem/50449802
