When a module extends a type from another module, serialize those symbols into
separated files dedicated to those extended modules. This makes it easier to
ingest and categorize those symbols under the extended module if desired.
rdar://58941718
* Remove dead ModuleSourceInfoFilename parameters
These were never actually used; we might find a way to bring them back later.
* Introduce SerializedModuleBaseName
This is intended to replace the _n_ filename parameters that tend to get passed around in the SerializedModuleLoader classes.
* Manipulate currPath in SerializedModuleLoader less often
* Don’t pass raw paths around SerializedModuleLoader
Only pass base names.
* Regularize module file opening functions
Add an alternative to getTopLevelDecls and getDeclChecked to limit which
decls are deserialized by first looking at their attributes. If the
attributes are accepted by a function passed as argument the decl is
fully deserialized, otherwise it is ignored.
The filter is included in the signature of existing functions in the
Serilalization services, but I’ve added new methods for it in FileUnit
and its subclasses to leave existing implementations untouched.
✔ More informative error messages in case of crashes.
✔ Handling and documenting different cases.
✔ Test cases for different cases.
✔ Make SDKDependencies.swift pass again.
We generate .swiftsourceinfo for stdlib in the build directory because ABI checker
could issue diagnostics to the stdlib source. However, this may also change other
diagnostic tests. Both Brent and Jordan have raised concern about this. After
adding this flag, other diagnostic tests could ignore .swiftsourceinfo files even
though when they are present so our tests will reflect what most users experience
when sources for stdlib are unavailable.
After setting up the .swiftsourceinfo file, this patch starts to actually serialize
and de-serialize source locations for declaration. The binary format of .swiftsourceinfo
currently contains these three records:
BasicDeclLocs: a hash table mapping from a USR ID to a list of basic source locations. The USR id
could be retrieved from the following DeclUSRs record using an actual decl USR. The basic source locations
include a file ID and the results from Decl::getLoc(), ValueDecl::getNameLoc(), Decl::getStartLoc() and Decl::getEndLoc().
The file ID could be used to retrieve the actual file name from the following SourceFilePaths record.
Each location is encoded as a line:column pair.
DeclUSRS: a hash table mapping from USR to a USR ID used by location records.
SourceFilePaths: a hash table mapping from a file ID to actual file name.
BasicDeclLocs should be sufficient for most diagnostic cases. If additional source locations
are needed, we could always add new source location records without breaking the backward compatibility.
When de-serializing the source location from a module-imported decl, we calculate its USR, retrieve the USR ID
from the DeclUSRS record, and use the USR ID to look up the basic location list in the BasicDeclLocs record.
For more details about .swiftsourceinfo file: https://forums.swift.org/t/proposal-emitting-source-information-file-during-compilation
Structurally prevent a number of common anti-patterns involving generic
signatures by separating the interface into GenericSignature and the
implementation into GenericSignatureBase. In particular, this allows
the comparison operators to be deleted which forces callers to
canonicalize the signature or ask to compare pointers explicitly.
Most of AST, Parse, and Sema deal with FileUnits regularly, but SIL
and IRGen certainly don't. Split FileUnit out into its own header to
cut down on recompilation times when something changes.
No functionality change.
Removes duplicated logic from the implementations of
FileUnit::lookupValue, and simplifies the interface to
ModuleDecl::lookupValue, where everyone was passing an empty
(non-filtering) access path anyway /except/ during actual lookup from
source code. No functionality change.
ABI checker imports Swift frameworks by using Swift interfaces for various
reasons. The existing way of controlling preferred importing mechanism is by
setting an environment variable (SWIFT_FORCE_MODULE_LOADING), which may lead
to performance issues because the stdlib could also be loaded in this way.
This patch adds a new front-end option to specify module names for
which we prefer to importing via Swift interface. The option currently is only
accessible via swift-api-digester.
rdar://54559888
We already do this for other ASTContext-allocated types (see
Decl.cpp). This will prevent the sort of mistakes in the previous two
commits.
Note that if any particular subclass of FileUnit wants to have its
destructor run, it can opt into that manually using
ASTContext::addDestructorCleanup. SourceFile and BuiltinUnit both do
this. But we generally don't /want/ to do this if we can avoid it
because it adds to compiler teardown time.
Implementing it in LoadedFile is nice in theory, but causes a leak in
practice because that type is ASTContext-allocated and usually never
destroyed.
https://bugs.swift.org/browse/SR-11366
...rather than the buffer, for a compiled module that came from a
module interface.
This was already happening at a higher level
(ModuleDecl::getModuleFilename) so pushing it down to the low-level
ModuleFile::getModuleFilename doesn't really change things much. The
important fix that goes with this is that SerializedASTFile no longer
leaks this name by storing it outside of ModuleFile.
https://bugs.swift.org/browse/SR-11365
Previously 'isSystemModule()' returns true only if the module is:
- Standard library
- Clang module and that is `IsSystem`
- Swift overlay for clang `IsSystem` module
Now:
- Clang module and that is `IsSystem`; or
- Swift overlay for clang `IsSystem` module
- Swift module found in either of these directories:
- Runtime library directoris (including stdlib)
- Frameworks in `-Fsystem` directories
- Frameworks in `$SDKROOT/System/Library/Frameworks/` (Darwin)
- Frameworks in `$SDKROOT/Library/Frameworks/` (Darwin)
rdar://problem/50516314
form SerializedModuleLoader into its own ModuleLoader class. (NFC-ish)
This gives better control over the order in which the various module
load mechanisms are applied.
When printing a swiftinterface, represent opaque result types using an attribute that refers to
the mangled name of the defining decl for the opaque type. To turn this back into a reference
to the right decl's implicit OpaqueTypeDecl, use type reconstruction. Since type reconstruction
doesn't normally concern itself with non-type decls, set up a lookup table in SourceFiles and
ModuleFiles to let us handle the mapping from mangled name to opaque type decl in type
reconstruction.
(Since we're invoking type reconstruction during type checking, when the module hasn't yet been
fully validated, we need to plumb a LazyResolver into the ASTBuilder in an unsightly way. Maybe
there's a better way to do this... Longer term, at least, this surface design gives space for
doing things more the right way--a more request-ified decl validator ought to be able to naturally
lazily service this request without the LazyResolver reference, and if type reconstruction in
the future learns how to reconstruct non-type decls, then the lookup tables can go away.)
When a Swift module built with debug info imports a library without
debug info from a textual interface, the textual interface is
necessary to reconstruct types defined in the library's interface. By
recording the Swift interface files in DWARF dsymutil can collect them
and LLDB can find them.
rdar://problem/49751363
Previously, the ParseableInterfaceModuleLoader relied on the assumption
that, if it returned `errc::not_supported`, it would fall through the
search paths and then move on to the SerializedModuleLoader. This did
not anticipate the possibility of a valid .swiftinterface coming later
in the search paths, which can cause issues for the standard library
which is in the resource-dir and should always be loaded from there.
Instead, make the module loading explicitly short-circuit when seeing
`errc::not_supported`, and document it.
Also add some more logging throughout `discoverLoadableModule` so we can
more easily catch issues like this in the future.
Fixes rdar://49479386
This patch modifies ParseableInterfaceBuilder::CollectDepsForSerialization to
avoid serializing dependencies from the runtime resource path into the
swiftmodules generated from .swiftinterface files. This means the module cache
should now be relocatable across machines.
It also modifies ParseableInterfaceModuleLoader to never add any dependencies
from the module cache and prebuilt cache to the dependency tracker (in addition
to the existing behaviour of not serializing them in the generated
swiftmodules). As a result, CollectDepsForSerialization no longer checks if the
dependencies it is given come from the cache as they are provided by the
dependency tracker. It now asserts that's the case instead.
When we build incrementally, we produce "partial swiftmodules" for
each input source file, then merge them together into the final
compiled module that, among other things, gets used for debugging.
Without this, we'd drop @_implementationOnly imports and any types
from the modules that were imported during the module-merging step
and then be unable to debug those types
In addition to being wasteful, this is a correctness issue -- the
compiler should only ever have one view of this file, and it should not
read a potentially different file after validating dependencies.
rdar://48654608
The previous 'openModuleFiles' interface in SerializedModuleLoaderBase
still assumed that swiftmodule files and swiftdoc files would be found
next to each other, but that's not true anymore with
swiftinterfaces-built-to-modules. Give up on this assumption (and on
the minor optimization of passing down a scratch buffer) and split out
the interface into the customization point
'findModuleFilesInDirectory' and the implementation 'openModuleFiles'.
The latter now takes two full paths: one for the swiftmodule, one for
the swiftdoc.
- Use the name for the cached module, so that we don't end up with a
zillion "x86_64-XXXXXXXX.swiftmodule" files in the cache when we're
working with architecture-specific swiftmodules.
- Diagnose if the expected name is different from the name specified
in the swiftinterface.
- Emit all diagnostics at the location of the import, instead of
without any location at all.
Added the 'Module::getPrecedenceGroups' API to separate precedence group lookup
from 'Module::lookupVisibleDecls', which together with 'FileUnit::lookupVisibleDecls',
to which the former is forwarded, are expected to look up only 'ValueDecl'. In particular, this
prevents completions like Module.PrecedenceGroup.
This can only happen in one case today: a module imports a bridging
header, but the header on disk has disappeared, and now we need to
fall back to the (often inadequate) version that's stored inside the
swiftmodule file. Even if the module fails to load, the bridging
header has already been imported, and so anything else that happens
might still emit diagnostics and need that text to be alive, which
means we need to keep the buffer alive too.
