Module interface builder used to maintain a separate compiler instance for
building Swift modules. The configuration of this compiler instance is also
useful for dependencies scanner because it needs to emit front-end compiler invocation
for building Swift modules explicitly.
This patch refactor the configuration out to a delegate class, and the
delegate class is also used by the dependency scanner.
Additional flags in interface files may change parsing behavior like #if
statements. We should use a fresh ASTContext with these additional
flags when parsing interface files to collect imports.
rdar://62612027
Implement a new "fast" dependency scanning option,
`-scan-dependencies`, in the Swift frontend that determines all
of the source file and module dependencies for a given set of
Swift sources. It covers four forms of modules:
1) Swift (serialized) module files, by reading the module header
2) Swift interface files, by parsing the source code to find imports
3) Swift source modules, by parsing the source code to find imports
4) Clang modules, using Clang's fast dependency scanning tool
A single `-scan-dependencies` operation maps out the full
dependency graph for the given Swift source files, including all
of the Swift and Clang modules that may need to be built, such
that all of the work can be scheduled up front by the Swift
driver or any other build system that understands this
option. The dependency graph is emitted as JSON, which can be
consumed by these other tools.
We weren't handling this case, so their generated interfaces / doc info
wouldn't include symbols from the cross-import overlays, and we wouldn't
map the underscored cross-import overlay name back to the declaring
framework's name in cusor-info, completion results or when indexing.
Resolves rdar://problem/62138551
When a type (class, enum, or struct) is annotated @main, it is required
to provide a function with the following signature:
static func main() -> ()
That function will be called when the executable the type is defined
within is launched.
Query the SourceLookupCache for the operator decls,
and use ModuleDecl::getOperatorDecls for both
frontend stats and to clean up some code
completion logic.
In addition, this commit switches getPrecedenceGroups
over to querying SourceLookupCache.
Serialize derivative function configurations per module.
`@differentiable` and `@derivative` attributes register derivatives for
`AbstractFunctionDecl`s for a particular "derivative function configuration":
parameter indices and dervative generic signature.
To find `@derivative` functions registered in other Swift modules, derivative
function configurations must be serialized per module. When configurations for
a `AbstractFunctionDecl` are requested, all configurations from imported
modules are deserialized. This module serialization technique has precedent: it
is used for protocol conformances (e.g. extension declarations for a nominal
type) and Obj-C members for a class type.
Add `AbstractFunctionDecl::getDerivativeFunctionConfigurations` entry point
for accessing derivative function configurations.
In the differentiation transform: use
`AbstractFunctionDecl::getDerivativeFunctionConfigurations` to implement
`findMinimalDerivativeConfiguration` for canonical derivative function
configuration lookup, replacing `getMinimalASTDifferentiableAttr`.
Resolves TF-1100.
Introduce DirectOperatorLookupRequest &
DirectPrecedenceGroupLookupRequest that lookup
operator and precedence groups within a given
file or module without looking through imports.
These will eventually be used as the basis for the
new operator lookup implementation, but for now
just use them when querying lookup results from
serialized module files.
When a module extends a type from another module, serialize those symbols into
separated files dedicated to those extended modules. This makes it easier to
ingest and categorize those symbols under the extended module if desired.
rdar://58941718
* Remove dead ModuleSourceInfoFilename parameters
These were never actually used; we might find a way to bring them back later.
* Introduce SerializedModuleBaseName
This is intended to replace the _n_ filename parameters that tend to get passed around in the SerializedModuleLoader classes.
* Manipulate currPath in SerializedModuleLoader less often
* Don’t pass raw paths around SerializedModuleLoader
Only pass base names.
* Regularize module file opening functions
Add an alternative to getTopLevelDecls and getDeclChecked to limit which
decls are deserialized by first looking at their attributes. If the
attributes are accepted by a function passed as argument the decl is
fully deserialized, otherwise it is ignored.
The filter is included in the signature of existing functions in the
Serilalization services, but I’ve added new methods for it in FileUnit
and its subclasses to leave existing implementations untouched.
✔ More informative error messages in case of crashes.
✔ Handling and documenting different cases.
✔ Test cases for different cases.
✔ Make SDKDependencies.swift pass again.
We generate .swiftsourceinfo for stdlib in the build directory because ABI checker
could issue diagnostics to the stdlib source. However, this may also change other
diagnostic tests. Both Brent and Jordan have raised concern about this. After
adding this flag, other diagnostic tests could ignore .swiftsourceinfo files even
though when they are present so our tests will reflect what most users experience
when sources for stdlib are unavailable.
After setting up the .swiftsourceinfo file, this patch starts to actually serialize
and de-serialize source locations for declaration. The binary format of .swiftsourceinfo
currently contains these three records:
BasicDeclLocs: a hash table mapping from a USR ID to a list of basic source locations. The USR id
could be retrieved from the following DeclUSRs record using an actual decl USR. The basic source locations
include a file ID and the results from Decl::getLoc(), ValueDecl::getNameLoc(), Decl::getStartLoc() and Decl::getEndLoc().
The file ID could be used to retrieve the actual file name from the following SourceFilePaths record.
Each location is encoded as a line:column pair.
DeclUSRS: a hash table mapping from USR to a USR ID used by location records.
SourceFilePaths: a hash table mapping from a file ID to actual file name.
BasicDeclLocs should be sufficient for most diagnostic cases. If additional source locations
are needed, we could always add new source location records without breaking the backward compatibility.
When de-serializing the source location from a module-imported decl, we calculate its USR, retrieve the USR ID
from the DeclUSRS record, and use the USR ID to look up the basic location list in the BasicDeclLocs record.
For more details about .swiftsourceinfo file: https://forums.swift.org/t/proposal-emitting-source-information-file-during-compilation
Structurally prevent a number of common anti-patterns involving generic
signatures by separating the interface into GenericSignature and the
implementation into GenericSignatureBase. In particular, this allows
the comparison operators to be deleted which forces callers to
canonicalize the signature or ask to compare pointers explicitly.
Most of AST, Parse, and Sema deal with FileUnits regularly, but SIL
and IRGen certainly don't. Split FileUnit out into its own header to
cut down on recompilation times when something changes.
No functionality change.
Removes duplicated logic from the implementations of
FileUnit::lookupValue, and simplifies the interface to
ModuleDecl::lookupValue, where everyone was passing an empty
(non-filtering) access path anyway /except/ during actual lookup from
source code. No functionality change.
ABI checker imports Swift frameworks by using Swift interfaces for various
reasons. The existing way of controlling preferred importing mechanism is by
setting an environment variable (SWIFT_FORCE_MODULE_LOADING), which may lead
to performance issues because the stdlib could also be loaded in this way.
This patch adds a new front-end option to specify module names for
which we prefer to importing via Swift interface. The option currently is only
accessible via swift-api-digester.
rdar://54559888
We already do this for other ASTContext-allocated types (see
Decl.cpp). This will prevent the sort of mistakes in the previous two
commits.
Note that if any particular subclass of FileUnit wants to have its
destructor run, it can opt into that manually using
ASTContext::addDestructorCleanup. SourceFile and BuiltinUnit both do
this. But we generally don't /want/ to do this if we can avoid it
because it adds to compiler teardown time.
Implementing it in LoadedFile is nice in theory, but causes a leak in
practice because that type is ASTContext-allocated and usually never
destroyed.
https://bugs.swift.org/browse/SR-11366
...rather than the buffer, for a compiled module that came from a
module interface.
This was already happening at a higher level
(ModuleDecl::getModuleFilename) so pushing it down to the low-level
ModuleFile::getModuleFilename doesn't really change things much. The
important fix that goes with this is that SerializedASTFile no longer
leaks this name by storing it outside of ModuleFile.
https://bugs.swift.org/browse/SR-11365
Previously 'isSystemModule()' returns true only if the module is:
- Standard library
- Clang module and that is `IsSystem`
- Swift overlay for clang `IsSystem` module
Now:
- Clang module and that is `IsSystem`; or
- Swift overlay for clang `IsSystem` module
- Swift module found in either of these directories:
- Runtime library directoris (including stdlib)
- Frameworks in `-Fsystem` directories
- Frameworks in `$SDKROOT/System/Library/Frameworks/` (Darwin)
- Frameworks in `$SDKROOT/Library/Frameworks/` (Darwin)
rdar://problem/50516314
form SerializedModuleLoader into its own ModuleLoader class. (NFC-ish)
This gives better control over the order in which the various module
load mechanisms are applied.
When printing a swiftinterface, represent opaque result types using an attribute that refers to
the mangled name of the defining decl for the opaque type. To turn this back into a reference
to the right decl's implicit OpaqueTypeDecl, use type reconstruction. Since type reconstruction
doesn't normally concern itself with non-type decls, set up a lookup table in SourceFiles and
ModuleFiles to let us handle the mapping from mangled name to opaque type decl in type
reconstruction.
(Since we're invoking type reconstruction during type checking, when the module hasn't yet been
fully validated, we need to plumb a LazyResolver into the ASTBuilder in an unsightly way. Maybe
there's a better way to do this... Longer term, at least, this surface design gives space for
doing things more the right way--a more request-ified decl validator ought to be able to naturally
lazily service this request without the LazyResolver reference, and if type reconstruction in
the future learns how to reconstruct non-type decls, then the lookup tables can go away.)
When a Swift module built with debug info imports a library without
debug info from a textual interface, the textual interface is
necessary to reconstruct types defined in the library's interface. By
recording the Swift interface files in DWARF dsymutil can collect them
and LLDB can find them.
rdar://problem/49751363
Previously, the ParseableInterfaceModuleLoader relied on the assumption
that, if it returned `errc::not_supported`, it would fall through the
search paths and then move on to the SerializedModuleLoader. This did
not anticipate the possibility of a valid .swiftinterface coming later
in the search paths, which can cause issues for the standard library
which is in the resource-dir and should always be loaded from there.
Instead, make the module loading explicitly short-circuit when seeing
`errc::not_supported`, and document it.
Also add some more logging throughout `discoverLoadableModule` so we can
more easily catch issues like this in the future.
Fixes rdar://49479386
This patch modifies ParseableInterfaceBuilder::CollectDepsForSerialization to
avoid serializing dependencies from the runtime resource path into the
swiftmodules generated from .swiftinterface files. This means the module cache
should now be relocatable across machines.
It also modifies ParseableInterfaceModuleLoader to never add any dependencies
from the module cache and prebuilt cache to the dependency tracker (in addition
to the existing behaviour of not serializing them in the generated
swiftmodules). As a result, CollectDepsForSerialization no longer checks if the
dependencies it is given come from the cache as they are provided by the
dependency tracker. It now asserts that's the case instead.
When we build incrementally, we produce "partial swiftmodules" for
each input source file, then merge them together into the final
compiled module that, among other things, gets used for debugging.
Without this, we'd drop @_implementationOnly imports and any types
from the modules that were imported during the module-merging step
and then be unable to debug those types
