If we have both loaded a swiftdoc, and the decl we
have should have had its doc comment serialized into
it, we can check it without needing to fall back
to the swiftsourceinfo.
This requires a couple of refactorings:
- Factoring out the `shouldIncludeDecl` logic
into `getDocCommentSerializationTargetFor` for
determining whether a doc comment should end up
in the swiftdoc or not.
- Factoring out `CommentProviderFinder` for searching
for the doc providing comment decl for brief
comments, in order to allow us to avoid querying
the raw comment when searching for it. This has the
added bonus of meaning we no longer need to fall
back to parsing the raw comment for the brief
comment if the comment is provided by another decl
in the swiftdoc.
This diff is best viewed without whitespace.
For a `@Testable` import in program source, if a Swift interface dependency is discovered, and has an adjacent binary `.swiftmodule`, open up the module, and pull in its optional dependencies. If an optional dependency cannot be resolved on the filesystem, fail silently without raising a diagnostic.
The macro name resolution in the source lookup cache was only looking at
macros in the current module, meaning that any names introduced by peer
or declaration macros declared in one module but used in another would
not be found by name lookup.
Switch the source lookup cache over to using the same
`forEachPotentialResolvedMacro` API that is used by lookup within
types, so we have consistent name-lookup-level macro resolution in both
places.
... except that would be horribly cyclic, of course, so introduce name
lookup flags to ignore top-level declarations introduced by macro
expansions. This is semantically correct because macro expansions are
not allowed to introduce new macros anyway, because that would have
been a terrible idea.
Fixes rdar://107321469. Peer and declaration macros at module scope
should work a whole lot better now.
Add a private discriminator to the mangling of an outermost-private `MacroExpansionDecl` so that declaration macros in different files won't have colliding macro expansion buffer names.
rdar://107462515
A @testable import allows a client to call internal decls which may
refer to non-public dependencies. To support such a use case, load
non-public transitive dependencies of a module when it's imported
@testable from the main module.
This replaces the previous behavior where we loaded those dependencies
for any modules built for testing. This was risky as we would load more
module for any debug build, opening the door to a different behavior
between debug and release builds. In contrast, applying this logic to
@testable clients will only change the behavior of test targets.
rdar://107329303
The new diagnoseSerializedASTLoadFailureTransitive diagnose problems for
transitive dependencies only: missing dependency, missing underlying
module, or circular dependency.
Since https://github.com/apple/swift/pull/63178 added support for Clang modules in the explicit module map, it is possible for there to be multiple modules with the same name: a Swift module and a Clang module. The current parsing logic just overwrites the corresponding entry module in a hashmap so we always only preserved the module that comes last, with the same name.
This change separates the parsing of the modulemap JSON file to produce a separate Swift module map and Clang module map. The Swift one is used by the 'ExplicitSwiftModuleLoader', as before, and the Clang one is only used to populate the ClangArgs with the requried -fmodule-... flags.
Weaken the precise tag check at loading swiftmodule to accept binary
modules build by a compiler with a tag where only the last digit is
different. We assume that the other digit in the version should ensure
compiler and stdlib compatibility. If the last digit doesn't match,
still raise a remark.
rdar://105158258
If a module was first read using the adjacent swiftmodule and then
reloaded using the swiftinterface, we would do an up to date check on
the adjacent module but write out the unit using the swiftinterface.
This would cause the same modules to be indexed repeatedly for the first
invocation using a new SDK. On the next run we would instead raad the
swiftmodule from the cache and thus the out of date check would match
up.
The impact of this varies depending on the size of the module graph in
the initial compilation and the number of jobs started at the same time.
Each SDK dependency is re-indexed *and* reloaded, which is a drain on
both CPU and memory. Thus, if many jobs are initially started and
they're all going down this path, it can cause the system to run out of
memory very quickly.
Resolves rdar://103119964.
Introduce a new flag `-export-as` to specify a name used to identify the
target module in swiftinterfaces. This provides an analoguous feature
for Swift module as Clang's `export_as` feature.
In practice it should be used when a lower level module `MyKitCore` is
desired to be shown publicly as a downstream module `MyKit`. This should
be used in conjunction with `@_exported import MyKitCore` from `MyKit`
that allows clients to refer to all services as being part of `MyKit`,
while the new `-export-as MyKit` from `MyKitCore` will ensure that the
clients swiftinterfaces also use the `MyKit` name for all services.
In the current implementation, the export-as name is used in the
module's clients and not in the declarer's swiftinterface (e.g.
`MyKitCore`'s swiftinterface still uses the `MyKitCore` module name).
This way the module swiftinterface can be verified. In the future, we
may want a similar behavior for other modules in between `MyKitCore` and
`MyKit` as verifying a swiftinterface referencing `MyKit` without it
being imported would fail.
rdar://103888618
This changes the scanner's behavior to "resolve" a discovered module's dependencies to a set of Module IDs: module name + module kind (swift textual, swift binary, clang, etc.).
The 'ModuleDependencyInfo' objects that are stored in the dependency scanner's cache now carry a set of kind-qualified ModuleIDs for their dependencies, in addition to unqualified imported module names of their dependencies.
Previously, the scanner's internal state would cache a module dependnecy as having its own set of dependencies which were stored as names of imported modules. This led to a design where any time we needed to process the dependency downstream from its discovery (e.g. cycle detection, graph construction), we had to query the ASTContext to resolve this dependency's imports, which shouldn't be necessary. Now, upon discovery, we "resolve" a discovered dependency by executing a lookup for each of its imported module names (this operation happens regardless of this patch) and store a fully-resolved set of dependencies in the dependency module info.
Moreover, looking up a given module dependency by name (via `ASTContext`'s `getModuleDependencies`) would result in iterating over the scanner's module "loaders" and querying each for the module name. The corresponding modules would then check the scanner's cache for a respective discovered module, and if no such module is found the "loader" would search the filesystem.
This meant that in practice, we searched the filesystem on many occasions where we actually had cached the required dependency, as follows:
Suppose we had previously discovered a Clang module "foo" and cached its dependency info.
-> ASTContext.getModuleDependencies("foo")
--> (1) Swift Module "Loader" checks caches for a Swift module "foo" and doesn't find one, so it searches the filesystem for "foo" and fails to find one.
--> (2) Clang Module "Loader" checks caches for a Clang module "foo", finds one and returns it to the client.
This means that we were always searching the filesystem in (1) even if we knew that to be futile.
With this change, queries to `ASTContext`'s `getModuleDependencies` will always check all the caches first, and only delegate to the scanner "loaders" if no cached dependency is found. The loaders are then no longer in the business of checking the cached contents.
To handle cases in the scanner where we must only lookup either a Swift-only module or a Clang-only module, this patch splits 'getModuleDependencies' into an alrady-existing 'getSwiftModuleDependencies' and a newly-added 'getClangModuleDependencies'.
Push the top level logic for writing out swiftmodules and associated files into the frontend library which has access to all the necessary dependencies.
For release-management purposes during development, LLDB's embedded Swift
compiler's version number can sometimes be off-by-one in the last digit
compared to the Swift compiler.
This patch restores the old behavior from before 17183629e4.
rdar://101299168
Specifically, we get an additional table like thing called sil_moveonlydeinit. It looks as follows:
sil_moveonlydeinit TYPE {
@FUNC_NAME
}
It always has a single entry.
Previously, when evaluating a `#if canImport(Module, _version: 42)` directive the compiler could diagnose and ignore the directive under the following conditions:
- The associated binary module is corrupt/bogus.
- The .tbd for an underlying Clang module is missing a current-version field.
This behavior is surprising when there is a valid `.swiftinterface` available and it only becomes apparent when building against an SDK with an old enough version of the module that the version in the `.swiftinterface` is too low, making this failure easy to miss. Some modules have different versioning systems for their Swift and Clang modules and it can also be intentional for a distributed binary `.swiftmodule` to contain bogus data (to force the compiler to recompile the `.swiftinterface`) so we need to handle both of these cases gracefully and predictably.
Now the compiler will enumerate all module loaders, ask each of them to attempt to parse the module version and then consistently use the parsed version from a single source. The `.swiftinterface` is preferred if present, then the binary module if present, and then finally the `.tbd`. The `.tbd` is still always used exclusively for the `_underlyingVersion` variant of `canImport()`.
Resolves rdar://88723492
The ObjCMethodLookupTable for protocols was not being serialized and rebuilt on load, so NominalTypeDecl::lookupDirect() on selectors was not working correctly for deserialized types. Correct this oversight.
Change the way swiftmodules built against a different SDK than their
clients are rejected. This makes them silently ignored when the module
can be rebuilt from their swiftinterface, instead of reporting a hard
error.
rdar://93257769
llvm/llvm-project d0262c2394f46bb7da2a75529413d625c70908e5 added a new
default bool param to the two constructors in `SmallVectorMemoryBuffer`.
Since `options.OutputPath` is a `const char *` and that can be promoted
to a `bool`, the constructor being called was changed to the first
constructor (with a default buffer name) - promotion is preferred over
conversion.
Convert the various output paths to a `StringRef` - all their uses
converted to `StringRef` anyway. Also specify the default parameter in
order to maintain the old behaviour, which didn't require a null
terminator.
The main point of this change is to make sure that a shared function always has a body: both, in the optimizer pipeline and in the swiftmodule file.
This is important because the compiler always needs to emit code for a shared function. Shared functions cannot be referenced from outside the module.
In several corner cases we missed to maintain this invariant which resulted in unresolved-symbol linker errors.
As side-effect of this change we can drop the shared_external SIL linkage and the IsSerializable flag, which simplifies the serialization and linkage concept.
ABI descriptors should always be emitted as sidecars for library-evolution-enabled modules.
However, generating these files requires traversing the entire module (like indexing), which may
hit additional deserialization issues. To unblock builds, this patch introduces a flag to skip
the traversing logic so that we emit an empty ABI descriptor file. The empty file serves as
a placeholder so that build system doesn't need to know the details.
When looking for a Swift module on disk, we were scanning all module search paths if they contain the module we are searching for. In a setup where each module is contained in its own framework search path, this scaled quadratically with the number of modules being imported. E.g. a setup with 100 modules being imported form 100 module search paths could cause on the order of 10,000 checks of `FileSystem::exists`. While these checks are fairly fast (~10µs), they add up to ~100ms.
To improve this, perform a first scan of all module search paths and list the files they contain. From this, create a lookup map that maps filenames to the search paths they can be found in. E.g. for
```
searchPath1/
Module1.framework
searchPath2/
Module1.framework
Module2.swiftmodule
```
we create the following lookup table
```
Module1.framework -> [searchPath1, searchPath2]
Module2.swiftmodule -> [searchPath2]
```
We noticed some Swift clients rely on the serialized search paths in the module to
find dependencies and droping these paths altogether can lead to build failures like
rdar://85840921.
This change teaches the serialization to obfuscate the search paths and the deserialization
to recover them. This allows clients to keep accessing these paths without exposing
them when shipping the module to other users.
We've recently added the -experimental-hermetic-seal-at-link compiler flag,
which turns on aggressive dead-stripping optimizations and assumes that library
code can be optimized against client code because all users of the library
code/types are present at link/LTO time. This means that any module that's
built with -experimental-hermetic-seal-at-link requires all clients of this
module to also use -experimental-hermetic-seal-at-link. This PR enforces that
by storing a bit in the serialized module, and checking the bit when importing
modules.
