By populating the memory cache before loading the module, we can avoid a cycle
where a module is imported that is an overlay, which then triggers
ClangImporter, which then (redundantly) triggers the import of the overlay
module, which would reimport the module again, since it's import is still
underway and it hasn't been entered into the cache yet.
rdar://118846313
A swiftmodule can only be correctly ingested by a compiler
that has a matching state of using or not-using
NoncopyableGenerics.
The reason for this is fundamental: the absence of a Copyable
conformance in the swiftmodule indicates that a type is
noncopyable. Thus, if a compiler with NoncopyableGenerics
reads a swiftmodule that was not compiled with that feature,
it will think every type in that module is noncopyable.
Similarly, if a compiler with NoncopyableGenerics produces a
swiftmodule, there will be Copyable requirements on each
generic parameter that the compiler without the feature will
become confused about.
The solution here is to trigger a module mismatch, so that
the compiler re-generates the swiftmodule file using the
swiftinterface, which has been kept compatible with the compiler
regardless of whether the feature is enabled.
This is a new attempt at a reexport feature for SPI decls. The previous
behavior was to reexport SPIs only between modules with both `@_exported` and
an export-as relationship. The limitation on export-as turned out to be too
restrictive in some cases.
We may be tempted to reexport SPIs through all exported imports. However,
the exported imports are very common between clang module and it can lead
to surprises if dependencies between clang modules end up exporting SPIs from
unexpected modules.
As a middle ground, reexport SPI only through Swift `@_exported` dependencies,
and not through clang reexports. While this is a new distinction between Swift
and clang dependencies, I believe it provides the expected behavior and
the result is more straightforward than the current logic.
rdar://115901208
This reverts commit 3cc2831608.
The compiler's revision check has been relaxed since the feature was introduced
and so it's nos better to reduce the number of special code paths for LLDB in
the compiler to facilitate reasoning about it.
rdar://117824367
Add a new flag to enable package interface loading.
Use the last value of package-name in case of dupes.
Rename PrintInterfaceContentMode as InterfaceMode.
Update diagnostics.
Test package interface loading with various scenarios.
Test duplicate package-name.
It has an extension .package.swiftinterface and contains package decls
as well as SPIs and public/inlinable decls. When a module is loaded
from interface, it now looks up the package-name in the interface
and checks if the importer is in the same package. If so, it uses
that package interface found to load the module. If not, uses the existing
logic to load modules.
Resolves rdar://104617854
We only record these dependencies in CAS mode, because we require explicit PCH tasks to be produced for imported header of binary module dependencies. In the meantime, in non-CAS mode loading clients will consume the `.h` files encoded in the `.swiftmodules` directly.
Followup changes to SwiftDriver will enable explicit PCH compilation of such dependenceis, but for the time being restore prior behavior for non-CAS explicit module builds.
Resolves rdar://116006619
- Add a flag to the serialized module (IsEmbeddedSwiftModule)
- Check on import that the mode matches (don't allow importing non-embedded module in embedded mode and vice versa)
- Drop TBD support, it's not expected to work in embedded Swift for now
- Drop auto-linking backdeploy libraries, it's not expected to backdeploy embedded Swift for now
- Drop prespecializations, not expected to work in embedded Swift for now
- Use CMO to serialize everything when emitting an embedded Swift module
- Change SILLinker to deserialize/import everything when importing an embedded Swift module
- Add an IR test for importing modules
- Add a deserialization validation test
Pointer `llvm/Support/Host.h` at `llvm/TargetParser/Host.h`.
Replacing deprecated API `startswith_insensitive` with replacement
`starts_with_insensitive`.
Reformatting everything now that we have `llvm` namespaces. I've
separated this from the main commit to help manage merge-conflicts and
for making it a bit easier to read the mega-patch.
This is phase-1 of switching from llvm::Optional to std::optional in the
next rebranch. llvm::Optional was removed from upstream LLVM, so we need
to migrate off rather soon. On Darwin, std::optional, and llvm::Optional
have the same layout, so we don't need to be as concerned about ABI
beyond the name mangling. `llvm::Optional` is only returned from one
function in
```
getStandardTypeSubst(StringRef TypeName,
bool allowConcurrencyManglings);
```
It's the return value, so it should not impact the mangling of the
function, and the layout is the same as `std::optional`, so it should be
mostly okay. This function doesn't appear to have users, and the ABI was
already broken 2 years ago for concurrency and no one seemed to notice
so this should be "okay".
I'm doing the migration incrementally so that folks working on main can
cherry-pick back to the release/5.9 branch. Once 5.9 is done and locked
away, then we can go through and finish the replacement. Since `None`
and `Optional` show up in contexts where they are not `llvm::None` and
`llvm::Optional`, I'm preparing the work now by going through and
removing the namespace unwrapping and making the `llvm` namespace
explicit. This should make it fairly mechanical to go through and
replace llvm::Optional with std::optional, and llvm::None with
std::nullopt. It's also a change that can be brought onto the
release/5.9 with minimal impact. This should be an NFC change.
'load-plugin-library', 'load-plugin-executable', '-plugin-path' and
'-external-plugin-path' should be searched in the order they are
specified in the arguments.
Previously, for example '-plugin-path' used to precede
'-external-plugin-path' regardless of the position in the arguments.
Teach swift dependency scanner to use CAS to capture the full dependencies for a build and construct build commands with immutable inputs from CAS.
This allows swift compilation caching using CAS.
This disallows building an interface file that imports such module which should be allowed
since interface does not contain package symbols unless usableFromInline or inlinable.
This change limits erroring only when building a .swift file.
Resolves rdar://108633068
If we have both loaded a swiftdoc, and the decl we
have should have had its doc comment serialized into
it, we can check it without needing to fall back
to the swiftsourceinfo.
This requires a couple of refactorings:
- Factoring out the `shouldIncludeDecl` logic
into `getDocCommentSerializationTargetFor` for
determining whether a doc comment should end up
in the swiftdoc or not.
- Factoring out `CommentProviderFinder` for searching
for the doc providing comment decl for brief
comments, in order to allow us to avoid querying
the raw comment when searching for it. This has the
added bonus of meaning we no longer need to fall
back to parsing the raw comment for the brief
comment if the comment is provided by another decl
in the swiftdoc.
This diff is best viewed without whitespace.
They may be a super-set of the ones that appear in the textual interface - e.g. 'internal' imports will be contained in the adjacent binary module, but not the textual interface
For a `@Testable` import in program source, if a Swift interface dependency is discovered, and has an adjacent binary `.swiftmodule`, open up the module, and pull in its optional dependencies. If an optional dependency cannot be resolved on the filesystem, fail silently without raising a diagnostic.
The macro name resolution in the source lookup cache was only looking at
macros in the current module, meaning that any names introduced by peer
or declaration macros declared in one module but used in another would
not be found by name lookup.
Switch the source lookup cache over to using the same
`forEachPotentialResolvedMacro` API that is used by lookup within
types, so we have consistent name-lookup-level macro resolution in both
places.
... except that would be horribly cyclic, of course, so introduce name
lookup flags to ignore top-level declarations introduced by macro
expansions. This is semantically correct because macro expansions are
not allowed to introduce new macros anyway, because that would have
been a terrible idea.
Fixes rdar://107321469. Peer and declaration macros at module scope
should work a whole lot better now.
Add a private discriminator to the mangling of an outermost-private `MacroExpansionDecl` so that declaration macros in different files won't have colliding macro expansion buffer names.
rdar://107462515
When using the -testable-import-module argument to insert a testable
import, there's no ImportDecl on which to show the diagnostics when
loading transitive dependencies. Clean up the logic to still load
dependencies in such a case.
A @testable import allows a client to call internal decls which may
refer to non-public dependencies. To support such a use case, load
non-public transitive dependencies of a module when it's imported
@testable from the main module.
This replaces the previous behavior where we loaded those dependencies
for any modules built for testing. This was risky as we would load more
module for any debug build, opening the door to a different behavior
between debug and release builds. In contrast, applying this logic to
@testable clients will only change the behavior of test targets.
rdar://107329303
The new diagnoseSerializedASTLoadFailureTransitive diagnose problems for
transitive dependencies only: missing dependency, missing underlying
module, or circular dependency.
When loading a swiftmodule A, read its package information to tell if
the current client should load A's dependencies imports by a package
import. Only clients belonging to the same package as A should load
those dependencies, clients outside of the package likely don't have
access to those dependencies.
This is specific to swiftmodules as swiftinterfaces never display a
package-only import. Clients are unaware of package dependencies when
building against a swiftinterface.
rdar://106164813
Swiftmodules built by a mismatching compiler are loaded if the mismatch
is only on the last digit of the compiler version. In such a case, write
a remark directly to stderr to avoid showing this error in IDE but keep
it for debugging purposes.
rdar://105881894
Weaken the precise tag check at loading swiftmodule to accept binary
modules build by a compiler with a tag where only the last digit is
different. We assume that the other digit in the version should ensure
compiler and stdlib compatibility. If the last digit doesn't match,
still raise a remark.
rdar://105158258
If a module was first read using the adjacent swiftmodule and then
reloaded using the swiftinterface, we would do an up to date check on
the adjacent module but write out the unit using the swiftinterface.
This would cause the same modules to be indexed repeatedly for the first
invocation using a new SDK. On the next run we would instead raad the
swiftmodule from the cache and thus the out of date check would match
up.
The impact of this varies depending on the size of the module graph in
the initial compilation and the number of jobs started at the same time.
Each SDK dependency is re-indexed *and* reloaded, which is a drain on
both CPU and memory. Thus, if many jobs are initially started and
they're all going down this path, it can cause the system to run out of
memory very quickly.
Resolves rdar://103119964.
These modules are not guaranteed to be found, which is okay, as compilation is meant to be possible in their absense since their contents are not used in the public API of the module which imports them as implementation-only.
Resolves rdar://103031296
This changes the scanner's behavior to "resolve" a discovered module's dependencies to a set of Module IDs: module name + module kind (swift textual, swift binary, clang, etc.).
The 'ModuleDependencyInfo' objects that are stored in the dependency scanner's cache now carry a set of kind-qualified ModuleIDs for their dependencies, in addition to unqualified imported module names of their dependencies.
Previously, the scanner's internal state would cache a module dependnecy as having its own set of dependencies which were stored as names of imported modules. This led to a design where any time we needed to process the dependency downstream from its discovery (e.g. cycle detection, graph construction), we had to query the ASTContext to resolve this dependency's imports, which shouldn't be necessary. Now, upon discovery, we "resolve" a discovered dependency by executing a lookup for each of its imported module names (this operation happens regardless of this patch) and store a fully-resolved set of dependencies in the dependency module info.
Moreover, looking up a given module dependency by name (via `ASTContext`'s `getModuleDependencies`) would result in iterating over the scanner's module "loaders" and querying each for the module name. The corresponding modules would then check the scanner's cache for a respective discovered module, and if no such module is found the "loader" would search the filesystem.
This meant that in practice, we searched the filesystem on many occasions where we actually had cached the required dependency, as follows:
Suppose we had previously discovered a Clang module "foo" and cached its dependency info.
-> ASTContext.getModuleDependencies("foo")
--> (1) Swift Module "Loader" checks caches for a Swift module "foo" and doesn't find one, so it searches the filesystem for "foo" and fails to find one.
--> (2) Clang Module "Loader" checks caches for a Clang module "foo", finds one and returns it to the client.
This means that we were always searching the filesystem in (1) even if we knew that to be futile.
With this change, queries to `ASTContext`'s `getModuleDependencies` will always check all the caches first, and only delegate to the scanner "loaders" if no cached dependency is found. The loaders are then no longer in the business of checking the cached contents.
To handle cases in the scanner where we must only lookup either a Swift-only module or a Clang-only module, this patch splits 'getModuleDependencies' into an alrady-existing 'getSwiftModuleDependencies' and a newly-added 'getClangModuleDependencies'.
