For clients, such as the debugger, who do not have access the full
output of the dependency scanner, it is a huger performance and
correctness improvement if each explicitly built Swift module not just
serialized all its Clang .pcm dependencies (via the serialized Clang
compiler invocation) but also its direct Swift module dependencies.
This patch changes the Swift module format to store the absolute path
or cas cache key for each dependency in the INPUT block, and makes
sure the deserialization makes these available to the ESML.
rdar://150969755
We already have -suppress-warnings and -suppress-remarks; this patch
adds support for suppressing notes too. Doing so is useful for -verify
tests where we don't really care about the emitted notes.
This change refactors the module loaders to explicitly take a parameter indicating whether or not the loader is handling a 'canImport' query, in order to avoid emitting an error when finding a dependency Swift binary module with only imcompatible architecture variants present.
Resolves rdar://161175498
Parser errors with large Swift module map files can be hard to diagnose.
Refactor the parser to return an llvm::Error so clearer diagnostics can
be passed to the user.
SwiftVerifyEmittedModuleInterface job is configured to be built within
the SubInvocation and it needs to inherit the caching replay prefix map
in order to load macro plugin library correctly.
rdar://158692095
When building a module interface for the -typecheck-module-from-interface or
-compile-module-from-interface actions, inherit and honor compiler debugging
options and emit debugging output at the end of the interface build.
Previously this flag was only used to pass explicit dependencies to compilation tasks. This change adds support for the dependency scanner to also consider these inputs when resolving dependencies.
Resolves https://github.com/swiftlang/swift-driver/issues/1951
An objcImpl bug previously caused `@_hasStorage` to be emitted inside some extensions in module interfaces. An earlier commit in this PR created an error for this, but for backwards compatibility, it would actually be better to simply ignore the attribute in module interfaces. Modify TypeCheckStorage to emit a warning, not an error, in this situation.
Additionally, modify the module interface loader to show warnings when you verify a module interface, but not for other module interface uses (like compiling or importing one). The assumption here is that if you’re verifying a module interface, you’re either the author of the module that created it or you’re investigating a problem with it, and in either case you’d like to be told about minor defects in case they’re related.
Fixes rdar://144811653 thoroughly.
For build systems that already generate these files, it makes sense to include the aliases so that the map file serves as a comprehensive index of how the module inputs are referenced.
Diagnostics are suppressed when parsing swiftinterface files, since the
warnings emitted from compiling the swiftinterface of a dependency would just
be a nuisance. It follows that warnings generated when parsing the arguments in
a swiftinterface file should also be suppressed, but that wasn't happening
because the diagnostic engine of the main compile was used for parsing. Pass
the diagnostic engine of the compiler subinstance instead, and proactively
suppress warnings before parsing begins.
Resolves rdar://142814164.
Duplicate module names on search paths produces an error, but
providing duplicate module names in a Swift explicit module map
file does not, instead the first entry will be chosen. Modify
the module map parser to error on duplicated module names as well.
Fix the problem that when the only module can be found is an
invalid/out-of-date swift binary module, canImport and import statement
can have different view for if the module can be imported or not.
Now canImport will evaluate to false if the only module can be found for
name is an invalid swiftmodule, with a warning with the path to the
module so users will not be surprised by such behavior.
rdar://128876895
Relying on the corresponding field in the '-explicit-swift-module-map-file' provided by the driver.
Only bridging headers require a module map because that's what aids header include resolution. With lazy module loading today, '.modulemap' parsing which happens when instantiating Clang is responsible for associating headers with modules. Then upon encountering a header include inside the bridging header the compiler knows which module corresponds to said header and is then able to load explicitly-provided PCM for that module. For all other module dependencies, they are only ever queried by-name from Swift, so '.modulemap' parsing is not necessary.
In certain cases (e.g. using arm64e interface to build arm64 target),
the target needs to be updated when building swiftinterface. Push the
target overwrite as early as possible to swiftinterface parsing by
providing a preferred target to relevant functions. In such cases, the
wrong target is never observed by other functions to avoid errors like
the sub-invocation was partially setup for the wrong target.
The SDK build version is a decent heuristic for expected changes in the
SDK. Any change in SDK, to clang headers in particular, can break
references from cached swiftmodules.
Track the SDK build version as part of the swiftmodule cache hash. This
will ensure we rebuild from swiftinterfaces on SDK updates.
rdar://122655978
A swiftmodule can only be correctly ingested by a compiler
that has a matching state of using or not-using
NoncopyableGenerics.
The reason for this is fundamental: the absence of a Copyable
conformance in the swiftmodule indicates that a type is
noncopyable. Thus, if a compiler with NoncopyableGenerics
reads a swiftmodule that was not compiled with that feature,
it will think every type in that module is noncopyable.
Similarly, if a compiler with NoncopyableGenerics produces a
swiftmodule, there will be Copyable requirements on each
generic parameter that the compiler without the feature will
become confused about.
The solution here is to trigger a module mismatch, so that
the compiler re-generates the swiftmodule file using the
swiftinterface, which has been kept compatible with the compiler
regardless of whether the feature is enabled.
Reformatting everything now that we have `llvm` namespaces. I've
separated this from the main commit to help manage merge-conflicts and
for making it a bit easier to read the mega-patch.
This is phase-1 of switching from llvm::Optional to std::optional in the
next rebranch. llvm::Optional was removed from upstream LLVM, so we need
to migrate off rather soon. On Darwin, std::optional, and llvm::Optional
have the same layout, so we don't need to be as concerned about ABI
beyond the name mangling. `llvm::Optional` is only returned from one
function in
```
getStandardTypeSubst(StringRef TypeName,
bool allowConcurrencyManglings);
```
It's the return value, so it should not impact the mangling of the
function, and the layout is the same as `std::optional`, so it should be
mostly okay. This function doesn't appear to have users, and the ABI was
already broken 2 years ago for concurrency and no one seemed to notice
so this should be "okay".
I'm doing the migration incrementally so that folks working on main can
cherry-pick back to the release/5.9 branch. Once 5.9 is done and locked
away, then we can go through and finish the replacement. Since `None`
and `Optional` show up in contexts where they are not `llvm::None` and
`llvm::Optional`, I'm preparing the work now by going through and
removing the namespace unwrapping and making the `llvm` namespace
explicit. This should make it fairly mechanical to go through and
replace llvm::Optional with std::optional, and llvm::None with
std::nullopt. It's also a change that can be brought onto the
release/5.9 with minimal impact. This should be an NFC change.
Teach swift dependency scanner to use CAS to capture the full dependencies for a build and construct build commands with immutable inputs from CAS.
This allows swift compilation caching using CAS.
When performing an implicit module build, the frontend was prepending
`-target-min-inlining-target target` to the command line. This was overriding
the implicit `-target-min-inlining-target min` argument that is implied when
`-library-level api` is specified. As a result, the wrong overload could be
picked when compiling the body of an inlinable function to SIL for emission
into the client, potentially resulting in crashes when the client of the module
is back deployed to an older OS.
Resolves rdar://109336472
For a `@Testable` import in program source, if a Swift interface dependency is discovered, and has an adjacent binary `.swiftmodule`, open up the module, and pull in its optional dependencies. If an optional dependency cannot be resolved on the filesystem, fail silently without raising a diagnostic.
Using a virutal output backend to capture all the outputs from
swift-frontend invocation. This allows redirecting and/or mirroring
compiler outputs to multiple location using different OutputBackend.
As an example usage for the virtual outputs, teach swift compiler to
check its output determinism by running the compiler invocation
twice and compare the hash of all its outputs.
Virtual output will be used to enable caching in the future.
Add '-validate-clang-modules-once' and '-clang-build-session-file' corresponding to Clang's '-fmodules-validate-once-per-build-session' and '-fbuild-session-file='. Ensure they are propagated to module interface build sub-invocations.
We require these to be first-class Swift options in order to ensure they are propagated to both: ClangImporter and implicit interface build compiler sub-invocations.
Compiler portion of rdar://105982120
Since https://github.com/apple/swift/pull/63178 added support for Clang modules in the explicit module map, it is possible for there to be multiple modules with the same name: a Swift module and a Clang module. The current parsing logic just overwrites the corresponding entry module in a hashmap so we always only preserved the module that comes last, with the same name.
This change separates the parsing of the modulemap JSON file to produce a separate Swift module map and Clang module map. The Swift one is used by the 'ExplicitSwiftModuleLoader', as before, and the Clang one is only used to populate the ClangArgs with the requried -fmodule-... flags.
If a module was first read using the adjacent swiftmodule and then
reloaded using the swiftinterface, we would do an up to date check on
the adjacent module but write out the unit using the swiftinterface.
This would cause the same modules to be indexed repeatedly for the first
invocation using a new SDK. On the next run we would instead raad the
swiftmodule from the cache and thus the out of date check would match
up.
The impact of this varies depending on the size of the module graph in
the initial compilation and the number of jobs started at the same time.
Each SDK dependency is re-indexed *and* reloaded, which is a drain on
both CPU and memory. Thus, if many jobs are initially started and
they're all going down this path, it can cause the system to run out of
memory very quickly.
Resolves rdar://103119964.
Do this by computing a transitive closure on the computed dependency graph, relying on the fact that it is a DAG.
The used algorithm is:
```
for each v ∈ V {
T(v) = { v }
}
for v ∈ V in reverse topological order {
for each (v, w) ∈ E {
T(v) = T(v) ∪ T(w)
}
}
```
This lets users of `-explicit-swift-module-map-file` use a single mapping
for all module dependencies, regardless of whether they're Swift or Clang
modules, instead of manually splitting them among this file and command
line flags.
If the json file doesn't contain a value for this, this was never set,
which results in UB.
Unfortunately clang doesn't warn about this but gcc does https://godbolt.org/z/M3sdE73zs
