An "API descriptor" file is JSON describing the externally accessible symbols
of a module and metadata associated with those symbols like availability and
SPI status. This output was previously only generated by the
`swift-api-extract` alias of `swift-frontend`, which is desgined to take an
already built module as input. Post-processing a built module to extract this
information is inefficient because the module and the module's dependencies
need to be deserialized in order to visit the entire AST. We can generate this
output more efficiently as a supplementary output of the -emit-module job that
originally produced the module (since the AST is already available in-memory).
The -emit-api-descriptor flag can be used to request this output.
This change lays the groundwork by introducing frontend flags. Follow up
changes are needed to make API descriptor emission during -emit-module
functional.
Part of rdar://110916764.
Conflict in CAS options when
`std::vector<std::string> CacheReplayPrefixMap;` was added.
Conflicts:
include/swift/Frontend/FrontendOptions.h
Resolution: Take both
Teach CachedDiagnosticsProcessor to replay diagnostics for all consumers
with a different path using a path prefix map. This allows diagnostics
to be replayed for the actual path on disk after the scanner
canonicalized the path for compilation.
Allow DependencyScanner to canonicalize path using a prefix map. When
option `-scanner-prefix-map` option is used, dependency scanner will
remap all the input paths in following:
* all the paths in the CAS file system or clang include tree
* all the paths related to input on the command-line returned by scanner
This allows all the input paths to be canonicalized so cache key can be
computed reguardless of the exact on disk path.
The sourceFile field is not remapped so build system can track the exact
file as on the local file system.
'ModuleDependencyScanner' maintains a Thread Pool along with a pool of workers
which are capable of executing a filesystem lookup of a named module dependency.
When resolving imports of a given Swift module, each import's resolution
operation can be issued asunchronously.
From being a scattered collection of 'static' methods in ScanDependencies.cpp
and member methods of ASTContext. This makes 'ScanDependencies.cpp' much easier
to read, and abstracts the actual scanning logic away to a place with common
state which will make it easier to reason about in the future.
When we run an interface verification tasks with Explicit module builds, we directly invoke a '-explicit-interface-module-build' instance with a '-typecheck-module-from-interface' action. So the builder needs to recognize this as a typechecking invocation. In implicit builds, this gets lowered into a separate compiler sub-instance with a '-typecheck' action, for some reason.
resolves rdar://115565571
To enable MCCAS, the following driver options have been added
-cas-backend: Enable MCCAS backend in swift, the option
-cache-compile-job must also be used.
-cas-backend-mode=native: Set the CAS Backend mode to emit an object
file after materializing it from the CAS.
-cas-backend-mode=casid: Emit a file with the CASID for the CAS that was
created.
-cas-backend-mode=verify: Verify that the object file created is
identical to the object file materialized from the CAS.
-cas-emit-casid-file: Emit a .casid file next to the object file when
CAS Backend is enabled.
We'll be using the new swift-syntax diagnostic formatter in the near
future, as it is nearly available on all host platforms. So, remove
the C++ formatter that did source-line annotation, falling back to the
"LLVM" style when swift-syntax is not compiled in.
This enables one to use varying prefixes when checking diagnostics with the
DiagnosticVerifier. So for instance, I can make a test work both with and
without send-non-sendable enabled by adding additional prefixes. As an example:
```swift
// RUN: %target-swift-frontend ... -verify-additional-prefix no-sns-
// RUN: %target-swift-frontend ... -verify-additional-prefix sns-
let x = ... // expected-error {{This is always checked no matter what prefixes I added}}
let y = ... // expected-no-sns-error {{This is only checked if send non sendable is disabled}}
let z = ... // expected-sns-error {{This is only checked if send non sendable is enabled}}
let w = ... // expected-no-sns-error {{This is checked for a specific error when sns is disabled...}}
// expected-sns-error @-1 {{and for a different error when sns is enabled}}
```
rdar://114643840
This option is designed to be used in conjunction with
`-experimental-lazy-typecheck` and `-experimental-skip-all-function-bodies`
when emitting a resilient module. The emitted binary module should contain only
the decls needed by clients and should contain roughly the same contents as it
would if the corresponding swiftinterface were emitted instead and then built.
This functionality is a work in progress. Some parts of the AST may still get
typechecked unnecessarily. Additionally, serialization does not trigger the
appropriate typechecking requests for some ASTs and then fails due to missing
types.
Resolves rdar://114230586
This action is currently just an alias of the `-resolve-imports` action.
However, it's named to more clearly reflect the purpose which is to do the
minimal typechecking needed in order to emit the requested outputs. This mode
is intended to improve performance when emitting `.swiftinterface` and `.tbd`
files.
Instead of the code querying the compiler's built-in Clang instance, refactor the
dependency scanner to explicitly keep track of module output path. It is still
set according to '-module-cache-path' as it has been prior to this change, but
now the scanner can use a different module cache for scanning PCMs, as specified
with '-clang-scanner-module-cache-path', without affecting module output path.
Resolves rdar://113222853
Clang dependency scanning produces scanner PCMs which we may want to live in a
different filesystem location than the main build module cache.
Resolves rdar://113222853
Allow `-typecheck-module-from-interface` using explicit module instead
of building implicit module.
This setups swift-frontend to accept explicit module build arguments and
loading explicit module during verifying. SwiftDriver needs to setup
correct arguments including the output path for swift module to fully
enable explicit module interface check.
Reformatting everything now that we have `llvm` namespaces. I've
separated this from the main commit to help manage merge-conflicts and
for making it a bit easier to read the mega-patch.
This is phase-1 of switching from llvm::Optional to std::optional in the
next rebranch. llvm::Optional was removed from upstream LLVM, so we need
to migrate off rather soon. On Darwin, std::optional, and llvm::Optional
have the same layout, so we don't need to be as concerned about ABI
beyond the name mangling. `llvm::Optional` is only returned from one
function in
```
getStandardTypeSubst(StringRef TypeName,
bool allowConcurrencyManglings);
```
It's the return value, so it should not impact the mangling of the
function, and the layout is the same as `std::optional`, so it should be
mostly okay. This function doesn't appear to have users, and the ABI was
already broken 2 years ago for concurrency and no one seemed to notice
so this should be "okay".
I'm doing the migration incrementally so that folks working on main can
cherry-pick back to the release/5.9 branch. Once 5.9 is done and locked
away, then we can go through and finish the replacement. Since `None`
and `Optional` show up in contexts where they are not `llvm::None` and
`llvm::Optional`, I'm preparing the work now by going through and
removing the namespace unwrapping and making the `llvm` namespace
explicit. This should make it fairly mechanical to go through and
replace llvm::Optional with std::optional, and llvm::None with
std::nullopt. It's also a change that can be brought onto the
release/5.9 with minimal impact. This should be an NFC change.
Rename `-enable-cas` to `-compile-cache-job` to align with clang option
names and promote that to a new driver only flag.
Few other additions to driver flag for caching behaviors:
* `-compile-cache-remarks`: now cache hit/miss remarks are guarded behind
this flag
* `-compile-cache-skip`: skip replaying from the cache. Useful as a
debugging tool to do the compilation using CAS inputs even the output
is a hit from the cache.
'load-plugin-library', 'load-plugin-executable', '-plugin-path' and
'-external-plugin-path' should be searched in the order they are
specified in the arguments.
Previously, for example '-plugin-path' used to precede
'-external-plugin-path' regardless of the position in the arguments.
Teach swift dependency scanner to use CAS to capture the full dependencies for a build and construct build commands with immutable inputs from CAS.
This allows swift compilation caching using CAS.
When performing an implicit module build, the frontend was prepending
`-target-min-inlining-target target` to the command line. This was overriding
the implicit `-target-min-inlining-target min` argument that is implied when
`-library-level api` is specified. As a result, the wrong overload could be
picked when compiling the body of an inlinable function to SIL for emission
into the client, potentially resulting in crashes when the client of the module
is back deployed to an older OS.
Resolves rdar://109336472
When swift-frontend is explicitly passed the pch file as bridging header
on command-line through `-import-objc-header`, it needs to print the
original source file name if needed to the generated objc header.
rdar://109411245
This will mean that '-disable-implicit-swift-modules' also automatically implies two things:
1. Clang modules must also be explicit, and the importer's clang instance will get '-fno-implicit-modules' and '-fno-implicit-module-maps'
2. The importer's clang instance will no longer get a '-fmodules-cache-path=', since it is not needed in explicit builds
