Instead of the code querying the compiler's built-in Clang instance, refactor the
dependency scanner to explicitly keep track of module output path. It is still
set according to '-module-cache-path' as it has been prior to this change, but
now the scanner can use a different module cache for scanning PCMs, as specified
with '-clang-scanner-module-cache-path', without affecting module output path.
Resolves rdar://113222853
This would otherwise result in false positives, since if the old parser
skipping a body with errors would cause a verification failure.
Don't perform round trip validation either, since we'll presumbly still
hit parsing the full file when not skipping bodies - there's no point
running it twice.
Resolves rdar://111032175.
Allow `-typecheck-module-from-interface` using explicit module instead
of building implicit module.
This setups swift-frontend to accept explicit module build arguments and
loading explicit module during verifying. SwiftDriver needs to setup
correct arguments including the output path for swift module to fully
enable explicit module interface check.
Reformatting everything now that we have `llvm` namespaces. I've
separated this from the main commit to help manage merge-conflicts and
for making it a bit easier to read the mega-patch.
This is phase-1 of switching from llvm::Optional to std::optional in the
next rebranch. llvm::Optional was removed from upstream LLVM, so we need
to migrate off rather soon. On Darwin, std::optional, and llvm::Optional
have the same layout, so we don't need to be as concerned about ABI
beyond the name mangling. `llvm::Optional` is only returned from one
function in
```
getStandardTypeSubst(StringRef TypeName,
bool allowConcurrencyManglings);
```
It's the return value, so it should not impact the mangling of the
function, and the layout is the same as `std::optional`, so it should be
mostly okay. This function doesn't appear to have users, and the ABI was
already broken 2 years ago for concurrency and no one seemed to notice
so this should be "okay".
I'm doing the migration incrementally so that folks working on main can
cherry-pick back to the release/5.9 branch. Once 5.9 is done and locked
away, then we can go through and finish the replacement. Since `None`
and `Optional` show up in contexts where they are not `llvm::None` and
`llvm::Optional`, I'm preparing the work now by going through and
removing the namespace unwrapping and making the `llvm` namespace
explicit. This should make it fairly mechanical to go through and
replace llvm::Optional with std::optional, and llvm::None with
std::nullopt. It's also a change that can be brought onto the
release/5.9 with minimal impact. This should be an NFC change.
Rename `-enable-cas` to `-compile-cache-job` to align with clang option
names and promote that to a new driver only flag.
Few other additions to driver flag for caching behaviors:
* `-compile-cache-remarks`: now cache hit/miss remarks are guarded behind
this flag
* `-compile-cache-skip`: skip replaying from the cache. Useful as a
debugging tool to do the compilation using CAS inputs even the output
is a hit from the cache.
'load-plugin-library', 'load-plugin-executable', '-plugin-path' and
'-external-plugin-path' should be searched in the order they are
specified in the arguments.
Previously, for example '-plugin-path' used to precede
'-external-plugin-path' regardless of the position in the arguments.
Teach swift dependency scanner to use CAS to capture the full dependencies for a build and construct build commands with immutable inputs from CAS.
This allows swift compilation caching using CAS.
Avoid parsing the syntax tree up-front, and instead
only parse it when required, which happens when either:
1. ASTGen parsing is enabled (currently disabled
by default)
2. Round trip checking is enabled for a primary
file (enabled by default in a debug build,
except when dep scanning or doing an IDE
operation)
3. We need to evaluate a macro in that file
This change therefore means that we now no longer
need to parse the syntax tree for secondary files
by default unless we specifically need to evaluate
a macro in them (e.g if we need to lookup a member
on a decl with an attached macro). And the same
for primaries in release builds.
rdar://109283847
When swift-frontend is explicitly passed the pch file as bridging header
on command-line through `-import-objc-header`, it needs to print the
original source file name if needed to the generated objc header.
rdar://109411245
Add a CachedDiagnosticsProcessor that is a DiagConsumer can capture all
the diagnostics during a compilation, serialized them into CAS with a
format that can be replayed without re-compiling.
* Factor out ASTContext plugin loading to newly introduced 'PluginLoader'
* Insert 'DependencyTracker' to 'PluginLoader'
* Add dependencies right before loading the plugins
rdar://104938481
Teach swift how to serialize its input into CAS to create a cache key
for compiler outputs. To compute the cache key for the output, it first
needs to compute a base-key for the compiler invocation. The base key is
computed from: swift compiler version and the command-line arguments for
the invocation.
Each compiler output from swift will gets its own key. The key for the
output is computed from: the base key for the compiler invocation + the
primary input for the output + the output type.
Teach swift compiler about CAS to allow compiler caching in the future.
1) Add flags to initiate CAS inside swift-frontend
2) Teach swift to compile using a CAS file system.
Using a virutal output backend to capture all the outputs from
swift-frontend invocation. This allows redirecting and/or mirroring
compiler outputs to multiple location using different OutputBackend.
As an example usage for the virtual outputs, teach swift compiler to
check its output determinism by running the compiler invocation
twice and compare the hash of all its outputs.
Virtual output will be used to enable caching in the future.
On both input moduel source-files and interface files.
This currently yields dramatic scanning performance improvements at no cost - we do not require an AST during scan.
This modifies the ClangImporter to introduce an opaque placeholder
representation for forward declared Objective-C interfaces and
protocols when imported into Swift.
In the compiler, the new functionality is hidden behind a frontend
flag -enable-import-objc-forward-declarations, and is on by default
for language mode >6.
The feature is disabled entirely in LLDB expression evaluation / Swift
REPL, regardless of language version.
Once the API has gone through Swift Evolution, we will want to implicitly
import the _Backtracing module. Add code to do that, but set it to off
by default for now.
rdar://105394140
If a module was first read using the adjacent swiftmodule and then
reloaded using the swiftinterface, we would do an up to date check on
the adjacent module but write out the unit using the swiftinterface.
This would cause the same modules to be indexed repeatedly for the first
invocation using a new SDK. On the next run we would instead raad the
swiftmodule from the cache and thus the out of date check would match
up.
The impact of this varies depending on the size of the module graph in
the initial compilation and the number of jobs started at the same time.
Each SDK dependency is re-indexed *and* reloaded, which is a drain on
both CPU and memory. Thus, if many jobs are initially started and
they're all going down this path, it can cause the system to run out of
memory very quickly.
Resolves rdar://103119964.
Introduce a new flag `-export-as` to specify a name used to identify the
target module in swiftinterfaces. This provides an analoguous feature
for Swift module as Clang's `export_as` feature.
In practice it should be used when a lower level module `MyKitCore` is
desired to be shown publicly as a downstream module `MyKit`. This should
be used in conjunction with `@_exported import MyKitCore` from `MyKit`
that allows clients to refer to all services as being part of `MyKit`,
while the new `-export-as MyKit` from `MyKitCore` will ensure that the
clients swiftinterfaces also use the `MyKit` name for all services.
In the current implementation, the export-as name is used in the
module's clients and not in the declarer's swiftinterface (e.g.
`MyKitCore`'s swiftinterface still uses the `MyKitCore` module name).
This way the module swiftinterface can be verified. In the future, we
may want a similar behavior for other modules in between `MyKitCore` and
`MyKit` as verifying a swiftinterface referencing `MyKit` without it
being imported would fail.
rdar://103888618
This lets users of `-explicit-swift-module-map-file` use a single mapping
for all module dependencies, regardless of whether they're Swift or Clang
modules, instead of manually splitting them among this file and command
line flags.
Introduces a concept of a dependency scanning action context hash, which is used to select an instance of a global dependency scanning cache which gets re-used across dependency scanning actions.
When opaque values are enabled, TypeConverter associates to an
address-only type an OpaqueValueTypeLowering. That lowering stores a
single lowered SIL type, and its value category is "object". So long as
the module has not yet been address-lowered, that type has the
appropriate value category. After the module has been address-lowered,
however, that type has the wrong value category: the type is
address-only, and in an address-lowered module, its lowered type's value
category must be "address".
Code that obtains a lowered type expects the value category to reflect
the state of the module. So somewhere, it's necessary to fixup that
single lowered type's value category.
One option would be to update all code that uses lowered types. That
would require many changes across the codebase and all new code that
used lowered types would need to account for this.
Another option would be to update some popular conveniences that call
through to TypeConverter, for example those on SILFunction, and ensure
that all code used those conveniences. Even if this were done
completely, it would be easy enough for new code to be added which
didn't use the conveniences.
A third option would be to update TypeLowering::getLoweredType to take
in the context necessary to determine whether the stored SILType should
be fixed up. That would require each callsite to be changed and
potentially to carry around more context than it already had in order to
be able to pass it along.
A fourth option would be to make TypeConverter aware of the
address-loweredness, and to update its state at the end of
AddressLowering.
Updating TypeConverter's state would entail updating all cached
OpaqueValueTypeLowering instances at the end of the AddressLowering
pass. Additionally, when TypeConverter produces new
OpaqueValueTypeLowerings, they would need to have the "address" value
category from creation.
Of all the options, the last is least invasive and least error-prone, so
it is taken here.
