This will allow individual module scans in batch-scanning mode to share the already-scanned Swift and Clang modules, and avoid instantiating a brand new Clang `DependencyScanningService` for each batch entry.
Performance-improvement anecdote:
A simple experiment which performs a batch scan of all modules involved in planning a build of SwiftPM (using SwiftPM in explicit module build mode), which includes 126 batch entries (scan entry-points), with 21 distinct Swift modules and 27 distinct Clang modules (many Clang modules are scanned multiple times, at different target versions), this modification reduces the total time taken from ~34 seconds to ~6.
We were previously forgetting about these.
On top of that, make sure we treat accumulated dependencies as a set, because adding clang overlay dependencies may duplicate existing already-added dependencies for some module.
When outputting strings for things like filenames, using `write_escaped` will result in Unicode characters being outputted with a full 3-character-octal or hex escape. Clients which expect a UTF-8 JSON output will not be able to parse such escape sequences.
This matches the behavior of the current client (`swift-driver`) and reduces ambiguity in how the nodes in the graph are to be treated. Swift dependencies with a textual interface, for example, must be built into a binary module by clients. Swift dependencies without a textual interface, with only a binary module, are to be used directly, without any up-to-date checks.
Note, this is distinct from Swift dependencies that have a textual interface, for which we also detect potential pre-build binary module candidates. Those are still reported in the `details` field of textual Swift dependencies as `prebuiltModuleCandidates`.
With this option enabled, the dependency scanner gathers all import statements in source files of the main module (non-transitive) and outputs a list of imported modules.
This will be used by build systems and the swift-driver as a way to avoid redundant re-scanning in incremental contexts.
Try to impose a simple structure that splits performing actions from the
pre and post-pipeline conditions. Wherever actions would take more than
a simple return, split them into functions. Refine functions that
perform effects to return status codes when they fail. Finally,
delineate functions that need semantic analysis from those that do not.
Overall this should be NFC.
To help solving rdar://67079780, this change allows swift-driver to configure scanner using additional
arguments passed down via the batch input JSON file for each module under scanning.
This scanning mode allows swift-driver to query module dependencies in a batch
and in a more granular way. In short term, it could help solve a problem that
clang module dependencies may vary if target triple changes. In a longer term,
we could break a holistic dependencies graph into smaller pieces for better caching
and reusing.
This change doesn't include the implementation of using the specified scanner
arguments to set up Clang dependencies scanner. It will come in later commits.
For the issue mentioned in rdar://67079780, swift-driver needs to run clang dependencies
scanner multiple times with different target triples for a Swift target. This patch adds
a new scanning action to generate the JSON file for a given clang module to accommodate
this requirement.
Resolves: rdar://problem/67269210
In the fast dependency scanner, depending on whether a module intrface was found via the import search path or framework search path, encode into the dependency graph Swift module details, whether a given module is a framework.
Instead of replacing an interface file with its up-to-date compile module,
the dep-scanner should report potentially up-to-date module candidates either adjacent to
the interface file or in the prebuilt module cache. swift-driver should later pass down
these candidates to -compile-module-from-interface invocation and the front-end job
will check if one of the candidates is ready to use. The front-end job then either emits a forwarding
module to an up-to-date candidate or a binary module.
For the explicit module mode, swift-driver uses -compile-module-from-interface to
generate modules from interfaces found by the dependency scanner. However, we don't
need to build the binary module if up-to-date modules are available, either adjacent
to the interface file or in the prebuilt module cache directory. This patch teaches
dependencies scanner to report these ready-to-use binary modules.
Expand the FrontendOptions to allow the enabling
of the dependency tracker for non-system
dependencies, and switch the previous clients of
`createDependencyTracker` over to using this
option. This ensures that the dependency tracker
is now set only during `CompilerInstance::setup`.
Swift interface files may specify the effective language version to use. When building
a PCM loadable for these textual interface files, we should respect the language
version. This patch moves -fapinotes-swift-version from the generic PCM
commands to the extra PCM arguments owned by each loading Swift module.
Building each Swift module explicitly requires dependency PCMs to be built
with the exactly same deployment target version. This means we may need to
build a Clang module multiple times with different target triples.
This patch removes the -target arguments from the reported PCM build
arguments and inserts extraPcmArgs fields to each Swift module.
swift-driver can combine the generic PCM arguments with these extra arguments
to get the command suitable for building a PCM specifically for
that loading Swift module.
When resolving direct dependencies for a given Swift module, we go over all Clang module dependencies and add, as additional dependencies, their Swift overlays. We find overlays by querying `ASTContext::getModuleDependencies` with the Clang module's name. If the Clang module in question is a dependency of a Swift module with the same name, we will end up adding the Swift module as its own dependence.
e.g.
- Swift A depends on Clang A
- Add Clang A to dependencies of Swift A
- We look for overlays of Clang A, by name, and find Swift A
- Add Swift A to dependencies of Swift A
From what I can tell, the logic upstream is sound, and `getModuleDependencies` is doing the right thing, so this change is simply restricting what gets added when we are looking for overlays.
Resolves rdar://problem/63731428
Module interface builder used to maintain a separate compiler instance for
building Swift modules. The configuration of this compiler instance is also
useful for dependencies scanner because it needs to emit front-end compiler invocation
for building Swift modules explicitly.
This patch refactor the configuration out to a delegate class, and the
delegate class is also used by the dependency scanner.
Additional flags in interface files may change parsing behavior like #if
statements. We should use a fresh ASTContext with these additional
flags when parsing interface files to collect imports.
rdar://62612027
