- '-o <output_path>'
- '-disable-implicit-swift-modules'
- '-Xcc -fno-implicit-modules' and '-Xcc -fno-implicit-module-maps'
- '-candidate-module-file'
These were previously supplied by the driver. Instead, they will now be ready to be run directly from the dependency scanner's output.
This changes the scanner's behavior to "resolve" a discovered module's dependencies to a set of Module IDs: module name + module kind (swift textual, swift binary, clang, etc.).
The 'ModuleDependencyInfo' objects that are stored in the dependency scanner's cache now carry a set of kind-qualified ModuleIDs for their dependencies, in addition to unqualified imported module names of their dependencies.
Previously, the scanner's internal state would cache a module dependnecy as having its own set of dependencies which were stored as names of imported modules. This led to a design where any time we needed to process the dependency downstream from its discovery (e.g. cycle detection, graph construction), we had to query the ASTContext to resolve this dependency's imports, which shouldn't be necessary. Now, upon discovery, we "resolve" a discovered dependency by executing a lookup for each of its imported module names (this operation happens regardless of this patch) and store a fully-resolved set of dependencies in the dependency module info.
Moreover, looking up a given module dependency by name (via `ASTContext`'s `getModuleDependencies`) would result in iterating over the scanner's module "loaders" and querying each for the module name. The corresponding modules would then check the scanner's cache for a respective discovered module, and if no such module is found the "loader" would search the filesystem.
This meant that in practice, we searched the filesystem on many occasions where we actually had cached the required dependency, as follows:
Suppose we had previously discovered a Clang module "foo" and cached its dependency info.
-> ASTContext.getModuleDependencies("foo")
--> (1) Swift Module "Loader" checks caches for a Swift module "foo" and doesn't find one, so it searches the filesystem for "foo" and fails to find one.
--> (2) Clang Module "Loader" checks caches for a Clang module "foo", finds one and returns it to the client.
This means that we were always searching the filesystem in (1) even if we knew that to be futile.
With this change, queries to `ASTContext`'s `getModuleDependencies` will always check all the caches first, and only delegate to the scanner "loaders" if no cached dependency is found. The loaders are then no longer in the business of checking the cached contents.
To handle cases in the scanner where we must only lookup either a Swift-only module or a Clang-only module, this patch splits 'getModuleDependencies' into an alrady-existing 'getSwiftModuleDependencies' and a newly-added 'getClangModuleDependencies'.
`getValue` -> `value`
`getValueOr` -> `value_or`
`hasValue` -> `has_value`
`map` -> `transform`
The old API will be deprecated in the rebranch.
To avoid merge conflicts, use the new API already in the main branch.
rdar://102362022
Move clangScanningTool and clangScanningService to be parts of 'ModuleDependenciesCache' state, getting rid of the 'ClangModuleDependenciesCacheImpl', which is no-longer needed since we moved moved to by-name lookup of Clang modules.
The Swift compiler does not have a concept of a working directory. It is instead handled by the Swift driver by resolving relative paths according to the driver's working directory argument. On the other hand, Clang does have a concept working directory which may be specified on this Clang invocation with '-working-directory'. If so, it is crucial that we use this directory as an argument to the Clang scanner API. Otherwiswe, we risk having a mismatch between the working directory specified on the scanner's Clang invocation and the one use from the scanner API entry-points, which leads to downstream inconsistencies and errors.
apple/llvm-project 000a90ab63ce0595d442b3e7e8adc9c0506d2525 removed
`getAdditionalArgsWithoutModulePaths`.
092b2ddf7a swapped to using
`getCommandLineWithoutModulePaths` instead, but this is (almost) all cc1
args used to build the module - which is very different to what was
being looped over previously.
Inline `getAdditionalArgsWithoutModulePaths` instead.
Resolves rdar://94545725.
apple/llvm-project 000a90ab63ce0595d442b3e7e8adc9c0506d2525 removed
`getAdditionalArgsWithoutModulePaths`. Based on the v2 libclang API, we
should be using `getCommandLineWithoutModulePaths` now.
When looking for a Swift module on disk, we were scanning all module search paths if they contain the module we are searching for. In a setup where each module is contained in its own framework search path, this scaled quadratically with the number of modules being imported. E.g. a setup with 100 modules being imported form 100 module search paths could cause on the order of 10,000 checks of `FileSystem::exists`. While these checks are fairly fast (~10µs), they add up to ~100ms.
To improve this, perform a first scan of all module search paths and list the files they contain. From this, create a lookup map that maps filenames to the search paths they can be found in. E.g. for
```
searchPath1/
Module1.framework
searchPath2/
Module1.framework
Module2.swiftmodule
```
we create the following lookup table
```
Module1.framework -> [searchPath1, searchPath2]
Module2.swiftmodule -> [searchPath2]
```
The clang importer sets the clang resource directory to
`RuntimeResourcePath`/clang (usually "lib/swift/clang", a symbolic link
into "lib/clang/<version>"). This was inadvertently being removed in a
check to remove the clang *executable* path.
Modify that check to only the first argument so the resource directory
is properly passed through.
Doing so will allow clients to know which Swift-specific PCM arguments are already captured from the scan that first discovered this module.
SwiftDriver, in particular, will be able to use this information to avoid re-scanning a given Clang module if the initial scan was sufficient for all possible sets of PCM arguments on Swift modules that depend on said Clang module.
And only resolve cached dependencies that came from scanning actions with the same target triple.
This change means that the `GlobalModuleDependenciesCache` must be configured with a specific target triple for every scannig action, and it will only resolve previously-found dependencies from previous scannig actions using the exact same triple.
Furthermore, the `GlobalModuleDependenciesCache` separately tracks source-file-based module dependencies as those represent main Swift modules of previous scanning actions, and we must be able to resolve those regardless of the target triple.
Resolves rdar://83105455
The SingleCommandCompilationDatabase type was removed from LLVM and the
API's that took it were changed to just take the vector of arguments
instead. This patch updates the calls to `getFullDependencies` to pass
in the argument vector to reflect that change and get things building
again.
These kinds of modules differ from `SwiftTextual` modules in that they do not have an interface and have source-files.
It is cleaner to enforce this distinction with types, instead of checking for interface optionality everywhere.
The dependency scanner's cache persists across different queries and answering a subsequent query's module lookup with a module not in the query's search path is not correct.
For example, suppose we are looking for a Swift module `Foo` with a set of search paths `SP`.
And dependency scanner cache already contains a module `Foo`, for which we found an interface file at location `L`. If `L`∉`SP`, then we cannot re-use the cached entry because we’d be resolving the scanning query to a filesystem location that the current scanning context is not aware of.
Resolves rdar://81175942
https://github.com/apple/swift/pull/37774 (related to rdar://72480261) has made it so that the target of built clang modules (even downstream from Swift interface dependencies) can be consistent with that of the main module. This means that when building transitive Clang dependencies of the main module, they will always have a matching triple to the main module itself (ensured with `-clang-target`). This means we no longer have to report the target triple in the set of `extraPCMArgs` which encode an interface-specific requirement for building its dependencies.
Pass a wrapped VFS down into `clang::createInvocationFromCommandLine` so
that the working directory is set and then used in the underlying Clang
`CompilerInstance`.
Fixes the possibility of differing modules hashes when the same
arguments are used in Clang directly vs from the importer.
Resolves rdar://79376364
This value was replaced with a call to 'getNonPathCommandLine()' in
llvm-project commit 2b73565210ef0b3b29e0f067eef150a32adbb967. The API
was later renamed 'getAdditionalArgsWithoutModulePaths()' in
llvm-project commit 77fc4d1feef5f3041f614206447b4461992054bb.
When building a set of command-line options required to build a Clang module, also add `NonPathCommandLine` from Clang's `ModuleDeps`. These flags are mandatory in order to build modules discovered by the scanner.
Resolves rdar://70212660
This matches the behavior of the current client (`swift-driver`) and reduces ambiguity in how the nodes in the graph are to be treated. Swift dependencies with a textual interface, for example, must be built into a binary module by clients. Swift dependencies without a textual interface, with only a binary module, are to be used directly, without any up-to-date checks.
Note, this is distinct from Swift dependencies that have a textual interface, for which we also detect potential pre-build binary module candidates. Those are still reported in the `details` field of textual Swift dependencies as `prebuiltModuleCandidates`.
We need ClangImporterOptions to be persistent for several scenarios: (1)
when creating a sub-ASTContext to build Swift modules from interfaces; and
(2) when creating a new Clang instance to invoke Clang dependencies scanner.
This change is NFC.
This ensures that when the dependency scanner is invoked with additional clang (`-Xcc`) options, the Clang scanner is correctly configured using these options.
Unlike Swift modules, building Clang PCMs requires search path options like -F and -I to
look for transitive header includes because in module maps, we can only find top-level headers.
rdar://67589328
LLVM, as of 77e0e9e17daf0865620abcd41f692ab0642367c4, now builds with
-Wsuggest-override. Let's clean up the swift sources rather than disable
the warning locally.
