Teach scanner to respect the working directory set in the invocation
through scanner C API.
Also add test infrastructure to testing scanner from C API. Break up
DependencyScan lib into two so the swift-scan-test and remain small
without understanding swift AST.
rdar://127626011
This change modifies the dependency scanner to keep track of source locations of each encountered 'import' statement, in order to be able to emit diagnostics with source locations if an import failed to resolve.
- Keep track of each 'import' statement's source buffer, line number, and column number when adding it. The dependency scanner utilizes separate compilation instances, and therefore separate Source Managers for scanning `import` statements of user sources and textual interfaces of Swift dependencies. Since import resolution may happen in the main scanner compilation instance while the `import` itself was found by an interface-scanning sub-instance, we cannot simply hold on to the import's `SourceLoc`.
- Add libSwiftScan API for diagnostics to carry above source locations to clients.
Teach dependency scanner to pass cross import overlay file to
swift-frontend for main module compilation. This allows swift-frontend
not to repeat the file system search for overlay files when loading
modules.
This also fixes the issue when caching is enabled, the cross import
doesn't work when the first module is a clang module because the module
built with caching using clang include tree does not preserve
DefinitionLoc which is used to inferred the modulemap location for cross
import overlay search.
rdar://127844120
Improve swift dependency scanner by validating and selecting dependency
module into scanner. This provides benefits that:
* Build system does not need to schedule interface compilation task if
the candidate module is picked, it can just use the candidate module
directly.
* There is no need for forwarding module in the explicit module build.
Since the build system is coordinating the build, there is no need for
the forwarding module in the module cache to avoid duplicated work,
* This also correctly supports all the module loading modes in the
dependency scanner.
This is achieved by only adding validate and up-to-date binary module as
the candidate module for swift interface module dependency. This allows
caching build to construct the correct dependency in the CAS. If there
is a candidate module for the interface module, dependency scanner will
return a binary module dependency in the dependency graph.
The legacy behavior is mostly preserved with a hidden frontend flag
`-no-scanner-module-validation`, while the scanner output is mostly
interchangeable with new scanner behavior with `prefer-interface` module
loading mode except the candidate module will not be returned.
rdar://123711823
On all '.swiftinterface' command-line recipes, ensure we specify '-fmodule-map-file=' argument for each module which is a dependency of a header input of all swift binary module dependnecies, direct and transitive
Add support for cross import modules by ingesting swiftoverlay files for
the cross import into CAS file system.
The long-term better fix will be just passing the cross import
information from scanner to swift-frontend so frontend doesn't need to
read overlay files again to figure out the cross import module.
rdar://123839248
Add an experimental option to tell dependency scanner to report clang
cc1 args should be used to construct clang importer in all constructed
swift-frontend tasks.
When caching build is enabled, teach dependency scanner to report
command-lines with `-direct-clang-cc1-module-build` so the later
compilation can instantiate clang importer with cc1 args directly. This
avoids running clang driver code, which might involve file system
lookups, which are the file deps that are not captured and might result
in different compilation mode.
rdar://119275464
Otherwise they may have module dependencies of their own which will not be detected by the scanner and included in the list of explicit inputs for compilation.
When prefix mapping paths that are used in clang, ensure we are
consistently using the same prefix mapper from clang. This prevents
mismatches that could cause modules to fail to load.
rdar://123324072
Fix the bridging header dependencies calculation for explicit module
build, especially for caching which needs an accurate list of deps for
compute cache key correctly.
Previously, the bridging header deps are computed from `ModuleGraph`
from the clang dependency scanner, which can be affected by already seen
modules. It causes the dependencies to be missing for bridging header if
the module is seen by main swift source module.
Now report only the directly module dependencies from bridging header,
then compute all the transitive dependencies before calculating all the
cache keys.
rdar://123156636
We previously blanket omitted `-Xcc -vfsoverlay` flags from Swift module dependencies' command-line recipes. This is incorrect as the Swift module must have an exact matching VFS overlay that its Clang dependencies use, in order to load said Clang dependnecies successfully and resolve their headers as expected and as was done during the scan.
Resolves rdar://122667530
Guard 'resetCache' and 'resetDiagnostics' as critical sections using 'DependencyScanningToolStateLock'. Otherwise there's a chance that one thread in the scanner is doing a reset on the diagnostic consumer, while some other thread is adding this diagnostic consumer to another scan instance which may also be populating said consumer at the time. Similarly, for resetting 'ScanningService' though it is much more unlikely to be reset while in-use by other scanning actions.
A swiftmodule can only be correctly ingested by a compiler
that has a matching state of using or not-using
NoncopyableGenerics.
The reason for this is fundamental: the absence of a Copyable
conformance in the swiftmodule indicates that a type is
noncopyable. Thus, if a compiler with NoncopyableGenerics
reads a swiftmodule that was not compiled with that feature,
it will think every type in that module is noncopyable.
Similarly, if a compiler with NoncopyableGenerics produces a
swiftmodule, there will be Copyable requirements on each
generic parameter that the compiler without the feature will
become confused about.
The solution here is to trigger a module mismatch, so that
the compiler re-generates the swiftmodule file using the
swiftinterface, which has been kept compatible with the compiler
regardless of whether the feature is enabled.
Previously, canImport lookup is not completely working with explicit
module due to two issues:
* For clang modules, canImport check still do a full modulemap lookup
which is repeated work from scanner. For caching builds, this lookup
cannot be performed because all modulemap and search path are dropped
after scanning.
* For swift module, if the canImport module was never actually imported
later, this canImport check will fail during the actual compilation,
causing different dependencies in the actual compilation.
To fix the problem, first unified the lookup method for clang and swift
module, which will only lookup the module dependencies reported by
scanner to determine if `canImport` succeed or not. Secondly, add all
the successful `canImport` check modules into the dependency of the
current module so this information can be used during actual
compilation.
Note the behavior change here is that if a module is only checked in
`canImport` but never imported still needs to be built. Comparing to
implicit module build, this can bring in additional clang modules if
they are only check inside `canImport` but should not increase work for
swift modules (where binary module needs to be on disk anyway) or the
most common usecase for `canImport` which is to check the same module
before importing.
rdar://121082031
Avoid path encoding difference (for example, real_path vs. path from
symlink) by eliminating the path from cache key. Cache key is now
encoded with the index of the input file from all the input files from
the command-line, reguardless if those inputs will produce output or
not. This is to ensure stable ordering even the batching is different.
Add a new cache computation API that is preferred for using input index
directly. Old API for cache key is deprecated but still updated to
fallback to real_path comparsion if needed.
As a result of swift scan API change, rename the feature in JSON file to
avoid version confusion between swift-driver and libSwiftScan.
rdar://119387650
Instead, only add the overlay itself, and let it refer to its own dependencies, which will still get recorded in the overall output.
Resolves rdar://117010118
