Previously Swift overlay lookup was performed for every directly and transitively-imported Clang module.
https://github.com/llvm/llvm-project/pull/147969 introduced the concept of "visible" Clang modules from a given named Clang dependency scanner query which closely maps to the set of modules for which Swift will attempt to load a Swift overlay. This change switches overlay querying to apply only to the set of such visible modules.
Resolves rdar://144797648
Previously this flag was only used to pass explicit dependencies to compilation tasks. This change adds support for the dependency scanner to also consider these inputs when resolving dependencies.
Resolves https://github.com/swiftlang/swift-driver/issues/1951
When querying a Swift module, the scanner now also keeps track of all discovered candidate binary modules which are not compatible with current compilation.
- If a Swift dependency is successfully resolved to a compatible binary module or a textual interface, a warning is emitted for every incompatible binary Swift module discovered along the way.
- If a Swift dependency is not resolved, but incompatible module candidates were found, an error is emitted - while it is likely that the scan would fail downstream, it is also possible that an underlying Clang module dependency (with the same name) is successfuly resolved and the Swift lookup failure is ignored, which is still going to lead to failures most of the time if the client code assumes the presence of the Swift overlay module in this scenario.
This change refactors common error reporting by the scanner into a 'ModuleDependencyIssueReporter' class, which also keeps track of all diagnosed failed lookups to avoid repeating diagnostics.
A prior change ensured that we forego this query when looking up Swift overlays for a textual interface which was built without C++ interop. This change introduced a bug where it also caused us to skip this lookup for the main source module. This commit resolves that by preserving the fix above but also ensuring we perform the lookup for the main source module under scan.
The note will point the user to where the "other" module with the same name is located and mention whether it is an SDK module. This is nice to have in various circumstances where developers attempt to define a module with the same name as a Swift module that already exists on their search paths, for example in the SDK.
Move per-query state out of ScanningService. There is still a check to
make sure the CASOptions are matching between queries because of the
requirement on clang scanner. Otherwise, the scanning service should
contain no per-query information anymore.
Resolves: https://github.com/swiftlang/swift/issues/82490
While this made sense in the distant past where the scanning service provided backing storage for the dependency cache, it no longer does so and now makes for awkard layering where clients get at the service via the cache. Now the cache is a simple data structure while all the clients that need access to the scanning service will get it explicitly.
- 'SwiftModuleScanner' will now be owned directly by the 'ModuleDependencyScanningWorker' and will contain all the necessary custom logic, instead of being instantiated by the module interface loader for each query
- Moves ownership over module output path and sdk module output path directly into the scanning worker, instead of the cache
This was used a long time ago for a design of a scanner which could rely on the client to specify that some modules *will be* present at a given location but are not yet during the scan. We have long ago determined that the scanner must have all modules available to it at the time of scan for soundness. This code has been stale for a couple of years and it is time to simplify things a bit by deleting it.
Adds an access control field for each imported module identified. When multiple imports of the same module are found, this keeps track of the most "open" access specifier.
Unlike with implicitly-built modules (prior to Swift 6 mode), explicitly-built modules require that all search paths be specified explicitly and no longer inherit search paths serialized into discovered Swift binary modules. This behavior was never intentional and is considered a bug. This change adds a diagnostic note to a scan failure: for each binary Swift module dependency, the scanner will attempt to execute a dependency scanning query for each serialized search path inside that module. If such diagnostic query returns a result, a diagnostic will be emitted to inform the user that the dependency may be found in the search path configuration of another Swift binary module dependency, specifying which search path contains the "missing" module, and stating that such search paths are not automatically inherited by the current compilation.
Textual interfaces for 'Darwin' built with recent compilers specify that it is built witout C++ interop enabled. However, to ensure compatibility with versions of the 'Darwin' module built with older compilers, we hard-code this fact. This is required to break the module cycle that occurs when building the 'Darwin' module with C++ interop enabled, where the underlying 'Darwin' clang module depends on C++ standard library for which the compiler brings in the 'CxxStdlib' Swift overlay, which depends on 'Darwin'.
On creation, 'ClangImporter' adds overlay modulemap files for non-modular platform libraries (e.g. glibc, libstdc++), which allows Swift code to import and use those libraries.
This change adds the same filesystem overlay to dependency scanning queries by applying them to the filesystem instantiated for each depndency scanning worker. Without these overlays EBM builds cannot discover and use non-modular system libraries on non-Darwin platforms.
Resolves rdar://151780437
Otherwise querying this clang module, e.g. from the corresponding Swift overlay's underlying module import, will fail, since no such module exists.
Resolves rdar://151718115
After removing the CASFS implementation for clang modules, there is no
need to capture clang extra file that sets up the VFS for the clang
modules since all content imported by ClangImporter is dependency
scanned and available via include-tree. This saves more ClangImporter
instance when caching is enabled.
Update the test to check that clang content found via `-Xcc` VFS options
can currently work without capture the headermaps and vfs overlays.
When we discover a textual module dependency which is a module which was not originally built from source using C++ interop (specifying '-formal-cxx-interoperability-mode=off'), avoid looking up the C++ standard library Swift overlay for it. This is required for the case of the 'Darwin' module, for example, which includes headers which map to C++ stdlib headers when the compiler is operating in C++ interop mode, but the C++ standard library Swift overlay module itself depends on 'Darwin', which results in a cycle. To resolve such situations, we can rely on the fact that Swift textual interfaces of modules which were not built with C++ interop must be able to build without importing the C++ standard library Swift overlay, so we avoid specifying it as a dependency for such modules.
The primary source module, as well as Swift textual module dependencies which *were* built with C++ interop will continue getting a direct depedency of the 'CxxStdlib' Swift module.
Resolves rdar://150222155
In expectation, this should never happen. Such a situation means that within the same scanning action, Clang Dependency Scanner has produced two different variants of the same module. This is not supposed to happen, but we are currently hunting down the rare cases where it does, seemingly due to differences in Clang Scanner direct by-name queries and transitive header lookup queries.
Currently, the macro plugin options are included as cache key and the
absolute path of the plugin executable and library will affect cache
hit, even the plugin itself is identical.
Using the new option `-resolved-plugin-validation` flag, the macro
plugin paths are remapped just like the other paths during dependency
scanning. `swift-frontend` will unmap to its original path during the
compilation, make sure the content hasn't changed, and load the plugin.
It also hands few other corner cases for macro plugins:
* Make sure the plugin options in the swift module is prefix mapped.
* Make sure the remarks of the macro loading is not cached, as the
mesasge includes the absolute path of the plugin, and is not
cacheable.
rdar://148465899
With '-sdk-module-cache-path', Swift textual interfaces found in the SDK will be built into a separate SDK-specific module cache.
Clang modules are not yet affected by this change, pending addition of the required API.
The algorithm already performs pairwise checks on module dependencies brought into compilation per-source-file. Previously, the algorithm considered the entire sub-graph of a given source file. Actual source compiles do not consider the full transitive module dependency set for cross-import-overlay lookup, but rather only directly-imported modules in a given source file, and '@_exported import' Swift transitive dependencies.
This change adds tracking of whether a given import statement is 'exported' to the dependency scanner and then refines the cross-import overlay lookup logic to only consider transitive modules that are exported by directly-imported dependencies.
Previous implementation took the entire transitive dependency set and cross-referenced all of its members to determine which ones introduce requried cross-import overlays. That implementation differed from the cross-import overlay loading logic during source compilation, where a corrsponding cross-import overlay module is only requested if the two constituent modules are reachable via direct 'import's from *the same source file*. Meaning the dependency scanner before this change would report cross-import overlay dependencies which never got loaded by the corresponding client source compile.
This change implements a new implementation of cross-import overlay discovery which first computes sub-graphs of module dependencies directly reachable by 'import's for each source file of the module under scan and then performs pairwise cross-import overlay query per each such sub-graph.
Resolves rdar://145157171
Add ability to automatically chaining the bridging headers discovered from all
dependencies module when doing swift caching build. This will eliminate all
implicit bridging header imports from the build and make the bridging header
importing behavior much more reliable, while keep the compatibility at maximum.
For example, if the current module A depends on module B and C, and both B and
C are binary modules that uses bridging header, when building module A,
dependency scanner will construct a new header that chains three bridging
headers together with the option to build a PCH from it. This will make all
importing errors more obvious while improving the performance.
Batch dependency scanning was added as a mechanism to support multiple compilation contexts within a single module dependency graph.
The Swift compiler and the Explicitly-built modules model has long since abandoned this approach and this code has long been stale. It is time to remove it and its associated C API.
This set, belonging to 'ModuleDependenciesCache', is only updated in a critical section behind a lock in the scanner. However, it is queried unsynchronized inside the Clang scanner itself. If an update causes a re-hash to happen, chaose can ensue with concurrent lookups.
Since this set only affects the produced set of results from teh Clang scanning query, we should simply pass in an immutable copy to scanning queries and rely on downstream de-duplication of scanning results.
With this, we can avoid passing in the reference to `ModuleDependenciesCache` to the 'scanFilesystemFor*ModuleDependency' altogether.
Resolves rdar://139414443
This change refactors the top-level dependency scanning flow to follow the following procedure:
Scan():
1. From the source target under scan, query all imported module identifiers for a *Swift* module. Leave unresolved identifiers unresolved. Proceed transitively to build a *Swift* module dependency graph.
2. Take every unresolved import identifier in the graph from (1) and, assuming that it must be a Clang module, dispatch all of them to be queried in-parallel by the scanner's worker pool.
3. Resolve bridging header Clang module dpendencies
4. Resolve all Swift overlay dependencies, relying on all Clang modules collected in (2) and (3)
5. For the source target under scan, use all of the above discovered module dependencies to resolve all cross-import overlay dependencies
