`-Xcc` flags are internal to the C compiler; we have `-vfsoverlay` now
which directly passes the VFS overlay. Because the VFS overlay flags are
automatically passed along to the clang scanner, we need to pass along
`-vfsoverlay` to the Swift frontend to ensure that any VFS overlays are
honoured. Most likely this was a workaround for `-vfsoverlay` not
getting propagated originally and when the restriction was lifted the
exercised codepath was not heavily tested.
When improving the speed of dependency scanning when the new clang API
to speed up bridging, some unintended change was introduced. This
restore the scanner to the behavior before #81454
rdar://153851818
While this made sense in the distant past where the scanning service provided backing storage for the dependency cache, it no longer does so and now makes for awkard layering where clients get at the service via the cache. Now the cache is a simple data structure while all the clients that need access to the scanning service will get it explicitly.
- 'SwiftModuleScanner' will now be owned directly by the 'ModuleDependencyScanningWorker' and will contain all the necessary custom logic, instead of being instantiated by the module interface loader for each query
- Moves ownership over module output path and sdk module output path directly into the scanning worker, instead of the cache
Adds an access control field for each imported module identified. When multiple imports of the same module are found, this keeps track of the most "open" access specifier.
Use the underlying compiler invocation inside module scanning result to
speed up the clang module dependency bridging. This avoids converting
cc1 arguments to compiler invocation and back, just to modify the cc1
arguments needed for building PCM using swift-frontend.
rdar://151705822
[Dependency Scanning] Update Uses of `ModuleDeps::ClangModuleDeps`
https://github.com/llvm/llvm-project/pull/137421 changes the type of `ModuleDeps::ClangModuleDeps`. This PR updates Swift's use of this data structure to use the correct fields.
rdar://144794793
In expectation, this should never happen. Such a situation means that within the same scanning action, Clang Dependency Scanner has produced two different variants of the same module. This is not supposed to happen, but we are currently hunting down the rare cases where it does, seemingly due to differences in Clang Scanner direct by-name queries and transitive header lookup queries.
With '-sdk-module-cache-path', Swift textual interfaces found in the SDK will be built into a separate SDK-specific module cache.
Clang modules are not yet affected by this change, pending addition of the required API.
The algorithm already performs pairwise checks on module dependencies brought into compilation per-source-file. Previously, the algorithm considered the entire sub-graph of a given source file. Actual source compiles do not consider the full transitive module dependency set for cross-import-overlay lookup, but rather only directly-imported modules in a given source file, and '@_exported import' Swift transitive dependencies.
This change adds tracking of whether a given import statement is 'exported' to the dependency scanner and then refines the cross-import overlay lookup logic to only consider transitive modules that are exported by directly-imported dependencies.
Add ability to automatically chaining the bridging headers discovered from all
dependencies module when doing swift caching build. This will eliminate all
implicit bridging header imports from the build and make the bridging header
importing behavior much more reliable, while keep the compatibility at maximum.
For example, if the current module A depends on module B and C, and both B and
C are binary modules that uses bridging header, when building module A,
dependency scanner will construct a new header that chains three bridging
headers together with the option to build a PCH from it. This will make all
importing errors more obvious while improving the performance.
When Swift passes search paths to clang, it does so directly into the HeaderSearch. That means that those paths get ordered inconsistently compared to the equivalent clang flag, and causes inconsistencies when building clang modules with clang and with Swift. Instead of touching the HeaderSearch directly, pass Swift search paths as driver flags, just do them after the -Xcc ones.
Swift doesn't have a way to pass a search path to clang as -isystem, only as -I which usually isn't the right flag. Add an -Isystem Swift flag so that those paths can be passed to clang as -isystem.
rdar://93951328
Previous behavior had the scanner simply proceed if the header input of a binary Swift module dependency could not be resolved on the filesystem
Resolves rdar://139736789
This change refactors the top-level dependency scanning flow to follow the following procedure:
Scan():
1. From the source target under scan, query all imported module identifiers for a *Swift* module. Leave unresolved identifiers unresolved. Proceed transitively to build a *Swift* module dependency graph.
2. Take every unresolved import identifier in the graph from (1) and, assuming that it must be a Clang module, dispatch all of them to be queried in-parallel by the scanner's worker pool.
3. Resolve bridging header Clang module dpendencies
4. Resolve all Swift overlay dependencies, relying on all Clang modules collected in (2) and (3)
5. For the source target under scan, use all of the above discovered module dependencies to resolve all cross-import overlay dependencies
Use IncludeTreeFileList instead of full feature CASFS for swift
dependency filesystem. This allows smaller CAS based VFS that is smaller
and faster. This is enabled by the CAS enabled compilation does not
need to iterate file system.
rdar://136787368
Use `-fsyntax-only` action to scan clang module dependencies instead of
`-c` option. This fixes a non-deterministic output on windows from
scan-dependency output because `-c` implies it needs a temporary object
file in the cc1 arguments that makes the pcm compilation command
different every run. This can also make the `-Xcc` commands for PCM
compilation simpler and more likely to be deduplicated by build system.
rdar://135319536
Avoid constructing tasks with `-Xcc` options that references clang
CASOptions. This is going to make the cache hit/miss to be dependent on
CAS location/configuraitons.
Instead, only give swift CASOptions to constructed tasks and propagate
the configurations to underlying clang importer.
rdar://132255889
Although I don't plan to bring over new assertions wholesale
into the current qualification branch, it's entirely possible
that various minor changes in main will use the new assertions;
having this basic support in the release branch will simplify that.
(This is why I'm adding the includes as a separate pass from
rewriting the individual assertions)
There is no need to pass output path of the PCM compilation twice, once
as swift `-o` flag and other time as `-Xcc -o` flag to clang importer.
This can also cause swift caching to miss when output path is different
because `-Xcc` options are not modeled in caching model to understand
`-o` output path doesn't affect compilation output content.
rdar://128650954
This fixes -gmodules when caching by adopting the new clang -cc1 option
-finclude-tree-preserve-pch-path. -gmodules is required to make
debugging work when examining types that come from clang modules or
bridging headers, but was previously being disabled by clang's caching
support.
rdar://126370706
Support `-vfsoverlay` swift option for explicit module build (including
caching build). Previously, if the interface file is discovered from a
location that is remapped by overlay, module cannot be built correctly.
Make sure the overlay options are passed down to all interface
compilaiton command.
For caching build, need to make sure the overlay itself is part of the CAS
file system so the downstream compilation can discover that.
rdar://123655183
LLVM is presumably moving towards `std::string_view` -
`StringRef::startswith` is deprecated on tip. `SmallString::startswith`
was just renamed there (maybe with some small deprecation inbetween, but
if so, we've missed it).
The `SmallString::startswith` references were moved to
`.str().starts_with()`, rather than adding the `starts_with` on
`stable/20230725` as we only had a few of them. Open to switching that
over if anyone feels strongly though.
Otherwise they may have module dependencies of their own which will not be detected by the scanner and included in the list of explicit inputs for compilation.
When prefix mapping paths that are used in clang, ensure we are
consistently using the same prefix mapper from clang. This prevents
mismatches that could cause modules to fail to load.
rdar://123324072
