With '-sdk-module-cache-path', Swift textual interfaces found in the SDK will be built into a separate SDK-specific module cache.
Clang modules are not yet affected by this change, pending addition of the required API.
The algorithm already performs pairwise checks on module dependencies brought into compilation per-source-file. Previously, the algorithm considered the entire sub-graph of a given source file. Actual source compiles do not consider the full transitive module dependency set for cross-import-overlay lookup, but rather only directly-imported modules in a given source file, and '@_exported import' Swift transitive dependencies.
This change adds tracking of whether a given import statement is 'exported' to the dependency scanner and then refines the cross-import overlay lookup logic to only consider transitive modules that are exported by directly-imported dependencies.
Add ability to automatically chaining the bridging headers discovered from all
dependencies module when doing swift caching build. This will eliminate all
implicit bridging header imports from the build and make the bridging header
importing behavior much more reliable, while keep the compatibility at maximum.
For example, if the current module A depends on module B and C, and both B and
C are binary modules that uses bridging header, when building module A,
dependency scanner will construct a new header that chains three bridging
headers together with the option to build a PCH from it. This will make all
importing errors more obvious while improving the performance.
When Swift passes search paths to clang, it does so directly into the HeaderSearch. That means that those paths get ordered inconsistently compared to the equivalent clang flag, and causes inconsistencies when building clang modules with clang and with Swift. Instead of touching the HeaderSearch directly, pass Swift search paths as driver flags, just do them after the -Xcc ones.
Swift doesn't have a way to pass a search path to clang as -isystem, only as -I which usually isn't the right flag. Add an -Isystem Swift flag so that those paths can be passed to clang as -isystem.
rdar://93951328
Previous behavior had the scanner simply proceed if the header input of a binary Swift module dependency could not be resolved on the filesystem
Resolves rdar://139736789
This change refactors the top-level dependency scanning flow to follow the following procedure:
Scan():
1. From the source target under scan, query all imported module identifiers for a *Swift* module. Leave unresolved identifiers unresolved. Proceed transitively to build a *Swift* module dependency graph.
2. Take every unresolved import identifier in the graph from (1) and, assuming that it must be a Clang module, dispatch all of them to be queried in-parallel by the scanner's worker pool.
3. Resolve bridging header Clang module dpendencies
4. Resolve all Swift overlay dependencies, relying on all Clang modules collected in (2) and (3)
5. For the source target under scan, use all of the above discovered module dependencies to resolve all cross-import overlay dependencies
Use IncludeTreeFileList instead of full feature CASFS for swift
dependency filesystem. This allows smaller CAS based VFS that is smaller
and faster. This is enabled by the CAS enabled compilation does not
need to iterate file system.
rdar://136787368
Use `-fsyntax-only` action to scan clang module dependencies instead of
`-c` option. This fixes a non-deterministic output on windows from
scan-dependency output because `-c` implies it needs a temporary object
file in the cc1 arguments that makes the pcm compilation command
different every run. This can also make the `-Xcc` commands for PCM
compilation simpler and more likely to be deduplicated by build system.
rdar://135319536
Avoid constructing tasks with `-Xcc` options that references clang
CASOptions. This is going to make the cache hit/miss to be dependent on
CAS location/configuraitons.
Instead, only give swift CASOptions to constructed tasks and propagate
the configurations to underlying clang importer.
rdar://132255889
Although I don't plan to bring over new assertions wholesale
into the current qualification branch, it's entirely possible
that various minor changes in main will use the new assertions;
having this basic support in the release branch will simplify that.
(This is why I'm adding the includes as a separate pass from
rewriting the individual assertions)
There is no need to pass output path of the PCM compilation twice, once
as swift `-o` flag and other time as `-Xcc -o` flag to clang importer.
This can also cause swift caching to miss when output path is different
because `-Xcc` options are not modeled in caching model to understand
`-o` output path doesn't affect compilation output content.
rdar://128650954
This fixes -gmodules when caching by adopting the new clang -cc1 option
-finclude-tree-preserve-pch-path. -gmodules is required to make
debugging work when examining types that come from clang modules or
bridging headers, but was previously being disabled by clang's caching
support.
rdar://126370706
Support `-vfsoverlay` swift option for explicit module build (including
caching build). Previously, if the interface file is discovered from a
location that is remapped by overlay, module cannot be built correctly.
Make sure the overlay options are passed down to all interface
compilaiton command.
For caching build, need to make sure the overlay itself is part of the CAS
file system so the downstream compilation can discover that.
rdar://123655183
LLVM is presumably moving towards `std::string_view` -
`StringRef::startswith` is deprecated on tip. `SmallString::startswith`
was just renamed there (maybe with some small deprecation inbetween, but
if so, we've missed it).
The `SmallString::startswith` references were moved to
`.str().starts_with()`, rather than adding the `starts_with` on
`stable/20230725` as we only had a few of them. Open to switching that
over if anyone feels strongly though.
Otherwise they may have module dependencies of their own which will not be detected by the scanner and included in the list of explicit inputs for compilation.
When prefix mapping paths that are used in clang, ensure we are
consistently using the same prefix mapper from clang. This prevents
mismatches that could cause modules to fail to load.
rdar://123324072
Fix the bridging header dependencies calculation for explicit module
build, especially for caching which needs an accurate list of deps for
compute cache key correctly.
Previously, the bridging header deps are computed from `ModuleGraph`
from the clang dependency scanner, which can be affected by already seen
modules. It causes the dependencies to be missing for bridging header if
the module is seen by main swift source module.
Now report only the directly module dependencies from bridging header,
then compute all the transitive dependencies before calculating all the
cache keys.
rdar://123156636
Currently, `-direct-clang-cc1-module-build` and `-only-use-extra-clang-opts`
have to be passed together for clang importer creation to succeed.
Missing either will result in error. Simplified the swift-frontend flags
by removing `-only-use-extra-clang-opts` and let
`-direct-clang-cc1-module-build` to do both.
Re-write and clean up how clang-importer is created from clang
arguments. Previously, it is unclear if `getClangArguments` will return
CC1 args or driver args and the logic is unnecessarily compilicated when
creating clang invocation. Now clang invocation is always created from
cc1 arguments, which can be directly provided via direct-cc1-mode or
converted from driver args.
There is no functional changes in this patch, other than
`-dump-clang-diagnostics` now will always print cc1 args, and also
driver args if that is applicable.
Switch to use clang-include-tree by default for clang module
building/caching when using a CAS. This is the default mode for clang
module and has less issues than CAS file system based implementation.
Allow DependencyScanner to canonicalize path using a prefix map. When
option `-scanner-prefix-map` option is used, dependency scanner will
remap all the input paths in following:
* all the paths in the CAS file system or clang include tree
* all the paths related to input on the command-line returned by scanner
This allows all the input paths to be canonicalized so cache key can be
computed reguardless of the exact on disk path.
The sourceFile field is not remapped so build system can track the exact
file as on the local file system.
From being a scattered collection of 'static' methods in ScanDependencies.cpp
and member methods of ASTContext. This makes 'ScanDependencies.cpp' much easier
to read, and abstracts the actual scanning logic away to a place with common
state which will make it easier to reason about in the future.
Instead of the code querying the compiler's built-in Clang instance, refactor the
dependency scanner to explicitly keep track of module output path. It is still
set according to '-module-cache-path' as it has been prior to this change, but
now the scanner can use a different module cache for scanning PCMs, as specified
with '-clang-scanner-module-cache-path', without affecting module output path.
Resolves rdar://113222853
