Improve caching related tests to make them faster and test the situation
closer to what actually happens during explicit module build. Most
noticable changes are:
* Avoid swift stdlib dependency during the tests to save time
* Use dependency scanner output to construct test cases
* Update old test cases that try to simulate caching from file system
inputs to using only CAS inputs.
Improve tests to call python script through python executable. Even this
doesn't cause any errors now, it will be a problem for future when we
run CAS tests on Windows platform. Clean up the current test to avoid
bad test example.
Redirecting file system can canonicalize the file path before forwarding
the path to IncludeTreeFileSystem, which is a simplied FS that can only
intepret the paths that has been seen by dep-scanner. Since all files
that need redirecting already added to underlying FS via DepScan, there
is no need for such layer when compiling using clang-include-tree.
rdar://119727344
When caching + clang include tree is enabled, don't take module map file
from command-line in clang importer. Those are resulted from `-Xcc`
arguments and do not needed in compilation since module maps are
included in include-tree.
rdar://119577349
Teach swift-frontend to correctly replay cache when using
batch-mode/primary-file. Currently, the cache replay logic will try to
look up cache key for all the inputs and only replay if all lookups are
successful. In batch-mode, not all inputs are producing outputs, which
causes lookup for those keys to be a miss and causing a false miss for
the compilation.
Update swift cache key computation mechanism from one cache key per
output, to one cache key per primary input file (for all outputs that
associated with that input).
The new schema allows fewer cache lookups while still preserving most of
the flexibility for batch mode and incremental mode.
-typecheck-module-from-interface doesn't emit textual output as
configured in frontend option. The real output is diagnostics but that
is not tracked by output computer which is using the option.
Teach `swift-frontend` that it doesn't actually emit any real output and
tell serialization not to emit swiftmodule.
Allow `-typecheck-module-from-interface` using explicit module instead
of building implicit module.
This setups swift-frontend to accept explicit module build arguments and
loading explicit module during verifying. SwiftDriver needs to setup
correct arguments including the output path for swift module to fully
enable explicit module interface check.
The code of `ScanDependencies.cpp` was creating invalid JSON since #66031
because in the case of having `extraPcmArgs` and `swiftOverlayDependencies`,
but not `bridgingHeader`, a comma will not be added at the end of
`extraPcmArgs`, creating an invalid JSON file. Additionally that same PR
added a trailing comma at the end of the `swiftOverlayDependencies`, which
valid JSON does not allow, but that bug was removed in #66366.
Both problems are, however, present in the 5.9 branch, because #66936
included #66031, but not #66366.
Besides fixing the problem in `ScanDependencies.cpp` I modified every test
that uses `--scan-dependencies` to pass the produced JSON through
Python's `json.tool` in order to validate proper JSON is produced. In
most cases I was able to pipe the output of the tool into `FileCheck`,
but in some cases the validation is done by itself because the checks
depend on the exact format generated by `--scan-dependencies`. In
a couple of tests I added a call to `FileCheck` that seemed to be
missing.
Without these changes, two tests seems to be generating invalid JSON in
my machine:
- `ScanDependencies/local_cache_consistency.swift` (which outputs `Expecting ',' delimiter: line 525 column 11 (char 22799)`)
- `ScanDependencies/placholder_overlay_deps.swift`
Make sure the sources for bridging header is added as part of the CAS
filesystem for the main module. Even bridging header should be compiled
into PCH, PCHs are not standalone that it can be used without source
file.
Rename `-enable-cas` to `-compile-cache-job` to align with clang option
names and promote that to a new driver only flag.
Few other additions to driver flag for caching behaviors:
* `-compile-cache-remarks`: now cache hit/miss remarks are guarded behind
this flag
* `-compile-cache-skip`: skip replaying from the cache. Useful as a
debugging tool to do the compilation using CAS inputs even the output
is a hit from the cache.
Teach swift dependency scanner to use CAS to capture the full dependencies for a build and construct build commands with immutable inputs from CAS.
This allows swift compilation caching using CAS.
