This refines how bridging header chaining is done, mainly fixing two
problems:
* On Windows, `#import` statement is not supported. Instead of relying
on #import to workaround bridging header missing header guard, do a
simple deduplication in header generation.
* The __has_include check can failed the dependency scanner if the search
path hits `permission_denied` error. In that case, we cannot rely on
the clang dependeny scanner to check if the bridging header exists or
not. Swift dependeny scanner need to know how to reverse mapping and
check before generating the header.
rdar://175196897
When CachedDiagnostics emits diagnostics for the first time (cache miss),
the file path shown was the internal prefix-mapped path instead of the real
on-disk path.
Apply remapFilePath when creating the buffer so first-time emission matches
the replay path.
rdar://175263981
For a given function, we might end up emitting it's definition as
object code, serialized SIL, or both. The @export, @inlinable, and
@inline(always) attributes provide control of this behavior at the
declaration level.
Centralize the query function that will look for each of these
attributes and map down to a specific "code generation model", whose 3
options follow the naming from SE-0497: interface, inlinable, and
implementation. Use this one computation to back the queries for
"always emit into client", "never emit into client", and "inlinable"
so we can't get inconsistent results from places that are doing
one-off checks for these attributes.
The swiftdeps output path is marked CacheInvariant and should not appear
in output content. Use the source file path (SF->getFilename()) instead
of the output path when constructing sourceFileProvide dependency nodes.
Add a LIBRARY_LEVEL record to the .swiftmodule options block so the
declared -library-level value survives across compilations. Without
this, imported modules have to always fell back to a path heuristic
that could only distinguish API vs SPI and never returned IPI.
rdar://174255626
For swift binary module deps, it's cache key and output are setup during
dependency scanning time. When loading the module dependency from the
incremental cache, need to validate the CAS key and output for swift
binary module both exist in the CAS, otherwise, it needs to be
invalidated so a rescan will re-ingest the binary modules into the CAS.
rdar://173647646
Fix that the `file` fields in ConstExtract JSON output are all prefix
mapped path when caching and prefix mapping are all enabled. This can
cause the consumers of the output failed to find the source file.
Now the JSON file is correct updated with the correct source file
location when caching is on. This allows both un-prefix-mapped source
location and relocatable source file location. As a bonus, now CAS
stores the compact version of the JSON file to save space without
affecting the pretty formatted final output file.
rdar://173736094
Make sure the CAS instances are not mixed during dependency scanning,
especially when used from libSwiftScan C APIs.
For clang scanning service and file system, it should always be created
from the global CAS instance.
For CAS instance inside the swift instance when performing dependency
scanning, it reuse the CAS instance from global as well if that is
already initialized.
rdar://173703843
This is a host property, not a target property. When cross-testing a
Unix-like environment on Windows, we find that we load the plugin from
the wrong location. Correct the definition to use the toolchain host as
the basis for the decision.
Improve bridging header chaining when prefix mapping is used so it
matches the behavior of non-prefix-map and non-caching builds.
This also fixes a corner case where the same bridging header is used
by different modules in the dependency chain, preventing it from being
imported twice.
The improvements are:
* Fully utilize the clang scanner prefix mapping option, which can
restore prefix-mapped paths during scanning to find the real file on
the file system. This allows the generated bridging header to always
reference the header path without worrying about introducing path
dependencies.
* Move the header existence check from a file system access in the
Swift scanner to a `__has_include` check in the clang scanner. This
allows direct header import even when the header path is previously
prefix mapped.
* Use `#import` to chain bridging headers so the same header will not
be imported twice.
rdar://172870182
In order to achieve sound caching, plugin loader double checks the
plugin loaded is the same as what dependency scanner sees during
compilation. However, user can pass a non-existing macro plugin to the
compiler and that should trigger a warning during compilation. Improve
the plugin verification to handle the case where the plugin is passed
but missing on the file system.
rdar://172930632
Don't diff `stat` output, which might be different due to last access
time. Using `getmtime` script to compare mtime only to make sure the
output is not updated.
rdar://172853933
Make sure cache replay matches the same behavior as normal compilation
that the output is not written (timestamp preserved) when replaying
cache hits. This make sure the downstream dependencies are not
invalidated on a cache hit but the output doesn't change.
Compute and propagate the library level (api/spi/ipi) of each module
dependency through the dependency scanner so that the compiler can
correctly enforce private module import diagnostics in CAS mode, where
path-based SPI detection fails because CAS abstracts file paths to
content IDs.
Swift modules:
- Detect library level from the module interface path using
libraryLevelFromPath() during scanning, for both textual (.swiftinterface)
and binary (.swiftmodule) Swift modules.
Clang modules:
- Expose ModuleMapIsPrivate from clang::Module in ModuleDeps via the
dependency scanning infrastructure.
- Set library level for clang modules in bridgeClangModuleDependency()
using ModuleMapIsPrivate (catches module.private.modulemap in any
SDK location) and libraryLevelFromPath() on the module map file
(catches modules under PrivateFrameworks directories).
The library level is:
- Stored in ModuleDependencyInfo and serialized in the module dependency
cache (format version bumped to v8).
- Exposed through the swiftscan C API via a new
swiftscan_module_info_get_library_level() function (API minor version
bumped to 3).
- Emitted in the dependency scanner JSON output as "libraryLevel" for
all module kinds (Swift textual, Swift binary, Clang, and main module).
- Parsed from the explicit module map JSON by ExplicitModuleMapParser
for both Swift (ExplicitSwiftModuleInputInfo) and Clang
(ExplicitClangModuleInputInfo) modules.
- Looked up in ModuleLibraryLevelRequest via
ASTContext::getExplicitModuleLibraryLevel(name, isClang), which
consults the appropriate map (Swift or Clang) based on module kind.
rdar://172693314
Assisted-By: Claude
With the new option, when doing caching we write the hash that we already
computed for the main output file to an extended attribute (xattr) on the file.
This is equivalent to clang's -fwrite-output-hash-xattr option.
The format of the xattr is
name: com.apple.clang.cas_output_hash
data:
* Null-terminated hash schema name, e.g. llvm.builtin.v2[BLAKE3].
* Hash length (4 bytes, little-endian).
* Hash bytes
rdar://171185394
Fix the debug info emitted for bridging header PCH when compilation
caching is used. This includes:
* Bridging header should have the correct dwo_name that uses CASID
* The search_path for PCH should be the cache key that can load the PCH
just like other modules.
Fix a module build regression in PR86881 that causes an increase of
swift interface compilation tasks due to including the
const-gather-protocol list files into the build dependencies for
swiftinterface files. This causes targets cannot sharing module
dependencies.
This make sures the CASFS system only captures the const-gather-protocol
list in the tasks for main module, not for any of its module
dependencies.
rdar://171506799
Add flag `-debug-module-cache-key` and flag `-debug-module-self-key` to
support embedded swift module output in the debug info via CASID.
rdar://169110002
Do not search for cross import when building swiftinterfaces. This is
uncessary because:
* The cross import modules are explicitly listed in the interface file
thus there is no need to walk the file system to discover more
cross imports.
* All cross imports are discovered by scanner already and the
dependencies are passed to the interface building job. There is no
need to search file system to find the module to consume.
This reduces the work needed when building interface and fix a bug that
the file system walk when caching is enabled can cause build failures on
windows platform.
As specified in SE-0441, -language-mode should appear in help output
while -swift-version should be the hidden alias.
Related to swiftlang/swift-driver#1894
During explicit module build, teach dependency scanner to emit the
module trace file instead of each following compile job command. This
reduces the duplicated info, and allows supporting fully cached build
that only loads module from CAS thus cannot produce the path to the
original module file on disk.
rdar://170007480
Replaces the other ExperimentalLanguageFeature (non-plural) SPI usages with the plural version. Also removes the `@_spi` flag from BodyMacroWithControlFlow as body macros are no longer an experimental feature. This is liked to a PR on swift-syntax: https://github.com/swiftlang/swift-syntax/pull/3272
Partially revert https://github.com/swiftlang/swift/pull/86309. Keep the
SDKInfo parsing in the ASTContext separately from the search path
configuration. This allows the availablity checking to load SDKSettings
from the correct VFS.
rdar://169886913
Previously, const-values output is not supported correctly as it is not
captured correctly as a dependency. The load of the input JSON file is
loaded via file system directly, that can cause issues like:
* False cache hit when the file is changed
* Cannot be prefix mapped to allow distributed caching
Try to support the input file by:
* Correctly capture the input
* Create a new driver flag `-const-gather-protocols-list` that can do
path remapping correctly in swift-driver
rdar://169109358
Every time DarwinSDKInfo reads a new key out of SDKSettings, a boatload of test SDKSettings files need to be updated across several repositories and forks and branches. It’s tedious to be careful to update those with real values so that the tests are properly regression testing older SDKs. It’s important to be careful so that the tests are accurate, e.g. to prevent the scenario where DarwinSDKInfo starts reading a new key out of SDKSettings and assumes that it’s always available everywhere, when in reality it was only added a few releases ago and will break with older SDKs. If the test SDKSettings files continue to be updated ad hoc, it’s going to be really easy to copy/paste a default value everywhere, and then clients will see incorrect behaviors with the real SDKs, or even compiler crashes if the key is unconditionally read. Preemptively add all of the maybe-possibly-compiler relevant keys to the test SDKSettings files from the real SDKs so that the test files are an accurate representation and shouldn't need to be touched in the future. Where the test SDKSettings have intentionally doctored data, add a Comments key explaining what is changed from the real SDK, and alter the SDK name with a tag indicating the change.
rdar://168700857
Fix a corner case when following conditions are met in a single module:
* Module A is imported in the current module
* There exists a versioned `canImport` check for a module A
* All `canImport` check within the module evaluated to false
If all above conditions are met, swift-frontend will not received the
version number from scanner, and it will be confused if version is
either not available (thus a warning, and treat as true), or condition
is not met (eval to false).
Now make sure if when scanner tries to resolve `canImport`, it will
record all instances that a module can be imported, or a version of the
module is found.
rdar://167784812
clang will start looking at CanonicalName and SupportedTargets soon, and error if those aren't as expected in the shipping SDKs. Pre-emptively add those to the test SDKSettings.
rdar://166277280
Fix a programming mistake when chaining bridging header for the main
module. Main module never has the binary module, thus it always need to
chain the bridging header content directly without falling back to
embedded binary module. Previously, the scanner was wrongly error out
because it failed to locate the binary module for main module.
rdar://167707148
When using compilation caching, make sure the debug info emitted in the
swift TU references PCM files via CASID instead of file path. This
allows dsymutil and lldb to load clang modules correctly from CAS,
instead of relying on file system to load the file.
rdar://167054494
This change adds collection of three metrics to the scanner:
- number of Swift module lookups
- number of named Clang module lookups
- recorded number of Clang modules which were imported into a Swift module by name
It introduces '-Rdependency-scan', which acts as a super-set flag to the existing '-Rdependency-scan-cache' and adds emission of the above metrics as remarks when this flag is enabled. Followup changes will add further remarks about dependency scanner progress.
Argyrios Kyrtzidis
Steven Wu
Doug Gregor
Saleem Abdulrasool
Sam Pyankov
Argyrios Kyrtzidis
Ben Langmuir
Adrian Prantl
Ryan Mansfield
Hunter Baker
Ian Anderson
Artem Chikin