Add the module-format machinery that lets a Swift library record the
physical layout of hidden types (currently limited to C types imported via internal bridging header).
into binary modules, so downstream consumers can pull the layouts of these hidden types without
loading the internal dependency.
To test this, this change also added a frontend action to print hidden types' layouts
from both the module under compilation and all the modules being imported.
Computes per-field abstract layout (offset, size, alignment, mangled name,
bitwiseCopyable, isOpaque) for structs with both Clang-imported and Swift-native fields.
This API will be used from the library-side to serialize layout information to binary modules.
function that queries Clang's ASTRecordLayout to extract size, alignment,
stride, bitwiseCopyable, and isOpaque for C-backed nominal types.
This also adds a -dump-abstract-layout frontend debugging flag that prints
layout info for Clang-backed stored property types in the primary source
file.
Fix that the `file` fields in ConstExtract JSON output are all prefix
mapped path when caching and prefix mapping are all enabled. This can
cause the consumers of the output failed to find the source file.
Now the JSON file is correct updated with the correct source file
location when caching is on. This allows both un-prefix-mapped source
location and relocatable source file location. As a bonus, now CAS
stores the compact version of the JSON file to save space without
affecting the pretty formatted final output file.
rdar://173736094
Add support for named location markers in the diagnostic verifier's
-verify mode. A marker is defined by placing `// #name` in a comment
and referenced with `@#name` in expected-diagnostic comments.
This enables stable references to diagnostic locations that don't break
when lines are added or removed, and supports cross-file references
where an expected-error in one file references a marker in another.
Compute and propagate the library level (api/spi/ipi) of each module
dependency through the dependency scanner so that the compiler can
correctly enforce private module import diagnostics in CAS mode, where
path-based SPI detection fails because CAS abstracts file paths to
content IDs.
Swift modules:
- Detect library level from the module interface path using
libraryLevelFromPath() during scanning, for both textual (.swiftinterface)
and binary (.swiftmodule) Swift modules.
Clang modules:
- Expose ModuleMapIsPrivate from clang::Module in ModuleDeps via the
dependency scanning infrastructure.
- Set library level for clang modules in bridgeClangModuleDependency()
using ModuleMapIsPrivate (catches module.private.modulemap in any
SDK location) and libraryLevelFromPath() on the module map file
(catches modules under PrivateFrameworks directories).
The library level is:
- Stored in ModuleDependencyInfo and serialized in the module dependency
cache (format version bumped to v8).
- Exposed through the swiftscan C API via a new
swiftscan_module_info_get_library_level() function (API minor version
bumped to 3).
- Emitted in the dependency scanner JSON output as "libraryLevel" for
all module kinds (Swift textual, Swift binary, Clang, and main module).
- Parsed from the explicit module map JSON by ExplicitModuleMapParser
for both Swift (ExplicitSwiftModuleInputInfo) and Clang
(ExplicitClangModuleInputInfo) modules.
- Looked up in ModuleLibraryLevelRequest via
ASTContext::getExplicitModuleLibraryLevel(name, isClang), which
consults the appropriate map (Swift or Clang) based on module kind.
rdar://172693314
Assisted-By: Claude
With the new option, when doing caching we write the hash that we already
computed for the main output file to an extended attribute (xattr) on the file.
This is equivalent to clang's -fwrite-output-hash-xattr option.
The format of the xattr is
name: com.apple.clang.cas_output_hash
data:
* Null-terminated hash schema name, e.g. llvm.builtin.v2[BLAKE3].
* Hash length (4 bytes, little-endian).
* Hash bytes
rdar://171185394
Migrate all callers of getOrCreateDatabases to either create their own CAS
instances using createDatabases() or to use the ones stored in the
CompilerInstance. Also forwards the instances into the ClangImporter and its
various clang::CompilerInstance instances.
Validated locally that this does not result in any additional calls to
createDatabases across all the swift tests. In fact, there are fewer due to an
improvement on the clange side, so I also checked that the overall reduction
doesn't hide any issues. Looking at individual cas paths that are opened, which
are per-test paths, there are none that open more than before.
Fix the debug info emitted for bridging header PCH when compilation
caching is used. This includes:
* Bridging header should have the correct dwo_name that uses CASID
* The search_path for PCH should be the cache key that can load the PCH
just like other modules.
Currently only the top level dependencies get serialized in Swift modules. In
practice this is not enough information to fully replay a module import
sequence, especially when the dependencies include binary SDK modules that were
built elsewhere. In this case we cannot follow the links to its depenencies,
since they refer to paths on a different machine or unavailable CAS. During an
EBM build, the dependency scanner writes the complete list of dependencies into
a json file called the explicit Swift module map -- including the local
locations of the dependencies of binary SDK modules. Using this LLDB can replay
a module import with 100% accuracy.
rdar://170514919
During explicit module build, teach dependency scanner to emit the
module trace file instead of each following compile job command. This
reduces the duplicated info, and allows supporting fully cached build
that only loads module from CAS thus cannot produce the path to the
original module file on disk.
rdar://170007480
Partially revert https://github.com/swiftlang/swift/pull/86309. Keep the
SDKInfo parsing in the ASTContext separately from the search path
configuration. This allows the availablity checking to load SDKSettings
from the correct VFS.
rdar://169886913
Windows paths can contain colons (etc. C:\foo\bar), which the diagnostic
verifier would confuse for the end of the path. By quoting them we can
ignore all colons until the matching single quote character.
Some Darwin platforms like DriverKit use a system prefix on all of their search paths. Even though DriverKit isn't supported, add support to get the system prefix from SDKSettings when constructing the default search paths.
This requires the DarwinSDKInfo to be gotten earlier in CompilerInvocation, pass that down to ASTContext through CompilerInstance.
-platform-availability-inheritance-map-path is no longer needed to support visionOS in tests, remove that and its supporting code that gets an alternative DarwinSDKInfo.
rdar://166277280
This change adds collection of three metrics to the scanner:
- number of Swift module lookups
- number of named Clang module lookups
- recorded number of Clang modules which were imported into a Swift module by name
It introduces '-Rdependency-scan', which acts as a super-set flag to the existing '-Rdependency-scan-cache' and adds emission of the above metrics as remarks when this flag is enabled. Followup changes will add further remarks about dependency scanner progress.
This virtual function is called when parsing the control block of a Swift module
to add the dependencies to the module loader's explicit Swift module map. For
the compiler this has no practical effect since the explicit Swift module map
should already cover all dependencies. LLDB doesn't have the explicit Swift
module map and depends on this mechanism to discover dependencies.
rdar://166494766
LLDB needs to control whether implicit Swift modules are on or off within a
single ASTContext:
- when importing the context of a Swift module in an EBM
project only explicit modules should be allowed
- when evaluating arbitrary expressions (which may include `import` directives)
implicit modules should be enabled
rdar://166494895
For clients, such as the debugger, who do not have access the full
output of the dependency scanner, it is a huger performance and
correctness improvement if each explicitly built Swift module not just
serialized all its Clang .pcm dependencies (via the serialized Clang
compiler invocation) but also its direct Swift module dependencies.
This patch changes the Swift module format to store the absolute path
or cas cache key for each dependency in the INPUT block, and makes
sure the deserialization makes these available to the ESML.
rdar://150969755
The _SwiftifyImport macro is emitted into an unnamed buffer and then
parsed, pretending it was in the header all along. This makes it hard to
add `expected-note` comments for `diag::in_macro_expansion` when they
point here. That's okay, because the macro expansion has already been
pointed out by `expected-expansion` directives. But
-verify-ignore-unrelated is too blunt of a tool, so this adds
-verify-ignore-macro-note to ignore these specific diagnostics.
We already have -suppress-warnings and -suppress-remarks; this patch
adds support for suppressing notes too. Doing so is useful for -verify
tests where we don't really care about the emitted notes.
This adds the implementation required for later changing the default
behaviour of the -verify flag to error when diagnostics are emitted
in buffers other than the main file and files added with
-verify-additional-file. To keep the current behaviour, use the flag
-verify-ignore-unrelated. This flag is added as a no-op so that tests
can start using it before the new behaviour is enabled by default.
Introduce the ability to form a `StaticBuildConfiguration` from
language options. Add a frontend option `-print-static-build-config`
to then print that static build configuration as JSON in a manner that
can be decoded into a `StaticBuildConfiguration`.
Most of the change here is in sinking the bridged ASTContext queries
of language options into a new BridgedLangOptions. The printing of the
static build configuration only has a LangOptions (not an ASTContext),
so this refactoring is required for printing.
This change refactors the module loaders to explicitly take a parameter indicating whether or not the loader is handling a 'canImport' query, in order to avoid emitting an error when finding a dependency Swift binary module with only imcompatible architecture variants present.
Resolves rdar://161175498
Replace the one-off compiler flag for Library Evolution with an
optional language feature. This makes the
`hasFeature(LibraryEvolution)` check work in an `#if`, and is
otherwise just cleanup.
Tracked by rdar://161125572.
The flags "-import-bridging-header" and "-import-pch" import a bridging
header, treating the contents as a public import. Introduce
"internal-" variants of both flags that provide the same semantics,
but are intended to treat the imported contents as if they came in
through an internal import. This is just plumbing of the options for
the moment.
Since freestanding macro expansion buffer names include the line number
of their invocation, it can become quite fiddly to try to update a test
file with multiple macro expansions. This adds the option to use an
expected-expansion block and use a relative line number, nesting other
expected diagnostic statements inside this block.
Example syntax:
```swift
let myVar = #myMacro
/*
expected-expansion@-2:1{{
expected-error@13:37{{I can't believe you've done this}}
expected-note@14:38{{look at this and ponder your mistake}}
}}
*/
```
An always enabled availability domain is implicitly available in all contexts,
so uses of declarations that are marked as `@available` in the domain are never
rejected. This is useful for an availability domain representing a feature flag
that has become permanently enabled.
Partially resolves rdar://157593409.
Argyrios Kyrtzidis
Argyrios Kyrtzidis
Xi Ge
Steven Wu
Doug Gregor
Alexis Laferrière
Henrik G. Olsson
Desel72
Ben Langmuir
Adrian Prantl
Henrik G. Olsson
Ian Anderson
Artem Chikin
Becca Royal-Gordon
Gabor Horvath
John Hui
Erik Eckstein
Hamish Knight
Allan Shortlidge