Use it to avoid unnecessary re-queries of Swift dependencies during a given scanning action.
Refactor 'hasSwiftDependency' API into 'hasQueriedSwiftDependency' to instead report whether or not the cache has recorded a prior lookup of a given Swift dependency.
Replace C++ implementation of swift_beginAccess and swift_endAccess with (almost) pure Swift implementation. Helpers remain in C++ for TLS, getting return addresses, and raising a fatal error on violations.
This change also moves the exclusivity access set head from the shared SwiftTLSContext structure to a dedicated TLS key. This improves performance, which is important for exclusivity checking. This is particularly the case where we can inline TLS access with a constant key, as on Darwin ARM64.
The code that bridges exclusivity tracking into Concurrency remains in C++. The new Swift implementation exposes a few helpers for it to use as a replacement for directly manipulating the C++ implementation.
rdar://161122309
This reverts commit e60ae24052 and fix
non-deterministic failures introduced by the commit.
Fix two issues when attempting to testing parallel scanning using
`swift-scan-test` tools:
* Make sure the BumpPtrAllocator in ScanningService is thread-safe so
there are no race condition when a new slab is allocated.
* Make sure the output of `swift-scan-test` only written from one
thread. This prevents some race conditions when writing to the same
raw_fd_ostream.
rdar://167760262
Currently, dependency scanner is not reporting the redirecting files
that are baked inside swift-frontend for platform support. This causes
dependency scanner returns virtual path for those files, and
swift-driver/build-system will not be able to correct validate the files
on incremental build, causing incremental build to be almost clean
builds.
This behavior issue is caused by the dependency scanning file system
layer inside clang dependency scanner that caches stats. If the
redirecting files are created underneath the layer, the real path is
lost. This fixes the issue by moving the redirecting files above the
caching layer using `-ivfsoverlay` option.
In addition to that, this commit also unifies how clang importer and
clang dependency scanner initiate the VFS, making the logic much
simpler.
Dependency Scanning is recursive over discovered Swift module overlays and cross-import overlays. In the main 'resolveImportedModuleDependencies' routine, after all Swift and Clang dependencies of the main module are discovered, the scanner looks up Swift overlays for discovered Clang modules. For each such Swift overlay, the scanner will then proceed to call 'resolveImportedModuleDependencies' on the overlay module.
On these subsequent recursive calls to 'resolveImportedModuleDependencies', Clang dependency resolution will re-query all imports of the overlay module which do not resolve to Swift modules, even if they had been queried previously in a parent invocation. This change adds the ability to re-use previously-queried-and-cached Clang module dependency information by having the dependency cache also store the set of visible modules which resulted from each by-name lookup.
This change adds collection of three metrics to the scanner:
- number of Swift module lookups
- number of named Clang module lookups
- recorded number of Clang modules which were imported into a Swift module by name
It introduces '-Rdependency-scan', which acts as a super-set flag to the existing '-Rdependency-scan-cache' and adds emission of the above metrics as remarks when this flag is enabled. Followup changes will add further remarks about dependency scanner progress.
This tweaks the existing logic that disables the import of the C++
math functions to avoid ambiguity with the C math functions by
handling the case where the module directory can be null in explicit
module builds.
For https://github.com/swiftlang/swift/issues/85606
We currently support static linking swiftCxxStdlib only on Windows. When
C++ interop is enabled but swiftCxxStdlib isn't actually used or only
when its underlying std module is used, we incorrectly emitted the
dynamic version of the lib name and caused the link error. This change
fixes it by always adding it to the dependency.
Issue https://github.com/swiftlang/swift/issues/85876
Add a script to build all dependencies for explicit module build in one
step. This reduces the boiling plate code needed to write a test for
explicit module build and allows the test to be more cross platform and
resilient to module dependencies changes.
Explicit module builds currently fail on Windows because
direct-clang-cc1-module-build emit-pcm commands take overlaid system
module map files as inputs but miss the clang VFS overlay. This change
adds the overlay and fixes explicit module builds on Windows.
For clients, such as the debugger, who do not have access the full
output of the dependency scanner, it is a huger performance and
correctness improvement if each explicitly built Swift module not just
serialized all its Clang .pcm dependencies (via the serialized Clang
compiler invocation) but also its direct Swift module dependencies.
This patch changes the Swift module format to store the absolute path
or cas cache key for each dependency in the INPUT block, and makes
sure the deserialization makes these available to the ESML.
rdar://150969755
We have long switched to delegate the checking of whether a module is up-to-date to the build system (SwiftDriver) and stopped serializing file dependencies for interface-built binary modules.
Resolves rdar://162881032
We have adopters who are relying on directly importing the underlying Clang module in the presence of incompatible Swift modules.
Resolves rdar://162549210
This one `CHECK-NEXT: cxx-overlay-source-lookup.swift` used to match
because the expansion of the `%t` temp dir contained the substring
"cxx-overlay-source-lookup.swift", but this is now sanitized to
`TMP_DIR`. The actual source file name is `client.swift`, created by
`split-file` inside the `%t` directory.
Adjust the rule for how bridging header chaining is performed to
increase compatibilities with existing project configuration.
Previously in #84442, bridging header chaining was done via content
concatenation, rather than `#include` via absolute path, to enable prefix
mapping to drop references to absolute path. This triggers an
incompatibility since the directory of the header file is also
considered as part of the search path when resolve the include
directives in the file. Simple concatenation will result in some
`#include` can no longer be resolved.
Now relax the rule to do header content concatenation only when prefix
map is used. This will keep the breakage minimal, and the breakage can
always be fixed by additional include paths when turning on prefix
mapping.
In the future, we should move towards internal bridging header
implementation to avoid the need of chaining completely.
rdar://161854282
To allow prefix mapping of the bridging header to achieve cache hit when
source files are located in different location, the generated chained
bridging header should not include absolute paths of the headers. Fix
the problem by concat the chained bridging header together.
Fixes: https://github.com/swiftlang/swift/issues/84088
Refactor 'maybeDiagnoseTargetMismatch' to separately collect mismatching target variant modules in 'identifyArchitectureVariants' and rename it to 'handlePossibleTargetMismatch'.
Prior uses of 'maybeDiagnoseTargetMismatch' will continue diagnosing errors/warnings on only discovering incompatible swift binary module target variants.
A new overload of 'handlePossibleTargetMismatch', in 'SwiftModuleScanner', instead collects it as a discovered incompatible candidate, for diagnosis downstream.
Avoid Swift Overlay lookup for the underlying clang module of a known Swift module.
i.e. When computing set of Swift Overlay dependencies for module 'A', which depends on a Clang module 'A', ensure we don't lookup Swift module 'A' itself here - this can lead to bizarre interactions where the source module under scan is queried as a dependency of itself.
Resolves rdar://159706486
This change re-enables the new semantic of looking up Swift Overlay modules only for "visible" clang modules.
We have investigated the failures we were seeing with this change enabled and addressed them with project-side fixes.
Previously, serialization of optional import statement node arrays erroneously used the node layout of a non-optional import node array while maintaining a separate index for these arrays which got serialized into each module's info node. This meant it got serialized out in a sequence of non-optional import arrays, and as a result, the consumer deserialized them as such, which broke ordering of which arrays of import nodes belong which module, causing mismatches between a module's actual import set and the import which got deserialized for it.
This change fixes the optional import serialization routine to use the purpose-made separate layout which gets read out according to the separate optional import array index.
In addition to skipping it on textual Swift module dependencies which were built without C++ interop enabled, also skip it over similarly on binary Swift dependencies
We have identified scenarios where this is insufficient and compilation still relies on the presence of certain overlay modules this excludes. Reverting to prior lookup logic while we investigate.
Resolves rdar://157519278
This is required in order to always have computed the set of visible Clang modules for each Swift module in the graph. Otherwise when some Clang module gets cached as a transitive dependency from a query and is later looked up as a direct dependency, there will not be any computed visible modules set.
Previously Swift overlay lookup was performed for every directly and transitively-imported Clang module.
https://github.com/llvm/llvm-project/pull/147969 introduced the concept of "visible" Clang modules from a given named Clang dependency scanner query which closely maps to the set of modules for which Swift will attempt to load a Swift overlay. This change switches overlay querying to apply only to the set of such visible modules.
Resolves rdar://144797648
Previously this flag was only used to pass explicit dependencies to compilation tasks. This change adds support for the dependency scanner to also consider these inputs when resolving dependencies.
Resolves https://github.com/swiftlang/swift-driver/issues/1951
When querying a Swift module, the scanner now also keeps track of all discovered candidate binary modules which are not compatible with current compilation.
- If a Swift dependency is successfully resolved to a compatible binary module or a textual interface, a warning is emitted for every incompatible binary Swift module discovered along the way.
- If a Swift dependency is not resolved, but incompatible module candidates were found, an error is emitted - while it is likely that the scan would fail downstream, it is also possible that an underlying Clang module dependency (with the same name) is successfuly resolved and the Swift lookup failure is ignored, which is still going to lead to failures most of the time if the client code assumes the presence of the Swift overlay module in this scenario.
This change refactors common error reporting by the scanner into a 'ModuleDependencyIssueReporter' class, which also keeps track of all diagnosed failed lookups to avoid repeating diagnostics.
