Commit Graph

350 Commits

Author SHA1 Message Date
Artem Chikin
5eb85acad5 Merge pull request #82031 from artemcm/RemovePlaceholdersInDepScanner
[Dependency Scanning] Remove obsolete placeholder module concept
2025-06-18 12:40:30 -07:00
Artem Chikin
a78ee29692 [Dependency Scanning] Remove obsolete placeholder module concept
This was used a long time ago for a design of a scanner which could rely on the client to specify that some modules *will be* present at a given location but are not yet during the scan. We have long ago determined that the scanner must have all modules available to it at the time of scan for soundness. This code has been stale for a couple of years and it is time to simplify things a bit by deleting it.
2025-06-12 08:32:25 -07:00
Artem Chikin
c3ce91e071 [Dependency Scanning] Add libSwiftScan API and JSON output for source import information 2025-06-12 06:56:33 -07:00
Artem Chikin
d4abba10a5 Add a new diagnostic group and documentation for the module-not-found failure which specifies that a missing module dependency can be found on a search path serialized in another Swift binary module dependency. 2025-06-04 16:57:01 -07:00
Artem Chikin
aedc2fadb3 [Dependency Scanning] Serialized Swift binary module serialized search paths 2025-06-04 16:55:45 -07:00
Artem Chikin
90f2fba2ae [Dependency Scanning] On failure to locate a module, attempt to diagnose if binary dependencies contain search paths with this module.
Unlike with implicitly-built modules (prior to Swift 6 mode), explicitly-built modules require that all search paths be specified explicitly and no longer inherit search paths serialized into discovered Swift binary modules. This behavior was never intentional and is considered a bug. This change adds a diagnostic note to a scan failure: for each binary Swift module dependency, the scanner will attempt to execute a dependency scanning query for each serialized search path inside that module. If such diagnostic query returns a result, a diagnostic will be emitted to inform the user that the dependency may be found in the search path configuration of another Swift binary module dependency, specifying which search path contains the "missing" module, and stating that such search paths are not automatically inherited by the current compilation.
2025-06-04 16:55:37 -07:00
Artem Chikin
4fb4945ab9 Merge pull request #81908 from artemcm/FixImplicitBuildCXXInteropCycle
[C++Interop] Do not query C++ Standard Library Swift overlays when building Swift modules which were built without C++ interop
2025-06-02 23:26:27 -07:00
Artem Chikin
5b501ad2c5 Hard-code the 'Darwin' module as having been built without C++ interop
Textual interfaces for 'Darwin' built with recent compilers specify that it is built witout C++ interop enabled. However, to ensure compatibility with versions of the 'Darwin' module built with older compilers, we hard-code this fact. This is required to break the module cycle that occurs when building the 'Darwin' module with C++ interop enabled, where the underlying 'Darwin' clang module depends on C++ standard library for which the compiler brings in the 'CxxStdlib' Swift overlay, which depends on 'Darwin'.
2025-06-02 14:16:57 -07:00
Artem Chikin
bb6967bcb9 Merge pull request #81850 from artemcm/BridgeHeaderScanDiagnostics
[Dependency Scanning] Bridge diagnostics emitted during bridging header scanning
2025-05-30 18:36:16 -07:00
Artem Chikin
8ec341eac4 [Dependency Scanning] Bridge diagnostics emitted during bridging header scanning
Instead of simply dumping them to stderr.

Resolves rdar://151993075
2025-05-30 13:50:37 -07:00
Artem Chikin
74e207b7bb Merge pull request #81713 from artemcm/ClangRemapsForDepScanQueries
[Dependency Scanning] Add ClangImporter's mandatory path remaps to dependency scanning query filesystem
2025-05-29 18:25:30 -07:00
Artem Chikin
a9e0a58b46 [Dependency Scanning] Add ClangImporter's mandatory path remaps to dependency scanning query filesystem
On creation, 'ClangImporter' adds overlay modulemap files for non-modular platform libraries (e.g. glibc, libstdc++), which allows Swift code to import and use those libraries.

This change adds the same filesystem overlay to dependency scanning queries by applying them to the filesystem instantiated for each depndency scanning worker. Without these overlays EBM builds cannot discover and use non-modular system libraries on non-Darwin platforms.

Resolves rdar://151780437
2025-05-29 13:33:36 -07:00
Steven Wu
2999ab722d Merge pull request #81454 from cachemeifyoucan/clang-scanner-invocation-callback
[DependencyScanning] Avoid extra cc1 arg round-trip to speed up bridging
2025-05-27 10:58:35 -07:00
Artem Chikin
c4fcee0875 [Dependency Scanning][C++ Interop] Remap lookup of Clang module 'CxxStdlib' to 'std'
Otherwise querying this clang module, e.g. from the corresponding Swift overlay's underlying module import, will fail, since no such module exists.

Resolves rdar://151718115
2025-05-21 09:54:28 -07:00
Steven Wu
391446d4b3 [DependencyScanning] Avoid extra cc1 arg round-trip to speed up bridging
Use the underlying compiler invocation inside module scanning result to
speed up the clang module dependency bridging. This avoids converting
cc1 arguments to compiler invocation and back, just to modify the cc1
arguments needed for building PCM using swift-frontend.

rdar://151705822
2025-05-20 09:54:54 -07:00
Artem Chikin
8ab726c2be Merge pull request #81361 from artemcm/DepScanFewerClangs
[Dependency Scanning][Refactor] Reduce the number of `ClangImporter` instances used by the scanner
2025-05-12 09:57:38 -07:00
Artem Chikin
94898aabf5 [Dependency Scanning][C++Interop] Do not query 'CxxStdlib' Swift overlay for textual modules which were not built with c++interop
When we discover a textual module dependency which is a module which was not originally built from source using C++ interop (specifying '-formal-cxx-interoperability-mode=off'), avoid looking up the C++ standard library Swift overlay for it. This is required for the case of the 'Darwin' module, for example, which includes headers which map to C++ stdlib headers when the compiler is operating in C++ interop mode, but the C++ standard library Swift overlay module itself depends on 'Darwin', which results in a cycle. To resolve such situations, we can rely on the fact that Swift textual interfaces of modules which were not built with C++ interop must be able to build without importing the C++ standard library Swift overlay, so we avoid specifying it as a dependency for such modules.

The primary source module, as well as Swift textual module dependencies which *were* built with C++ interop will continue getting a direct depedency of the 'CxxStdlib' Swift module.

Resolves rdar://150222155
2025-05-09 13:00:14 -07:00
Artem Chikin
8cd193fc08 [Dependency Scanning] Remove 'ClangImporter' instance from dependency scanning worker
Move relevant logic directly into the worker
2025-05-07 16:43:45 -07:00
Artem Chikin
a448cef35f Include system-ness of framework and import search paths in the PCH hash
This hash is also used for the dependency scanning hash. In both cases, PCH contents may differ based on whether a certain module they depend on is found in a system or non-system search path. In dependency scanning, systemness should cause a full change of scanning context requiring a from-scratch scan.

Resolves rdar://150334077
2025-05-01 16:39:12 -07:00
Artem Chikin
7c5d8b24d3 [Dependency Scanning] Add a field of source-imported dependencies
For the main source module, provide info on which dependencies are directly imported into the user program, explicitly ('import' statement) or implicitly (e.g. stdlib). Thist list does not include Swift overlay dependencies, cross-import dependencies, bridging header dependencies.
2025-04-30 10:15:20 -07:00
Artem Chikin
e2f50b56f4 [Dependency Scanning] Miscellaneous fixes to import statement info serialization
- Deserialization of binary module dependencies was still relying on stale code (e.g. 'currentModuleImports', 'currentOptionalModuleImports') from serialized import strings, instead of the now in-use import infos.
- Imports without a source location (e.g. implicit imports of stdlib) were not getting serialized at all
- Optional import arrays were not being written out at all
2025-04-10 10:20:36 -07:00
Steven Wu
1194b59fce Merge pull request #80512 from cachemeifyoucan/eng/PR-148538787
[BridgingHeaderChaining] Bind bridging header module when load module
2025-04-09 10:24:11 -07:00
Steven Wu
02ee2f4d62 [BridgingHeaderChaining] Bind bridging header module when load module
When loading a module with embedded bridging header, bind the bridging
header module in the context when bridging header auto chaining is used.
This is because all the bridging header contents are chained into a PCH
file so binary module with bridging header should reference the PCH file
for all declarations.

rdar://148538787
2025-04-03 15:04:03 -07:00
Steven Wu
80f726a815 [ExplicitModuleBuild] Don't leak chained bridging header in objc header
When generating an objc header from the swift module when a bridging
header is used, make sure to use the original bridging header, not
the chained bridging header. This also avoids incorrectly generated a
header include when no actual bridging header is used, just a chained
bridging header that is coming from a dependency.

rdar://148446465
2025-04-02 11:21:17 -07:00
Artem Chikin
2e31e4692f Merge pull request #80414 from artemcm/RestorePriorCanImportBehaviour
[Explicit Module Builds] Switch versioned `canImport` to return `true` when encountering unversioned candidate
2025-04-02 09:02:56 -07:00
Artem Chikin
c9ba79c8da [Dependency Scanning] Always record best version of discovered 'canImport'-ed modules
Suppose module 'Foo' exists in the search paths and specifies user module version '1.0'.

If the first encountered 'canImport' query is unversioned:
...

Followed by a versioned one:
...

The success of the first check will record an unversioned successful canImport, which will cause the second check to evaluate to 'true', which is incorrect.

This change causes even unversioned 'canImport' checks to track and record the discovered user module version.
2025-04-01 14:55:37 -07:00
Artem Chikin
d2ea34c0bd [Explicit Module Builds] Switch versioned 'canImport' to return 'true' when encountering unversioned candidate
Specifically, when the scanner found a candidate which does not carry a user-specified version, it will pass '-module-can-import Foo' to compilation. During compilation, if the check is versioned but the candidate is unversioned, evaluate the check to 'true' to restore the behavior we had with implicitly-built modules.

Resolves rdar://148134993
2025-03-31 16:39:02 -07:00
Hiroshi Yamauchi
42fe7d90cb Merge pull request #79926 from hjyamauchi/explicit-module-build
Propagate vfs overlays and -fbuiltin-headers-in-system-modules
2025-03-28 21:58:40 -07:00
Hiroshi Yamauchi
8312f7ad9a Propagate vfs overlays and -fbuiltin-headers-in-system-modules
This fixes explicit module builds for a hello world program on Windows
as well as the ucrt import build failure as in the included test.
2025-03-27 20:57:13 -07:00
Artem Chikin
04015fef77 [Dependency Scanning] Adopt new clang scanner API to place stable modules into an SDK-specific module cache 2025-03-19 10:43:42 -06:00
Artem Chikin
88dec5199e [Dependency Scanning] Add support for placing explicitly-built SDK modules into a separate module cache
With '-sdk-module-cache-path', Swift textual interfaces found in the SDK will be built into a separate SDK-specific module cache.
Clang modules are not yet affected by this change, pending addition of the required API.
2025-03-19 09:17:04 -06:00
Steven Wu
6de939156d [ExplicitModule] Propagate deterministic check to explicit modules
Make `-enable-deterministic-check` a driver option and teach dependency
scanner to propagate the option to explicit module build commmands. This
allows to the option to check every build output from the compiler is
deterministic.
2025-03-17 14:58:11 -07:00
Artem Chikin
27bac69527 Merge pull request #79930 from artemcm/ClangTargetVariant
[Explicit Module Builds] Add '-clang-target-variant' flag
2025-03-13 07:21:09 -07:00
Artem Chikin
a5d6126e64 Merge pull request #79954 from artemcm/NoTouchyTouchy
[Test Only] Avoid 'touch'ing test inputs in dependency scanning suite
2025-03-12 21:50:48 -07:00
Artem Chikin
1e1bd1a187 [Test Only] Avoid 'touch'ing test inputs in dependency scanning suite
It may interfere with other tests running in-parallel which perform up-to-date consistency checking

Resolves rdar://144133085
2025-03-12 11:10:22 -07:00
Artem Chikin
c0927cb382 Merge pull request #79915 from artemcm/RestrictMacOSOnlyTest
[Test][Dependency Scanning] Restrict macos-only test
2025-03-11 17:26:43 -07:00
Artem Chikin
e96a690cc7 [Explicit Module Builds] Add '-clang-target-variant' flag
https://github.com/swiftlang/swift/pull/37774 added '-clang-target' which allows us to specify a target triple that only differs from '-target' by the OS version, when we want to provide a different OS version for API availability and type-checking, in order to set a common/unified target triple for the entire Clang module dependency graph, for presenting a unified API surface to the Swift client, serving as a maximum type-checking epoch.

This change adds an equivalent flag for the '-target-variant' configuration, as a mechanism to ensure that the entire module dependency graph presents a consistent os version.
2025-03-11 15:48:06 -07:00
Artem Chikin
3a1b41f22d [Test][Dependency Scanning] Restrict macos-only test
The headers this test is relying on have some macos-specific dependency behavior. Restrict the test for now, to unblock testing on other platforms.

Resolves rdar://146771469
2025-03-11 11:23:38 -07:00
Artem Chikin
148fb369fe [Dependency Scanning] Unique collected cross-import overlay files with a set
Resolves rdar://146141228
2025-03-05 15:13:07 -08:00
Qiongsi Wu
3271c193aa [Dependency Scanning] Remove Unnecessary Code that Computes Module Output Path (#79752)
https://github.com/swiftlang/swift/pull/79297 implemented current working directory pruning but left some unnecessary code
that computes Swift interface module output path prematurely. This PR removes the code that computes the output path too 
early. The `ExplicitModuleDependencyResolver` now adds the path to the command line after it can correctly compute it. 

Context: https://github.com/swiftlang/swift/pull/79297/files#r1955314542
2025-03-04 08:44:25 -08:00
Artem Chikin
6f0bc1ede1 [Dependency Scanning] Remove hard failure mode (crash) during cross-import resolution 2025-02-28 13:02:54 -08:00
Artem Chikin
0555764bb4 [Dependency Scanning] Refine cross-import overlay detection algorithm
The algorithm already performs pairwise checks on module dependencies brought into compilation per-source-file. Previously, the algorithm considered the entire sub-graph of a given source file. Actual source compiles do not consider the full transitive module dependency set for cross-import-overlay lookup, but rather only directly-imported modules in a given source file, and '@_exported import' Swift transitive dependencies.

This change adds tracking of whether a given import statement is 'exported' to the dependency scanner and then refines the cross-import overlay lookup logic to only consider transitive modules that are exported by directly-imported dependencies.
2025-02-27 15:48:11 -08:00
Artem Chikin
14dd14d1a4 Merge pull request #79550 from artemcm/EBM_CrossImportsPerSourceFile
[Dependency Scanning] Perform cross-import overlay resolution per source-file
2025-02-25 20:36:33 -08:00
Artem Chikin
214fe59139 [Dependency Scanning] Perform cross-import overlay resolution per source-file
Previous implementation took the entire transitive dependency set and cross-referenced all of its members to determine which ones introduce requried cross-import overlays. That implementation differed from the cross-import overlay loading logic during source compilation, where a corrsponding cross-import overlay module is only requested if the two constituent modules are reachable via direct 'import's from *the same source file*. Meaning the dependency scanner before this change would report cross-import overlay dependencies which never got loaded by the corresponding client source compile.

This change implements a new implementation of cross-import overlay discovery which first computes sub-graphs of module dependencies directly reachable by 'import's for each source file of the module under scan and then performs pairwise cross-import overlay query per each such sub-graph.

Resolves rdar://145157171
2025-02-25 13:40:23 -08:00
Artem Chikin
e0359f7988 Merge pull request #79297 from qiongsiwu/upstream_rehash_after_vfs_overlay_prune
[Dependency Scanning] Update Swift Interface Module's Output Path after `vfs` Pruning
2025-02-14 09:03:36 -08:00
Qiongsi Wu
23e863dc7a Address code review: adding a test for -experimental-clang-importer-direct-cc1-scan and renaming a function. 2025-02-13 14:02:53 -08:00
Qiongsi Wu
15726fc45d Adding a test. 2025-02-12 20:19:39 -08:00
Steven Wu
719edaf7e2 [BridgingHeader] Fix auto-chaining when only dependency has bridging header
When enable bridging header auto chaining, it is possible for the
compilation to have a PCH file input for the bridging header from a
binary swift module dependency. In this case, we should not report a
bridging header for current module as bridging header can be leaking out
through swiftinterface file.

To fully distinguish the PCH files passed in through different
situation, here are the situations:
* If no chaining is used, only `-import-objc-header` option is used and
  it can be used to pass either a header file or a PCH file depending if
  GeneratePCH job is requested or not.
* If chaining is enabled, `-import-objc-header` is only used to pass the
  header file and `-import-pch` is used to pass PCH file. Chaining mode
  requires PCH generation if bridging header is used.

rdar://144623388
2025-02-12 07:30:14 -08:00
Artem Chikin
2bb86e4ecf Merge pull request #79270 from qiongsiwu/exclude_file_comp_dir_from_dep
[Dependency Scanning] Ignore `-ffile-compilation-dir` when Determining Swift Module Variants
2025-02-10 15:00:17 -08:00
Steven Wu
b91842b8d5 Merge pull request #79175 from cachemeifyoucan/eng/PR-144261730 2025-02-06 20:17:01 -08:00