Commit Graph

327 Commits

Author SHA1 Message Date
Steven Wu
80f726a815 [ExplicitModuleBuild] Don't leak chained bridging header in objc header
When generating an objc header from the swift module when a bridging
header is used, make sure to use the original bridging header, not
the chained bridging header. This also avoids incorrectly generated a
header include when no actual bridging header is used, just a chained
bridging header that is coming from a dependency.

rdar://148446465
2025-04-02 11:21:17 -07:00
Artem Chikin
2e31e4692f Merge pull request #80414 from artemcm/RestorePriorCanImportBehaviour
[Explicit Module Builds] Switch versioned `canImport` to return `true` when encountering unversioned candidate
2025-04-02 09:02:56 -07:00
Artem Chikin
c9ba79c8da [Dependency Scanning] Always record best version of discovered 'canImport'-ed modules
Suppose module 'Foo' exists in the search paths and specifies user module version '1.0'.

If the first encountered 'canImport' query is unversioned:
...

Followed by a versioned one:
...

The success of the first check will record an unversioned successful canImport, which will cause the second check to evaluate to 'true', which is incorrect.

This change causes even unversioned 'canImport' checks to track and record the discovered user module version.
2025-04-01 14:55:37 -07:00
Artem Chikin
d2ea34c0bd [Explicit Module Builds] Switch versioned 'canImport' to return 'true' when encountering unversioned candidate
Specifically, when the scanner found a candidate which does not carry a user-specified version, it will pass '-module-can-import Foo' to compilation. During compilation, if the check is versioned but the candidate is unversioned, evaluate the check to 'true' to restore the behavior we had with implicitly-built modules.

Resolves rdar://148134993
2025-03-31 16:39:02 -07:00
Hiroshi Yamauchi
42fe7d90cb Merge pull request #79926 from hjyamauchi/explicit-module-build
Propagate vfs overlays and -fbuiltin-headers-in-system-modules
2025-03-28 21:58:40 -07:00
Hiroshi Yamauchi
8312f7ad9a Propagate vfs overlays and -fbuiltin-headers-in-system-modules
This fixes explicit module builds for a hello world program on Windows
as well as the ucrt import build failure as in the included test.
2025-03-27 20:57:13 -07:00
Artem Chikin
04015fef77 [Dependency Scanning] Adopt new clang scanner API to place stable modules into an SDK-specific module cache 2025-03-19 10:43:42 -06:00
Artem Chikin
88dec5199e [Dependency Scanning] Add support for placing explicitly-built SDK modules into a separate module cache
With '-sdk-module-cache-path', Swift textual interfaces found in the SDK will be built into a separate SDK-specific module cache.
Clang modules are not yet affected by this change, pending addition of the required API.
2025-03-19 09:17:04 -06:00
Steven Wu
6de939156d [ExplicitModule] Propagate deterministic check to explicit modules
Make `-enable-deterministic-check` a driver option and teach dependency
scanner to propagate the option to explicit module build commmands. This
allows to the option to check every build output from the compiler is
deterministic.
2025-03-17 14:58:11 -07:00
Artem Chikin
27bac69527 Merge pull request #79930 from artemcm/ClangTargetVariant
[Explicit Module Builds] Add '-clang-target-variant' flag
2025-03-13 07:21:09 -07:00
Artem Chikin
a5d6126e64 Merge pull request #79954 from artemcm/NoTouchyTouchy
[Test Only] Avoid 'touch'ing test inputs in dependency scanning suite
2025-03-12 21:50:48 -07:00
Artem Chikin
1e1bd1a187 [Test Only] Avoid 'touch'ing test inputs in dependency scanning suite
It may interfere with other tests running in-parallel which perform up-to-date consistency checking

Resolves rdar://144133085
2025-03-12 11:10:22 -07:00
Artem Chikin
c0927cb382 Merge pull request #79915 from artemcm/RestrictMacOSOnlyTest
[Test][Dependency Scanning] Restrict macos-only test
2025-03-11 17:26:43 -07:00
Artem Chikin
e96a690cc7 [Explicit Module Builds] Add '-clang-target-variant' flag
https://github.com/swiftlang/swift/pull/37774 added '-clang-target' which allows us to specify a target triple that only differs from '-target' by the OS version, when we want to provide a different OS version for API availability and type-checking, in order to set a common/unified target triple for the entire Clang module dependency graph, for presenting a unified API surface to the Swift client, serving as a maximum type-checking epoch.

This change adds an equivalent flag for the '-target-variant' configuration, as a mechanism to ensure that the entire module dependency graph presents a consistent os version.
2025-03-11 15:48:06 -07:00
Artem Chikin
3a1b41f22d [Test][Dependency Scanning] Restrict macos-only test
The headers this test is relying on have some macos-specific dependency behavior. Restrict the test for now, to unblock testing on other platforms.

Resolves rdar://146771469
2025-03-11 11:23:38 -07:00
Artem Chikin
148fb369fe [Dependency Scanning] Unique collected cross-import overlay files with a set
Resolves rdar://146141228
2025-03-05 15:13:07 -08:00
Qiongsi Wu
3271c193aa [Dependency Scanning] Remove Unnecessary Code that Computes Module Output Path (#79752)
https://github.com/swiftlang/swift/pull/79297 implemented current working directory pruning but left some unnecessary code
that computes Swift interface module output path prematurely. This PR removes the code that computes the output path too 
early. The `ExplicitModuleDependencyResolver` now adds the path to the command line after it can correctly compute it. 

Context: https://github.com/swiftlang/swift/pull/79297/files#r1955314542
2025-03-04 08:44:25 -08:00
Artem Chikin
6f0bc1ede1 [Dependency Scanning] Remove hard failure mode (crash) during cross-import resolution 2025-02-28 13:02:54 -08:00
Artem Chikin
0555764bb4 [Dependency Scanning] Refine cross-import overlay detection algorithm
The algorithm already performs pairwise checks on module dependencies brought into compilation per-source-file. Previously, the algorithm considered the entire sub-graph of a given source file. Actual source compiles do not consider the full transitive module dependency set for cross-import-overlay lookup, but rather only directly-imported modules in a given source file, and '@_exported import' Swift transitive dependencies.

This change adds tracking of whether a given import statement is 'exported' to the dependency scanner and then refines the cross-import overlay lookup logic to only consider transitive modules that are exported by directly-imported dependencies.
2025-02-27 15:48:11 -08:00
Artem Chikin
14dd14d1a4 Merge pull request #79550 from artemcm/EBM_CrossImportsPerSourceFile
[Dependency Scanning] Perform cross-import overlay resolution per source-file
2025-02-25 20:36:33 -08:00
Artem Chikin
214fe59139 [Dependency Scanning] Perform cross-import overlay resolution per source-file
Previous implementation took the entire transitive dependency set and cross-referenced all of its members to determine which ones introduce requried cross-import overlays. That implementation differed from the cross-import overlay loading logic during source compilation, where a corrsponding cross-import overlay module is only requested if the two constituent modules are reachable via direct 'import's from *the same source file*. Meaning the dependency scanner before this change would report cross-import overlay dependencies which never got loaded by the corresponding client source compile.

This change implements a new implementation of cross-import overlay discovery which first computes sub-graphs of module dependencies directly reachable by 'import's for each source file of the module under scan and then performs pairwise cross-import overlay query per each such sub-graph.

Resolves rdar://145157171
2025-02-25 13:40:23 -08:00
Artem Chikin
e0359f7988 Merge pull request #79297 from qiongsiwu/upstream_rehash_after_vfs_overlay_prune
[Dependency Scanning] Update Swift Interface Module's Output Path after `vfs` Pruning
2025-02-14 09:03:36 -08:00
Qiongsi Wu
23e863dc7a Address code review: adding a test for -experimental-clang-importer-direct-cc1-scan and renaming a function. 2025-02-13 14:02:53 -08:00
Qiongsi Wu
15726fc45d Adding a test. 2025-02-12 20:19:39 -08:00
Steven Wu
719edaf7e2 [BridgingHeader] Fix auto-chaining when only dependency has bridging header
When enable bridging header auto chaining, it is possible for the
compilation to have a PCH file input for the bridging header from a
binary swift module dependency. In this case, we should not report a
bridging header for current module as bridging header can be leaking out
through swiftinterface file.

To fully distinguish the PCH files passed in through different
situation, here are the situations:
* If no chaining is used, only `-import-objc-header` option is used and
  it can be used to pass either a header file or a PCH file depending if
  GeneratePCH job is requested or not.
* If chaining is enabled, `-import-objc-header` is only used to pass the
  header file and `-import-pch` is used to pass PCH file. Chaining mode
  requires PCH generation if bridging header is used.

rdar://144623388
2025-02-12 07:30:14 -08:00
Artem Chikin
2bb86e4ecf Merge pull request #79270 from qiongsiwu/exclude_file_comp_dir_from_dep
[Dependency Scanning] Ignore `-ffile-compilation-dir` when Determining Swift Module Variants
2025-02-10 15:00:17 -08:00
Steven Wu
b91842b8d5 Merge pull request #79175 from cachemeifyoucan/eng/PR-144261730 2025-02-06 20:17:01 -08:00
Qiongsi Wu
f24a1325d3 Ignore -ffile-compilation-dir when processing swift modules. 2025-02-06 15:13:42 -08:00
Saleem Abdulrasool
9c85fbc8da AST,DependencyScan,IRGen,Serialization,Tooling: track library style (#78777)
Track if the dependency is static or dynamic. This is in preparation for
helping rename the static library to differentiate it from import
libraries.
2025-02-06 13:22:56 -08:00
Steven Wu
0e8d794ec8 [Dependency Scanning] Scan embedded header content if file doesn't exist
Fallback to scan embedded header content when the bridging header path
encoded in the swift binary module doesn't exit on the disk.

rdar://144261730
2025-02-06 09:28:11 -08:00
Steven Wu
c3550e81ce [Dependency Scan] Do not pass clang's -dwarf-debug-flags to swift
When turning on directCC1 scanning, swift will ask clang driver to
expand all cc1 flags and pass that explicitly on command-line. In such
case, if `RC_DEBUG_OPTIONS` env is set on darwin platform, it will cause
clang driver to encode all the options it gets and pass those clang
driver options as a cc1 options `-dwarf-debug-flags`.

After the change, swift scanner will no longer generate a long
`-Xcc -dwarf-debug-flag -Xcc <long list of flags>` for later
compilation. Losing such information in DWARF doesn't affect swift
debugging. It will instead, make command-line a lot shorter, save spaces
in swift binary module, and make swift caching build more reliable.

rdar://144267483
2025-02-05 16:37:00 -08:00
Steven Wu
9d59044bb1 [BrdigingHeader] Auto bridging header chaining
Add ability to automatically chaining the bridging headers discovered from all
dependencies module when doing swift caching build. This will eliminate all
implicit bridging header imports from the build and make the bridging header
importing behavior much more reliable, while keep the compatibility at maximum.

For example, if the current module A depends on module B and C, and both B and
C are binary modules that uses bridging header, when building module A,
dependency scanner will construct a new header that chains three bridging
headers together with the option to build a PCH from it. This will make all
importing errors more obvious while improving the performance.
2025-02-05 09:41:04 -08:00
Arnold Schwaighofer
26fa30d543 This test seems to block some PR testing
Disable it until folks get a chance to look at it

rdar://144133085
2025-02-04 09:34:36 -08:00
Artem Chikin
06637ae948 Merge pull request #78962 from artemcm/IncrementalScanValidate
[Dependency Scanning] Add functionality to validate contents of a loaded scanner cache state for incremental scans
2025-02-03 15:10:17 -08:00
Artem Chikin
acb4e847f5 [Dependency Scanning] Add functionality to validate contents of a loaded scanner cache state
Checking each module dependency info if it is up-to-date with respect to when the cache contents were serialized in a prior scan.

- Add a timestamp field to the serialization format for the dependency scanner cache
- Add a flag "-validate-prior-dependency-scan-cache" which, when combined with "-load-dependency-scan-cache" will have the scanner prune dependencies from the deserialized cache which have inputs that are newer than the prior scan itself

With the above in-place, the scan otherwise proceeds as-is, getting cache hits for entries still valid since the prior scan.
2025-02-03 10:33:43 -08:00
Steven Wu
c25d2afdc3 Merge pull request #79022 from cachemeifyoucan/eng/PR-bridging-header-swift-overlays
[ScanDependency] Fix a corner case for swift overlay module discovery
2025-01-31 09:47:24 -08:00
Steven Wu
8b02532d78 [ScanDependency] Fix a corner case for swift overlay module discovery
Make sure the clang headers imported from bridging headers can bring in
the swift overlay as well.
2025-01-29 11:41:23 -08:00
Artem Chikin
477ba0dd97 [Dependency Scanning] Remove references to per-triple PCM variant compilation 2025-01-29 11:32:07 -08:00
Artem Chikin
41e471288a [Dependency Scanning] Deprecate/Remove batch scanning capability
Batch dependency scanning was added as a mechanism to support multiple compilation contexts within a single module dependency graph.
The Swift compiler and the Explicitly-built modules model has long since abandoned this approach and this code has long been stale. It is time to remove it and its associated C API.
2025-01-28 15:30:39 -08:00
Artem Chikin
ea2c501131 Merge pull request #78672 from artemcm/DepScanInvalidRejectedBinaryModulesWarn
[Dependency Scanning] Emit a warning when failing to load a binary module of a non-resilient dependency
2025-01-22 15:03:43 -08:00
Artem Chikin
550538f3b5 [Dependency Scanning] Emit a warning when failing to load a binary module of a non-resilient dependency
This failure will most-likely result in the dependency query failure which will fail the scan. It will be helpful if the scanner emitted diagnostic for each such module it rejected to explain the reason why.

Resolves rdar://142906530
2025-01-17 10:52:10 -08:00
Artem Chikin
06e7b6f3a7 [Dependency Scanner] Refactor the global scanning service to no longer maintain scanner cache state
Instead, each scan's 'ModuleDependenciesCache' will hold all of the data corresponding to discovered module dependencies.

The initial design presumed the possibility of sharing a global scanning cache amongs different scanner invocations, possibly even different concurrent scanner invocations.

This change also deprecates two libSwiftScan entry-points: 'swiftscan_scanner_cache_load' and 'swiftscan_scanner_cache_serialize'. They never ended up getting used, and since this code has been largely stale, we are confident they have not otherwise had users, and they do not fit with this design.

A follow-up change will re-introduce moduele dependency cache serialization on a per-query basis and bring the binary format up-to-date.
2025-01-06 09:07:15 -08:00
Allan Shortlidge
3f0eb8ce2c Frontend: Fix -target-variant subarch normalization.
In https://github.com/swiftlang/swift/pull/77156, normalization was introduced
for -target-variant triples. That PR also caused -target-variant arguments to
be inherited from the main compilation options whenever building dependency
modules from their interfaces, which is incorrect. The -target-variant option
must only be specified when compiling a "zippered" module, but the dependencies
of zippered modules are not necessarily zippered themselves and
indiscriminantly propagating the option can cause miscompilation.

The new, more targeted approach to normalizing arm64e triples simply uses the
arch and subarch of the -target argument of the main compile to decide whether
the subarch of both the -target and -target-variant arguments of a dependency
need adjustment.

Resolves rdar://135322077 and rdar://141640919.
2025-01-02 13:51:26 -08:00
Steven Wu
1e399d511c [ScanDependency] Swift source module should not have build args for modules
When planning for a swift source module, it should not get build
commands for its module dependencies. Those dependencies should be
planned and added by swift-driver.

This is another regression from #76700 that causes unnecessary increase
of build command-line size.

rdar://141843125
2024-12-20 16:32:34 -08:00
Richard Howell
8dd9a12618 Error on duplicate module names in Swift module maps
Duplicate module names on search paths produces an error, but
providing duplicate module names in a Swift explicit module map
file does not, instead the first entry will be chosen. Modify
the module map parser to error on duplicated module names as well.
2024-12-10 14:14:06 -08:00
Artem Chikin
f4accedf9a Merge pull request #77794 from artemcm/RefactorGlobalDependencyScanningCache
[Dependency Scanner] Refactor the global scanning service to no longer maintain scanner cache state
2024-12-09 14:50:04 -08:00
Artem Chikin
faf0b21339 Add additional implicit framework search path for Darwin platforms
'/System/Library/SubFrameworks' will be searched for frameworks by the compiler implicitly when an SDK path is specified.

Resolves rdar://137454957
2024-12-05 13:56:52 -08:00
Steven Wu
b66f524e80 [ScanDependency] Fix a regression caused by rewrite in #76700
In the refactoring change #76700, it accidentally introduced a behavior
change that causes the generated PCM command-line to have useful
VFSOverlay files getting dropped. Clang module command-line and its
unused VFS pruning should be done by the clang dependency scanner
already so there is no need to touch that in the swift scanner. Since
the original logics is not used to handle clang module commands, it will
actually dropped the useful vfs overlay that is needed when none of the
dependencies uses it. Fix the regression by restoring the old behavior
and ignoring clang modules when pruning VFS overlay.

rdar://139233781
2024-12-04 16:36:46 -08:00
Artem Chikin
9055a401a3 [Dependency Scanner] Refactor the global scanning service to no longer maintain scanner cache state
Instead, each scan's 'ModuleDependenciesCache' will hold all of the data corresponding to discovered module dependencies.

The initial design presumed the possibility of sharing a global scanning cache amongs different scanner invocations, possibly even different concurrent scanner invocations.

This change also deprecates two libSwiftScan entry-points: 'swiftscan_scanner_cache_load' and 'swiftscan_scanner_cache_serialize'. They never ended up getting used, and since this code has been largely stale, we are confident they have not otherwise had users, and they do not fit with this design.

A follow-up change will re-introduce moduele dependency cache serialization on a per-query basis and bring the binary format up-to-date.
2024-12-04 11:13:05 -08:00
Artem Chikin
f461a817b5 Fix 'clang-target-clang-irgen.swift' test to build correctly against newer SDKs
Resolves rdar://138880276
2024-11-07 13:33:46 -08:00