Commit Graph

327 Commits

Author SHA1 Message Date
swift-ci
926498e947 Merge remote-tracking branch 'origin/main' into rebranch 2024-08-08 11:55:29 -07:00
Allan Shortlidge
1083557779 Dependency Scanning: Address -Wmissing-field-initializers warnings.
Fix warnings introduced by https://github.com/swiftlang/swift/pull/75734.
2024-08-08 08:38:25 -07:00
Egor Zhdan
059f0f97d1 [cxx-interop] Allow compiling with libc++ on Linux
This makes sure that Swift respects `-Xcc -stdlib=libc++` flags.

Clang already has existing logic to discover the system-wide libc++ installation on Linux. We rely on that logic here.

Importing a Swift module that was built with a different C++ stdlib is not supported and emits an error.

The Cxx module can be imported when compiling with any C++ stdlib. The synthesized conformances, e.g. to CxxRandomAccessCollection also work. However, CxxStdlib currently cannot be imported when compiling with libc++, since on Linux it refers to symbols from libstdc++ which have different mangled names in libc++.

rdar://118357548 / https://github.com/swiftlang/swift/issues/69825
2024-08-08 16:24:58 +01:00
swift-ci
4d726b37a8 Merge remote-tracking branch 'origin/main' into rebranch 2024-08-06 19:55:44 -07:00
Xi Ge
739e6581d9 Merge pull request #75739 from swiftlang/fix-dep-serialization 2024-08-06 19:44:29 -07:00
Xi Ge
192581811a DependencyScanner: fix misuses of variables when deserializing dependency graph 2024-08-06 16:19:24 -07:00
swift-ci
719a0a9889 Merge remote-tracking branch 'origin/main' into rebranch 2024-08-06 15:54:06 -07:00
Xi Ge
98f4ad840a DependenciesScanner: report user module version for textual Swift interfaces 2024-08-06 15:45:45 -07:00
Xi Ge
b4d64bde55 DependencyScanner: report user module version in dependency scanning results for binary Swift modules 2024-08-06 11:43:53 -07:00
Artem Chikin
e7a76653aa [Dependency Scanning] Enable parallel dependency scanning by-default 2024-08-01 14:26:05 -07:00
swift-ci
bf722c963e Merge remote-tracking branch 'origin/main' into rebranch 2024-07-29 15:14:47 -07:00
Artem Chikin
5dd4977b0f Revert "[Dependency Scanning] Use a standalone DependencyScanningFilesystem on the scanner, sharing a common status cache from Clang's dependency service cache"
This reverts commit 1804a8486e.
2024-07-29 11:58:59 -07:00
swift-ci
b24dc98148 Merge remote-tracking branch 'origin/main' into rebranch 2024-07-24 15:55:32 -07:00
Steven Wu
2bc4080116 [ExplicitModule] Include -Xcc -D into module hash
The decision to exclude `-Xcc -D` options from swift module hash
actually doesn't help to solve the problem. It wouldn't reduce the
module variants (or the number of swiftmodule build commands) because
the command-line also encodes all the clang PCM dependencies that do get
affected by `-Xcc` flags.

To avoid the false sharing and the nondeterministic build products,
include most of the `-Xcc` flags, except include search path, into swift
module hash.

rdar://132046247
2024-07-23 14:57:44 -07:00
swift-ci
6ffea9d35d Merge remote-tracking branch 'origin/main' into rebranch 2024-07-18 13:34:58 -07:00
Steven Wu
b51436ed6c [CrossImport][PrefixMap] Make sure overlay file is remapped if used
When dependency scanner construct the overlay file path for the
swift-frontend invocation, make sure the path is remapped if prefix map
is used.

rdar://131940130
2024-07-17 12:10:30 -07:00
swift-ci
4ba49ce78f Merge remote-tracking branch 'origin/main' into rebranch 2024-07-16 09:56:11 -07:00
Steven Wu
ed936853dc [ScanDependencies] Fix a memory leak in dependency graph
Fix a memory leak from https://github.com/swiftlang/swift/pull/75134.
2024-07-15 20:41:40 -07:00
swift-ci
25be15072e Merge remote-tracking branch 'origin/main' into rebranch 2024-07-11 09:56:08 -07:00
Artem Chikin
a55beaa79d Merge pull request #75139 from artemcm/FixScanSourceLocUseAfterFree
[Dependency Scanning][NFC] Associate single-shot scanner service lifetime with the owning compilation instance's `ASTContext`
2024-07-11 09:45:34 -07:00
Artem Chikin
0d254174c6 Merge pull request #75146 from artemcm/DepScanNoGlobalLLVMOpts
[Dependency Scanning] Avoid configuration (and reset) of LLVM Options by the scanner
2024-07-11 09:45:13 -07:00
Steven Wu
156b0a9b47 Merge pull request #75133 from cachemeifyoucan/eng/PR-131408266
[Modules] Avoid false swift module sharing
2024-07-11 09:36:23 -07:00
swift-ci
fffd12b39b Merge remote-tracking branch 'origin/main' into rebranch 2024-07-11 09:16:06 -07:00
Ben Barham
2715d0e9d6 Merge branch 'main' into 20240710-main-to-rebranch
Conflicts:
  - `test/Interop/Cxx/class/method/methods-this-and-indirect-return-irgen-itanium.swift`
    previously fixed on rebranch, now fixed on main (slightly differently).
2024-07-10 20:42:09 -07:00
Steven Wu
a32dd9539b [Modules] Avoid false swift module sharing
When the swiftmodule is built with different clang importer arguments,
they can have the same module hash, causing them to be wrongly re-used even
they contains different interfaces. Add ReducedExtraArgs to the module hash to
disambiguate them.

However, some Xcc arguments, most commonly `-D` options do not affect the
swiftmodule being generated. Do not pass `-Xcc -DARGS` to swift
interface compilation to reduce the amount of module variants in the
build.

rdar://131408266
2024-07-10 14:48:12 -07:00
Artem Chikin
6d0f52152a [Dependency Scanning] Avoid configuration (and reset) of LLVM Options by the scanner
The scanning action does not have any need for handling `-llvm` options, since it will never perform any code-gen. LLVM option processing relies on global option parsing structures, and the scanner has needed to carefully attempt to synchronize access to them. This change guards the configuration of LLVM options to not happen at all for dependency scanning actions, and removes calls to `llvm::cl::ResetAllOptionOccurrences()` that were previously needed.

Resolves rdar://120754696
2024-07-10 14:33:48 -07:00
Artem Chikin
613277554e [Dependency Scanning][NFC] Associate single-shot scanner service lifetime with the owning compilation instance's ASTContext
Resolves rdar://131388478
2024-07-10 13:49:14 -07:00
Artem Chikin
a47bf3641a [Dependency Scanning] Construct a hollow output on query failure to carry diagnostic output 2024-07-10 12:46:54 -07:00
Artem Chikin
4b7bf3a59d [Dependency Scanning] Emit diagnostics from compilation instance initialization 2024-07-10 10:34:40 -07:00
Steven Wu
da10a02231 [Macro] Fix bugs in macro dependency scanning
Fix few issues from previous implementation from explicit module build
with macros and accurate macro dependency scanning in
https://github.com/swiftlang/swift/pull/73421.

First, there is a crash when propagating the macro dependencies. It
turns out that the current macro plugin implementation doesn't need the
downstream users to know about the plugin search path from the upstream
dependencies.

Secondly, fix a bug that the swiftinterface that has macro usage won't
build because the build command doesn't inherit the plugin search path
option.

Finally, add JSON output for macro dependencies so it is easier to
debug the macro dependencies.

rdar://131214106
2024-07-09 10:37:27 -07:00
swift-ci
f40f3be8a8 Merge remote-tracking branch 'origin/main' into rebranch 2024-07-03 18:14:58 -07:00
Artem Chikin
2464f87f8f [Dependency Scanning] Resolve cross-import overlays relative to defining interface for prebuilt binary Swift dependencies
When the dependency scanner picks a pre-built binary module candidate for a given dependency, it needs to be able to attempt to resolve its cross-import overlays relative to the textual interface that the binary module was built from. For example, if a collection of binary modules are located in, and resolved as dependencies from, a pre-built module directory, the scanner must lookup their corresponding cross-import overlays relative to the defining interface as read out from the binary module's MODULE_INTERFACE_PATH. https://github.com/swiftlang/swift/pull/70817 ensures that binary modules serialize the path to their defining textual interface.

Resolves rdar://130778577
2024-07-03 10:05:54 -07:00
Ben Barham
aa7a3a8268 Cleanup std includes
Remove `deque` from files it isn't actually used in. Add it and `stack`
to files that it is - presumably they were previously transitively found
through other includes.
2024-07-02 16:13:49 -07:00
Artem Chikin
29a8981057 Merge pull request #74750 from artemcm/EBM_DisableCrossImports
[Dependency Scanning] Respect the Language Option for cross-import overlay enablement
2024-06-27 17:35:28 -07:00
Steven Wu
4125e8c3cd [ScanDependencies] Get accurate macro dependency
Build an accurate macro dependency for swift caching. Specifically, do
not include not used macro plugins into the dependency, which might
cause false negatives for cache hits.

This also builds the foundation for future improvement when dependency
scanning will determine the macro plugin to load and swift-frontend do
not need to redo the work.

rdar://127116512
2024-06-27 10:45:14 -07:00
Artem Chikin
643fb60a3e [Dependency Scanning] Respect the Language Option for cross-import overlay enablement 2024-06-27 10:04:08 -07:00
Tim Kientzle
1098054291 Merge branch 'main' into tbkka-assertions2 2024-06-18 17:52:00 -07:00
Tim Kientzle
1d961ba22d Add #include "swift/Basic/Assertions.h" to a lot of source files
Although I don't plan to bring over new assertions wholesale
into the current qualification branch, it's entirely possible
that various minor changes in main will use the new assertions;
having this basic support in the release branch will simplify that.
(This is why I'm adding the includes as a separate pass from
rewriting the individual assertions)
2024-06-05 19:37:30 -07:00
artemcm
d70863501e [Dependency Scanning] Collect and report each module dependency's Link Libraries 2024-06-05 10:59:41 -07:00
Steven Wu
026fcd24fe [ScanDependency][canImport] Improve canImport handling in explicit build
Teach dependency scanner to report all the module canImport check result
to swift-frontend, so swift-frontend doesn't need to parse swiftmodule
or parse TBD file to determine the versions. This ensures dependency
scanner and swift-frontend will have the same resolution for all
canImport checks.

This also fixes two related issues:
* Previously, in order to get consistant results between scanner and
  frontend, scanner will request building the module in canImport check
  even it is not imported later. This slightly alters the definition of
  the canImport to only succeed when the module can be found AND be
  built. This also can affect the auto-link in such cases.
* For caching build, the location of the clang module is abstracted away
  so swift-frontend cannot locate the TBD file to resolve
  underlyingVersion.

rdar://128067152
2024-05-31 15:36:55 -07:00
artemcm
60af9b8b93 [Dependency Scanning] Add forgotten '-Xcc' qualifier to modulemap dependency flags
Resolves rdar://128945712
2024-05-30 12:14:11 -07:00
Steven Wu
47598bbe8f [ScanDependency] Respect working directory set inside invocation
Teach scanner to respect the working directory set in the invocation
through scanner C API.

Also add test infrastructure to testing scanner from C API. Break up
DependencyScan lib into two so the swift-scan-test and remain small
without understanding swift AST.

rdar://127626011
2024-05-22 11:04:45 -07:00
artemcm
9aeadd0507 [Dependency Scanning] Specify Source Locations For Missing Module Dependencies
This change modifies the dependency scanner to keep track of source locations of each encountered 'import' statement, in order to be able to emit diagnostics with source locations if an import failed to resolve.

- Keep track of each 'import' statement's source buffer, line number, and column number when adding it. The dependency scanner utilizes separate compilation instances, and therefore separate Source Managers for scanning `import` statements of user sources and textual interfaces of Swift dependencies. Since import resolution may happen in the main scanner compilation instance while the `import` itself was found by an interface-scanning sub-instance, we cannot simply hold on to the import's `SourceLoc`.
- Add libSwiftScan API for diagnostics to carry above source locations to clients.
2024-05-21 13:12:09 -07:00
Artem Chikin
31337dd004 Merge pull request #69706 from artemcm/DepScanCPPInterop
[Explicit Module Builds] C++ Interoperability mode fixes
2024-05-20 20:29:17 -04:00
Artem Chikin
eb1c0e71d8 [Dependency Scanning] Add required additional C++ interop overlay module queries 2024-05-20 10:23:10 -07:00
Steven Wu
cf8187a4fe [ScanDependency] Pass crossimport overlay file to swift-frontend
Teach dependency scanner to pass cross import overlay file to
swift-frontend for main module compilation. This allows swift-frontend
not to repeat the file system search for overlay files when loading
modules.

This also fixes the issue when caching is enabled, the cross import
doesn't work when the first module is a clang module because the module
built with caching using clang include tree does not preserve
DefinitionLoc which is used to inferred the modulemap location for cross
import overlay search.

rdar://127844120
2024-05-14 15:24:35 -07:00
Artem Chikin
a4f72c3078 Revert "Revert "[Dependency Scanning] Use a standalone DependencyScanningFilesystemon the scanner, sharing a common status cache from Clang's dependency service cache"" 2024-05-01 09:45:12 -07:00
Artem Chikin
983a95c199 Revert "[Dependency Scanning] Use a standalone DependencyScanningFilesystemon the scanner, sharing a common status cache from Clang's dependency service cache" 2024-04-24 15:08:49 -07:00
Artem Chikin
1804a8486e [Dependency Scanning] Use a standalone DependencyScanningFilesystem on the scanner, sharing a common status cache from Clang's dependency service cache 2024-04-09 13:36:46 -07:00
Steven Wu
0e12f2042e [ScanDependency] Move binary module validation into scanner
Improve swift dependency scanner by validating and selecting dependency
module into scanner. This provides benefits that:
* Build system does not need to schedule interface compilation task if
  the candidate module is picked, it can just use the candidate module
  directly.
* There is no need for forwarding module in the explicit module build.
  Since the build system is coordinating the build, there is no need for
  the forwarding module in the module cache to avoid duplicated work,
* This also correctly supports all the module loading modes in the
  dependency scanner.

This is achieved by only adding validate and up-to-date binary module as
the candidate module for swift interface module dependency. This allows
caching build to construct the correct dependency in the CAS. If there
is a candidate module for the interface module, dependency scanner will
return a binary module dependency in the dependency graph.

The legacy behavior is mostly preserved with a hidden frontend flag
`-no-scanner-module-validation`, while the scanner output is mostly
interchangeable with new scanner behavior with `prefer-interface` module
loading mode except the candidate module will not be returned.

rdar://123711823
2024-04-05 07:52:14 -07:00