Commit Graph

27 Commits

Author SHA1 Message Date
Artem Chikin
6176657285 [Dependency Scanning] Include initial PCM arguments on Clang module dependency details
Doing so will allow clients to know which Swift-specific PCM arguments are already captured from the scan that first discovered this module.
SwiftDriver, in particular, will be able to use this information to avoid re-scanning a given Clang module if the initial scan was sufficient for all possible sets of PCM arguments on Swift modules that depend on said Clang module.
2021-09-22 11:34:02 -07:00
Artem Chikin
709676c20c [Dependency Scanning] Make GlobalModuleDependenciesCache aware of the current scanning action's target triple
And only resolve cached dependencies that came from scanning actions with the same target triple.

This change means that the `GlobalModuleDependenciesCache` must be configured with a specific target triple for every scannig action, and it will only resolve previously-found dependencies from previous scannig actions using the exact same triple.

Furthermore, the `GlobalModuleDependenciesCache` separately tracks source-file-based module dependencies as those represent main Swift modules of previous scanning actions, and we must be able to resolve those regardless of the target triple.

Resolves rdar://83105455
2021-09-20 11:21:16 -07:00
Artem Chikin
e64a40451b [Dependency Scanning] Model main module as separate dependency kind: SwiftSource
These kinds of modules differ from `SwiftTextual` modules in that they do not have an interface and have source-files.
It is cleaner to enforce this distinction with types, instead of checking for interface optionality everywhere.
2021-09-15 09:31:20 -07:00
Artem Chikin
9429136112 [Dependency Scanning] Split the ModuleDependencyCache into two: current scan cache & global.
This change causes the cache to be layered with a local "cache" that wraps the global cache, which will serve as the source of truth. The local cache persists only for the duration of a given scanning action, and has a store of references to dependencies resolved as a part of the current scanning action only, while the global cache is the one that persists across scanning actions (e.g. in `DependencyScanningTool`) and stores actual module dependency info values.

Only the local cache can answer dependency lookup queries, checking current scanning action results first, before falling back to querying the global cache, with queries disambiguated by the current scannning action's search paths, ensuring we never resolve a dependency lookup query with a module info that could not be found in the current action's search paths.

This change is required because search-path disambiguation can lead to false-negatives: for example, the Clang dependency scanner may find modules relative to the compiler's path that are not on the compiler's direct search paths. While such false-negative query responses should be functionally safe, we rely on the current scanning action's results being always-present-in-the-cache for the scanner's functionality. This layering ensures that the cache use-sites remain unchanged and that we get both: preserved global state which can be queried disambiguated with the search path details, and an always-consistent local (current action) cache state.
2021-08-06 09:13:42 -07:00
Artem Chikin
6a12dc0070 [Dependency Scanning] Have the scanner cache answer queries relevant to current search paths only.
The dependency scanner's cache persists across different queries and answering a subsequent query's module lookup with a module not in the query's search path is not correct.

For example, suppose we are looking for a Swift module `Foo` with a set of search paths `SP`.
And dependency scanner cache already contains a module `Foo`, for which we found an interface file at location `L`. If `L`∉`SP`, then we cannot re-use the cached entry because we’d be resolving the scanning query to a filesystem location that the current scanning context is not aware of.

Resolves rdar://81175942
2021-07-30 09:53:04 -07:00
Artem Chikin
14229f13b0 [Dependency Scanning] Add a binary serialization format for the Inter-Module Dependencies Cache
- Adds serialization format based on the LLVM Bitcode File Format (https://llvm.org/docs/BitCodeFormat.html).
- Adds Serialization and Deserialization code.
2021-05-21 14:31:00 -07:00
Artem Chikin
3221734e7b Merge pull request #34244 from artemcm/RefactorScannerBinaryDependencies
[Dependency Scanner] Refactor ModuleDependencies to represent binary-only Swift modules explicitly
2020-10-12 17:03:25 -07:00
Artem Chikin
f9d6c6a619 [Dependency Scanner] Refactor ModuleDependencies to represent binary-only Swift modules explicitly
This matches the behavior of the current client (`swift-driver`) and reduces ambiguity in how the nodes in the graph are to be treated. Swift dependencies with a textual interface, for example, must be built into a binary module by clients. Swift dependencies without a textual interface, with only a binary module, are to be used directly, without any up-to-date checks.

Note, this is distinct from Swift dependencies that have a textual interface, for which we also detect potential pre-build binary module candidates. Those are still reported in the `details` field of textual Swift dependencies as `prebuiltModuleCandidates`.
2020-10-12 09:56:03 -07:00
Brent Royal-Gordon
d02b34cccc [NFC] Add conveniences to clean up import code 2020-10-09 18:59:19 -07:00
Artem Chikin
721f903567 Address code-review feedback for #33430 2020-08-13 13:26:41 -07:00
Artem Chikin
40c1687fd2 [Fast Dependency Scanner] Produce information on whether an explicit module is a framework
In the fast dependency scanner, depending on whether a module intrface was found via the import search path or framework search path, encode into the dependency graph Swift module details, whether a given module is a framework.
2020-08-12 10:12:35 -07:00
Artem Chikin
965ca698d5 Add in-source rationale for why Placeholder dependencies are needed. 2020-07-28 11:34:27 -07:00
Artem Chikin
746e89b252 Rename external dependencies to placeholder dependencies. 2020-07-27 09:24:36 -07:00
Artem Chikin
7a736f6582 Refactor ModuleDependenciesStorageBase to self-identify with ModuleDependenciesKind 2020-07-27 09:24:35 -07:00
Artem Chikin
b85f547b13 Add an external module dependency test 2020-07-27 09:24:35 -07:00
Artem Chikin
b5af62f93f Add new dependency kind (swiftExternal) to model external dependencies. 2020-07-27 09:24:35 -07:00
Xi Ge
056f00322d DependenciesScanner: refactor cross import overlay resolver to a standalone function. NFC
This change facilitates resolving cross-import overlays from transitive dependencies.
2020-07-23 17:47:47 -07:00
Xi Ge
b93ff79cb0 DependenciesScanner: include compiled module candidates for textual module interface in JSON output
Instead of replacing an interface file with its up-to-date compile module,
the dep-scanner should report potentially up-to-date module candidates either adjacent to
the interface file or in the prebuilt module cache. swift-driver should later pass down
these candidates to -compile-module-from-interface invocation and the front-end job
will check if one of the candidates is ready to use. The front-end job then either emits a forwarding
module to an up-to-date candidate or a binary module.
2020-07-17 12:32:38 -07:00
Xi Ge
84dd4db0b1 Dependencies Scanner: report compiled Swift module paths if they are available
For the explicit module mode, swift-driver uses -compile-module-from-interface to
generate modules from interfaces found by the dependency scanner. However, we don't
need to build the binary module if up-to-date modules are available, either adjacent
to the interface file or in the prebuilt module cache directory. This patch teaches
dependencies scanner to report these ready-to-use binary modules.
2020-07-01 11:54:15 -07:00
Xi Ge
00872ba53e DependencyScanner: add a new extraPcmArgs field for each Swift module
Building each Swift module explicitly requires dependency PCMs to be built
with the exactly same deployment target version. This means we may need to
build a Clang module multiple times with different target triples.

This patch removes the -target arguments from the reported PCM build
arguments and inserts extraPcmArgs fields to each Swift module.
swift-driver can combine the generic PCM arguments with these extra arguments
to get the command suitable for building a PCM specifically for
that loading Swift module.
2020-06-16 09:42:59 -07:00
Xi Ge
42cc989136 DependenciesScanner: teach the scanner to handle cross-import overlays 2020-06-09 10:38:52 -07:00
Varun Gandhi
7df7378048 [NFC] Remove redundant includes for llvm/ADT/DenseMap.h. 2020-05-31 13:07:45 -07:00
Xi Ge
7d08a24161 ModuleInterface: reconstruct command-line arguments for building Swift module from interface explicitly 2020-05-15 10:49:45 -07:00
Doug Gregor
012878630a [Dependency scanning] Improve overview documentation somewhat. 2020-04-24 12:58:45 -07:00
Doug Gregor
6b5e7efc0e [Dependency scanner] Capture Clang context hash and command-line arguments. 2020-04-24 12:58:45 -07:00
Doug Gregor
d6484c8280 [Dependency scanning] Scan and record bridging header dependencies.
When there is a bridging header associated with the module, scan and record
its dependencies. Note them in a separate structure to capture the specific
dependencies of the bridging header.
2020-04-24 12:58:45 -07:00
Doug Gregor
33cdd61835 Fast dependency scanning for Swift
Implement a new "fast" dependency scanning option,
`-scan-dependencies`, in the Swift frontend that determines all
of the source file and module dependencies for a given set of
Swift sources. It covers four forms of modules:

1) Swift (serialized) module files, by reading the module header
2) Swift interface files, by parsing the source code to find imports
3) Swift source modules, by parsing the source code to find imports
4) Clang modules, using Clang's fast dependency scanning tool

A single `-scan-dependencies` operation maps out the full
dependency graph for the given Swift source files, including all
of the Swift and Clang modules that may need to be built, such
that all of the work can be scheduled up front by the Swift
driver or any other build system that understands this
option. The dependency graph is emitted as JSON, which can be
consumed by these other tools.
2020-04-24 12:58:41 -07:00