For the main source module, provide info on which dependencies are directly imported into the user program, explicitly ('import' statement) or implicitly (e.g. stdlib). Thist list does not include Swift overlay dependencies, cross-import dependencies, bridging header dependencies.
This change refactors the top-level dependency scanning flow to follow the following procedure:
Scan():
1. From the source target under scan, query all imported module identifiers for a *Swift* module. Leave unresolved identifiers unresolved. Proceed transitively to build a *Swift* module dependency graph.
2. Take every unresolved import identifier in the graph from (1) and, assuming that it must be a Clang module, dispatch all of them to be queried in-parallel by the scanner's worker pool.
3. Resolve bridging header Clang module dpendencies
4. Resolve all Swift overlay dependencies, relying on all Clang modules collected in (2) and (3)
5. For the source target under scan, use all of the above discovered module dependencies to resolve all cross-import overlay dependencies
Improve swift dependency scanner by validating and selecting dependency
module into scanner. This provides benefits that:
* Build system does not need to schedule interface compilation task if
the candidate module is picked, it can just use the candidate module
directly.
* There is no need for forwarding module in the explicit module build.
Since the build system is coordinating the build, there is no need for
the forwarding module in the module cache to avoid duplicated work,
* This also correctly supports all the module loading modes in the
dependency scanner.
This is achieved by only adding validate and up-to-date binary module as
the candidate module for swift interface module dependency. This allows
caching build to construct the correct dependency in the CAS. If there
is a candidate module for the interface module, dependency scanner will
return a binary module dependency in the dependency graph.
The legacy behavior is mostly preserved with a hidden frontend flag
`-no-scanner-module-validation`, while the scanner output is mostly
interchangeable with new scanner behavior with `prefer-interface` module
loading mode except the candidate module will not be returned.
rdar://123711823
In case import resolution order somehow sometimes matters, it's prudent to process/resolve/locate implicitly-imported modules first.
Resolves rdar://113917657
The code of `ScanDependencies.cpp` was creating invalid JSON since #66031
because in the case of having `extraPcmArgs` and `swiftOverlayDependencies`,
but not `bridgingHeader`, a comma will not be added at the end of
`extraPcmArgs`, creating an invalid JSON file. Additionally that same PR
added a trailing comma at the end of the `swiftOverlayDependencies`, which
valid JSON does not allow, but that bug was removed in #66366.
Both problems are, however, present in the 5.9 branch, because #66936
included #66031, but not #66366.
Besides fixing the problem in `ScanDependencies.cpp` I modified every test
that uses `--scan-dependencies` to pass the produced JSON through
Python's `json.tool` in order to validate proper JSON is produced. In
most cases I was able to pipe the output of the tool into `FileCheck`,
but in some cases the validation is done by itself because the checks
depend on the exact format generated by `--scan-dependencies`. In
a couple of tests I added a call to `FileCheck` that seemed to be
missing.
Without these changes, two tests seems to be generating invalid JSON in
my machine:
- `ScanDependencies/local_cache_consistency.swift` (which outputs `Expecting ',' delimiter: line 525 column 11 (char 22799)`)
- `ScanDependencies/placholder_overlay_deps.swift`
