Improve swift dependency scanner by validating and selecting dependency
module into scanner. This provides benefits that:
* Build system does not need to schedule interface compilation task if
the candidate module is picked, it can just use the candidate module
directly.
* There is no need for forwarding module in the explicit module build.
Since the build system is coordinating the build, there is no need for
the forwarding module in the module cache to avoid duplicated work,
* This also correctly supports all the module loading modes in the
dependency scanner.
This is achieved by only adding validate and up-to-date binary module as
the candidate module for swift interface module dependency. This allows
caching build to construct the correct dependency in the CAS. If there
is a candidate module for the interface module, dependency scanner will
return a binary module dependency in the dependency graph.
The legacy behavior is mostly preserved with a hidden frontend flag
`-no-scanner-module-validation`, while the scanner output is mostly
interchangeable with new scanner behavior with `prefer-interface` module
loading mode except the candidate module will not be returned.
rdar://123711823
Otherwise they may have module dependencies of their own which will not be detected by the scanner and included in the list of explicit inputs for compilation.
We only record these dependencies in CAS mode, because we require explicit PCH tasks to be produced for imported header of binary module dependencies. In the meantime, in non-CAS mode loading clients will consume the `.h` files encoded in the `.swiftmodules` directly.
Followup changes to SwiftDriver will enable explicit PCH compilation of such dependenceis, but for the time being restore prior behavior for non-CAS explicit module builds.
Resolves rdar://116006619
The code of `ScanDependencies.cpp` was creating invalid JSON since #66031
because in the case of having `extraPcmArgs` and `swiftOverlayDependencies`,
but not `bridgingHeader`, a comma will not be added at the end of
`extraPcmArgs`, creating an invalid JSON file. Additionally that same PR
added a trailing comma at the end of the `swiftOverlayDependencies`, which
valid JSON does not allow, but that bug was removed in #66366.
Both problems are, however, present in the 5.9 branch, because #66936
included #66031, but not #66366.
Besides fixing the problem in `ScanDependencies.cpp` I modified every test
that uses `--scan-dependencies` to pass the produced JSON through
Python's `json.tool` in order to validate proper JSON is produced. In
most cases I was able to pipe the output of the tool into `FileCheck`,
but in some cases the validation is done by itself because the checks
depend on the exact format generated by `--scan-dependencies`. In
a couple of tests I added a call to `FileCheck` that seemed to be
missing.
Without these changes, two tests seems to be generating invalid JSON in
my machine:
- `ScanDependencies/local_cache_consistency.swift` (which outputs `Expecting ',' delimiter: line 525 column 11 (char 22799)`)
- `ScanDependencies/placholder_overlay_deps.swift`
