[Dependency Scanning] Refactor batch clang dependency query to preserve only partial results

Previously, with the change to bridge Clang dependency scanning results on-demand, the scanner would execute Clang dependency scanning queries for each unresolved import, in parallel, and aggregate all of the results to post-process (including on-demand bridging) later. As a consequence of that change, all of the Clang scanner queries' results ('TranslationUnitDeps') got aggregated during a scan and had their lifetimes extended until a later point when they got processed and added to the scanner's cache.

This change refactors the Clang dependency scanner invocation to, upon query completion, accumulate only the 'ModuleDeps' nodes which have not been registered by a prior scan, discarding the rest of the 'TranslationUnitDeps' graph. The arrgegated 'ModuleDeps' objects are still bridged on-demand downstream.

This change further splits up the 'resolveAllClangModuleDependencies' method's functionality to improve readability and maintainability, into:
- 'gatherUnresolvedImports' method which collects all of collected Swift dependents' imports which did not get resolved to Swift dependencies
- 'performParallelClangModuleLookup' which actually executes the parallel queries and includes the new logic described above
- 'cacheComputedClangModuleLookupResults' method which takes the result of the parallel Clang scanner query and records in in the Swift scanner cache
- 'reQueryMissedModulesFromCache' method which covers the scenario where Clang scanner query returned no result because either the dependency can only be found transitively, or the query is for a dependency previously-queried.

include/swift/DependencyScan/ModuleDependencyScanner.h

@@ -32,6 +32,11 @@ using LookupModuleOutputCallback = llvm::function_ref<std::string(
     clang::tooling::dependencies::ModuleOutputKind)>;
 using RemapPathCallback = llvm::function_ref<std::string(StringRef)>;
 
+/// A map from a module id to a collection of import statement infos.
+using ImportStatementInfoMap =
+    std::unordered_map<ModuleDependencyID,
+                       std::vector<ScannerImportStatementInfo>>;
+
 /// A dependency scanning worker which performs filesystem lookup
 /// of a named module dependency.
 class ModuleDependencyScanningWorker {
@@ -329,6 +334,56 @@ private:
   /// for a given module name.
   Identifier getModuleImportIdentifier(StringRef moduleName);
 
+private:
+  struct BatchClangModuleLookupResult {
+    llvm::StringMap<clang::tooling::dependencies::ModuleDeps>
+        discoveredDependencyInfos;
+    llvm::StringMap<std::vector<std::string>> visibleModules;
+  };
+
+  /// For the provided collection of unresolved imports
+  /// belonging to identified Swift dependnecies, execute a parallel
+  /// query to the Clang dependency scanner for each import's module identifier.
+  void performParallelClangModuleLookup(
+      const ImportStatementInfoMap &unresolvedImportsMap,
+      const ImportStatementInfoMap &unresolvedOptionalImportsMap,
+      ModuleDependenciesCache &cache,
+      BatchClangModuleLookupResult &result);
+
+  /// Given a result of a batch Clang module dependency lookup,
+  /// record its results in the cache:
+  /// 1. Record all discovered Clang module dependency infos
+  ///    in the \c cache.
+  /// 1. Update the set of visible Clang modules from each Swift module
+  ///    in the \c cache.
+  /// 2. Update the total collection of all disovered clang modules
+  ///    in \c allDiscoveredClangModules.
+  /// 3. Record all import identifiers which the scan failed to resolve
+  ///    in \c failedToResolveImports.
+  /// 4. Update the set of resolved Clang dependencies for each Swift
+  ///    module dependency in \c resolvedClangDependenciesMap.
+  void cacheComputedClangModuleLookupResults(
+      const BatchClangModuleLookupResult &lookupResult,
+      const ImportStatementInfoMap &unresolvedImportsMap,
+      const ImportStatementInfoMap &unresolvedOptionalImportsMap,
+      ArrayRef<ModuleDependencyID> swiftModuleDependents,
+      ModuleDependenciesCache &cache,
+      ModuleDependencyIDSetVector &allDiscoveredClangModules,
+      std::vector<std::pair<ModuleDependencyID, ScannerImportStatementInfo>>
+          &failedToResolveImports,
+      std::unordered_map<ModuleDependencyID, ModuleDependencyIDSetVector>
+          &resolvedClangDependenciesMap);
+
+  /// Re-query some failed-to-resolve Clang imports from cache
+  /// in chance they were brought in as transitive dependencies.
+  void reQueryMissedModulesFromCache(
+      ModuleDependenciesCache &cache,
+      const std::vector<
+          std::pair<ModuleDependencyID, ScannerImportStatementInfo>>
+          &failedToResolveImports,
+      std::unordered_map<ModuleDependencyID, ModuleDependencyIDSetVector>
+          &resolvedClangDependenciesMap);
+
   /// Assuming the \c `moduleImport` failed to resolve,
   /// iterate over all binary Swift module dependencies with serialized
   /// search paths and attempt to diagnose if the failed-to-resolve module