averello/swift-mirror
1
0
Fork 0
mirror of https://github.com/apple/swift.git synced 2025-12-14 20:36:38 +01:00
Code Issues Packages Projects Releases Wiki Activity

[Dependency Scanning] Refactor the scanner to resolve unqualified module imports

Browse Source 
This changes the scanner's behavior to "resolve" a discovered module's dependencies to a set of Module IDs: module name + module kind (swift textual, swift binary, clang, etc.).

The 'ModuleDependencyInfo' objects that are stored in the dependency scanner's cache now carry a set of kind-qualified ModuleIDs for their dependencies, in addition to unqualified imported module names of their dependencies.

Previously, the scanner's internal state would cache a module dependnecy as having its own set of dependencies which were stored as names of imported modules. This led to a design where any time we needed to process the dependency downstream from its discovery (e.g. cycle detection, graph construction), we had to query the ASTContext to resolve this dependency's imports, which shouldn't be necessary. Now, upon discovery, we "resolve" a discovered dependency by executing a lookup for each of its imported module names (this operation happens regardless of this patch) and store a fully-resolved set of dependencies in the dependency module info.

Moreover, looking up a given module dependency by name (via `ASTContext`'s `getModuleDependencies`) would result in iterating over the scanner's module "loaders" and querying each for the module name. The corresponding modules would then check the scanner's cache for a respective discovered module, and if no such module is found the "loader" would search the filesystem.

This meant that in practice, we searched the filesystem on many occasions where we actually had cached the required dependency, as follows:
Suppose we had previously discovered a Clang module "foo" and cached its dependency info.
-> ASTContext.getModuleDependencies("foo")
--> (1) Swift Module "Loader" checks caches for a Swift module "foo" and doesn't find one, so it searches the filesystem for "foo" and fails to find one.
--> (2) Clang Module "Loader" checks caches for a Clang module "foo", finds one and returns it to the client.

This means that we were always searching the filesystem in (1) even if we knew that to be futile.
With this change, queries to `ASTContext`'s `getModuleDependencies` will always check all the caches first, and only delegate to the scanner "loaders" if no cached dependency is found. The loaders are then no longer in the business of checking the cached contents.

To handle cases in the scanner where we must only lookup either a Swift-only module or a Clang-only module, this patch splits 'getModuleDependencies' into an alrady-existing 'getSwiftModuleDependencies' and a newly-added 'getClangModuleDependencies'.
This commit is contained in:
Artem Chikin
2022-12-19 10:58:05 -08:00
parent a6e056ba7e
commit 12477b7b79
22 changed files with 688 additions and 749 deletions
Download Patch File Download Diff File Expand all files Collapse all files

53
include/swift/DependencyScan/SerializedModuleDependencyCacheFormat.h
View File

@@ -36,7 +36,7 @@ using llvm::BCVBR;
/// Every .moddepcache file begins with these 4 bytes, for easy identification.
const unsigned char MODULE_DEPENDENCY_CACHE_FORMAT_SIGNATURE[] = {'I', 'M', 'D','C'};
const unsigned MODULE_DEPENDENCY_CACHE_FORMAT_VERSION_MAJOR = 3;
const unsigned MODULE_DEPENDENCY_CACHE_FORMAT_VERSION_MAJOR = 4;
/// Increment this on every change.
const unsigned MODULE_DEPENDENCY_CACHE_FORMAT_VERSION_MINOR = 0;
@@ -45,19 +45,21 @@ const unsigned MODULE_DEPENDENCY_CACHE_FORMAT_VERSION_MINOR = 0;
using IdentifierIDField = BCVBR<13>;
using FileIDField = IdentifierIDField;
using ModuleIDField = IdentifierIDField;
using ContextHashField = IdentifierIDField;
using ContextHashIDField = IdentifierIDField;
/// A bit that indicates whether or not a module is a framework
using IsFrameworkField = BCFixed<1>;
/// Arrays of various identifiers, distinguished for readability
using IdentifierIDArryField = llvm::BCArray<IdentifierIDField>;
using ModuleIDArryField = llvm::BCArray<IdentifierIDField>;
/// Identifiers used to refer to the above arrays
using FileIDArrayIDField = IdentifierIDField;
using ContextHashIDField = IdentifierIDField;
using DependencyIDArrayIDField = IdentifierIDField;
using ImportArrayIDField = IdentifierIDField;
using FlagIDArrayIDField = IdentifierIDField;
using DependencyIDArrayIDField = IdentifierIDField;
/// The ID of the top-level block containing the dependency graph
const unsigned GRAPH_BLOCK_ID = llvm::bitc::FIRST_APPLICATION_BLOCKID;
@@ -115,34 +117,35 @@ using IdentifierArrayLayout =
// - SwiftPlaceholderModuleDetails
// - ClangModuleDetails
using ModuleInfoLayout =
BCRecordLayout<MODULE_NODE, // ID
IdentifierIDField, // module name
ContextHashIDField, //
DependencyIDArrayIDField // directDependencies
BCRecordLayout<MODULE_NODE, // ID
IdentifierIDField, // moduleName
ContextHashIDField, // contextHash
ImportArrayIDField, // moduleImports
DependencyIDArrayIDField // resolvedModuleDependencies
>;
using SwiftInterfaceModuleDetailsLayout =
BCRecordLayout<SWIFT_INTERFACE_MODULE_DETAILS_NODE, // ID
FileIDField, // outputFilePath
FileIDField, // swiftInterfaceFile
FileIDArrayIDField, // compiledModuleCandidates
FlagIDArrayIDField, // buildCommandLine
FlagIDArrayIDField, // extraPCMArgs
ContextHashField, // contextHash
IsFrameworkField, // isFramework
FileIDField, // bridgingHeaderFile
FileIDArrayIDField, // sourceFiles
FileIDArrayIDField, // bridgingSourceFiles
FileIDArrayIDField // bridgingModuleDependencies
FileIDField, // outputFilePath
FileIDField, // swiftInterfaceFile
FileIDArrayIDField, // compiledModuleCandidates
FlagIDArrayIDField, // buildCommandLine
FlagIDArrayIDField, // extraPCMArgs
ContextHashIDField, // contextHash
IsFrameworkField, // isFramework
FileIDField, // bridgingHeaderFile
FileIDArrayIDField, // sourceFiles
FileIDArrayIDField, // bridgingSourceFiles
FileIDArrayIDField // bridgingModuleDependencies
>;
using SwiftSourceModuleDetailsLayout =
BCRecordLayout<SWIFT_SOURCE_MODULE_DETAILS_NODE, // ID
FlagIDArrayIDField, // extraPCMArgs
FileIDField, // bridgingHeaderFile
FileIDArrayIDField, // sourceFiles
FileIDArrayIDField, // bridgingSourceFiles
FileIDArrayIDField // bridgingModuleDependencies
FlagIDArrayIDField, // extraPCMArgs
FileIDField, // bridgingHeaderFile
FileIDArrayIDField, // sourceFiles
FileIDArrayIDField, // bridgingSourceFiles
FileIDArrayIDField // bridgingModuleDependencies
>;
using SwiftBinaryModuleDetailsLayout =
@@ -157,14 +160,14 @@ using SwiftPlaceholderModuleDetailsLayout =
BCRecordLayout<SWIFT_PLACEHOLDER_MODULE_DETAILS_NODE, // ID
FileIDField, // compiledModulePath
FileIDField, // moduleDocPath
FileIDField // moduleSourceInfoPath
FileIDField // moduleSourceInfoPath
>;
using ClangModuleDetailsLayout =
BCRecordLayout<CLANG_MODULE_DETAILS_NODE, // ID
FileIDField, // pcmOutputPath
FileIDField, // moduleMapPath
ContextHashField, // contextHash
ContextHashIDField, // contextHash
FlagIDArrayIDField, // commandLine
FileIDArrayIDField, // fileDependencies
FlagIDArrayIDField // capturedPCMArgs