[Serialization] Improve module loading performance

When looking for a Swift module on disk, we were scanning all module search paths if they contain the module we are searching for. In a setup where each module is contained in its own framework search path, this scaled quadratically with the number of modules being imported. E.g. a setup with 100 modules being imported form 100 module search paths could cause on the order of 10,000 checks of `FileSystem::exists`. While these checks are fairly fast (~10µs), they add up to ~100ms.

To improve this, perform a first scan of all module search paths and list the files they contain. From this, create a lookup map that maps filenames to the search paths they can be found in. E.g. for
```
searchPath1/
  Module1.framework

searchPath2/
  Module1.framework
  Module2.swiftmodule
```
we create the following lookup table
```
Module1.framework -> [searchPath1, searchPath2]
Module2.swiftmodule -> [searchPath2]
```
This commit is contained in:
Alex Hoppen
2021-12-08 22:54:38 +01:00
parent 5a6341bd65
commit fe7878ecce
24 changed files with 581 additions and 212 deletions

View File

@@ -70,27 +70,23 @@ std::error_code ModuleDependencyScanner::findModuleFilesInDirectory(
return dependencies.getError();
}
std::error_code PlaceholderSwiftModuleScanner::findModuleFilesInDirectory(
ImportPath::Element ModuleID, const SerializedModuleBaseName &BaseName,
SmallVectorImpl<char> *ModuleInterfacePath,
std::unique_ptr<llvm::MemoryBuffer> *ModuleBuffer,
std::unique_ptr<llvm::MemoryBuffer> *ModuleDocBuffer,
std::unique_ptr<llvm::MemoryBuffer> *ModuleSourceInfoBuffer,
bool skipBuildingInterface, bool IsFramework) {
StringRef moduleName = Ctx.getRealModuleName(ModuleID.Item).str();
bool PlaceholderSwiftModuleScanner::findModule(
ImportPath::Element moduleID, SmallVectorImpl<char> *moduleInterfacePath,
std::unique_ptr<llvm::MemoryBuffer> *moduleBuffer,
std::unique_ptr<llvm::MemoryBuffer> *moduleDocBuffer,
std::unique_ptr<llvm::MemoryBuffer> *moduleSourceInfoBuffer,
bool skipBuildingInterface, bool &isFramework, bool &isSystemModule) {
StringRef moduleName = Ctx.getRealModuleName(moduleID.Item).str();
auto it = PlaceholderDependencyModuleMap.find(moduleName);
// If no placeholder module stub path is given matches the name, return with an
// error code.
if (it == PlaceholderDependencyModuleMap.end()) {
return std::make_error_code(std::errc::not_supported);
return false;
}
auto &moduleInfo = it->getValue();
auto dependencies = ModuleDependencies::forPlaceholderSwiftModuleStub(
moduleInfo.modulePath, moduleInfo.moduleDocPath,
moduleInfo.moduleSourceInfoPath);
this->dependencies = std::move(dependencies);
return std::error_code{};
return true;
}
static std::vector<std::string> getCompiledCandidates(ASTContext &ctx,