Commit Graph

67 Commits

Author SHA1 Message Date
Alex Hoppen
ed3b64af2c [Index] Add an option to compress the record and unit files
Companion of https://github.com/swiftlang/llvm-project/pull/10977.
2025-07-10 15:23:51 +02:00
Egor Zhdan
b51cfa5c76 [cxx-interop] Remove symbolic import mode
Importing C++ class templates in symbolic mode has proven to be problematic in interaction with other compiler features, and it isn't used widely. This change removes the feature.

rdar://150528798
2025-05-02 18:43:09 +01:00
Hiroshi Yamauchi
9ecb8465ce Handle delete_pending error code
Apply the same fix as
https://github.com/swiftlang/llvm-project/pull/8838 to the index
writer in swift/lib/Index/IndexRecord.cpp noting it is meant to be
merged with the IndexUnitWriter in LLVM.

This should fix intermittent permission denied errors during builds
that https://github.com/swiftlang/llvm-project/pull/8838 fixes (but
missed this one).
2025-01-13 13:49:40 -08:00
Artem Chikin
fdda02b596 [Indexing] Disable indexing textual-interface-blocklisted modules during Explicit Module Builds
If a module is blocklisted from the compiler using its textual interface, then under Implicitly-Built modules it will not get indexed, since indexing will not be able to spawn swiftinterface compilation. With explicitly-built modules, none of the dependency modules get built from interface during indexing, which means we directly index input binary modules.

For now, for functional parity with Implicit Module Builds, disable indexing of modules during Explicit Module Builds which would not get indexed during Implicit Module Builds.
2024-12-12 09:18:10 -08:00
Ben Barham
10f43a8365 [Index] Convert to hashing to HashBuilder with BLAKE3
`Hashing.h` is non-deterministic between runs. Update the index hashing
to use BLAKE3 for the record hash. xxhash is faster in benchmarks that
I've found, but there's no easy `HashBuilder` option for it today.
2024-10-31 14:46:00 +10:00
Steven Wu
3b332bf125 [IndexRecord] Fix an handled error in IndexRecord
Fix an unhandled error when querying file system. Just properly ignore
and continue if the file can't be read.
2024-08-06 12:18:28 -07:00
Xi Ge
736ccef626 Merge remote-tracking branch 'apple/main' into rebranch 2024-06-20 15:16:55 -07:00
Tim Kientzle
1d961ba22d Add #include "swift/Basic/Assertions.h" to a lot of source files
Although I don't plan to bring over new assertions wholesale
into the current qualification branch, it's entirely possible
that various minor changes in main will use the new assertions;
having this basic support in the release branch will simplify that.
(This is why I'm adding the includes as a separate pass from
rewriting the individual assertions)
2024-06-05 19:37:30 -07:00
swift-ci
2029cc9354 Merge remote-tracking branch 'origin/main' into rebranch 2024-05-29 09:54:31 -07:00
Alex Hoppen
0614fc972d [Index] Write empty record files
When indexing an empty Swift file, we get a unit file that doesn’t have any record dependency declared on the source file. Because of this, we always assume that the empty file has an out of date index store entry and re-index it on project open. Write empty record files to fix this.

rdar://128711594
2024-05-25 08:21:25 -07:00
Ben Barham
a9077d7129 FileEntry to FileEntryRef updates 2024-04-08 08:58:58 -07:00
Hamish Knight
8a9be104a2 [Index] Scope the setting of IgnoreAdjacentModules
I don't _think_ this currently causes any issues,
but make sure we unset `IgnoreAdjacentModules` when
done loading the module.
2024-03-12 21:14:14 +00:00
Artem Chikin
e0275f4f0e [Explicit Module Builds] Do not attempt to re-load module dependency from
interface for index.

An explicit module build compile is unable to do so because it does not have
access to the interfaces. Doing this in the first place is a workaround for a
known bug, which will require to be solved at the root cause instead (e.g.
Deserialization Safety feature).

Resolves rdar://113165898
2023-08-25 16:34:09 -07:00
Ben Barham
5cdc7db420 [Index] Write stdlib records in alphabetical order
The order for writing records of the stdlib currently depends on
`StringMap` iteration (in a slightly roundabout manner). Sort these
alphabetically instead.
2023-08-18 19:38:26 -07:00
Alexis Laferrière
205a2edf38 [Sema] Intro set of import filters for general use
Calling getImportedModules requires to list the desired kind of imports.
With the new kind of imports this has become cumbersome. Let's simplify
it by offering common sets of imports. Advanced call sites can still
list the desired imports explicitly.
2023-03-17 16:05:44 -07:00
Ben Barham
eec2848508 [SourceKit] Stop using isSystemModule to represent "non-user" modules
Rather than using `ModuleDecl::isSystemModule()` to determine whether a
module is not a user module, instead check whether the module was
defined adjacent to the compiler or if it's part of the SDK.

If no SDK path was given, then `isSystemModule` is still used as a
fallback.

Resolves rdar://89253201.
2023-03-15 14:29:48 -07:00
Ben Barham
6269643b4d [Index] Prevent re-indexing system modules repeatedly
If a module was first read using the adjacent swiftmodule and then
reloaded using the swiftinterface, we would do an up to date check on
the adjacent module but write out the unit using the swiftinterface.
This would cause the same modules to be indexed repeatedly for the first
invocation using a new SDK. On the next run we would instead raad the
swiftmodule from the cache and thus the out of date check would match
up.

The impact of this varies depending on the size of the module graph in
the initial compilation and the number of jobs started at the same time.
Each SDK dependency is re-indexed *and* reloaded, which is a drain on
both CPU and memory. Thus, if many jobs are initially started and
they're all going down this path, it can cause the system to run out of
memory very quickly.

Resolves rdar://103119964.
2023-02-09 11:49:13 -08:00
Alex Lorenz
c352200d3e [interop] review fixes for symbolic interface emission 2023-02-07 13:42:10 -08:00
Alex Lorenz
bacd5ad67f [interop] emit symbolic interface files for re-exported clang module from a Swift module
This allows us to produce an interface for libc++ again
2023-02-02 18:46:46 -08:00
Alex Lorenz
0e60775e9a NFC, refactor importer::requiresCPlusPlus into reusable function 2023-01-31 14:58:25 -08:00
Alex Lorenz
d4aa18ab9f [cxx-interop][index] emit symbolic interface files for C++ modules 2023-01-31 14:58:19 -08:00
Alexis Laferrière
82486f2bfd [Index] Index any system module when there's no SDK 2022-12-13 11:37:20 -08:00
Alexis Laferrière
ac704e4e7f [Index] Prepare to support more distributed modules than the SDKs 2022-12-12 16:08:48 -08:00
Alexis Laferrière
14c639b549 [Index] Index only system modules in the SDK
Indexing a module from the swiftinterface puts the swiftinterface
version in the cache. Subsequent builds don't see the adjacent
swiftmodule file. This can break test clients that need the adjacent
swiftmodule information. To avoid this scenario, ensure that we only
index system modules in the SDK, not local versions.

rdar://102207620
2022-12-07 13:27:08 -08:00
Erik Eckstein
ab1b343dad use new llvm::Optional API
`getValue` -> `value`
`getValueOr` -> `value_or`
`hasValue` -> `has_value`
`map` -> `transform`

The old API will be deprecated in the rebranch.
To avoid merge conflicts, use the new API already in the main branch.

rdar://102362022
2022-11-21 19:44:24 +01:00
Alexis Laferrière
ae0077979b [Index] Don't reload modules already built from swiftinterface 2022-10-31 10:58:57 -07:00
Alexis Laferrière
3b63f4986a [Index] Do not reload the stdlib when indexing system modules 2022-10-31 10:58:57 -07:00
Alexis Laferrière
5ed1f1236d [Sema|Index] Ignore the cache when loading a system module for indexing 2022-10-31 10:58:57 -07:00
Alexis Laferrière
520537e42b [Index] Apply small comments from PR review 2022-10-31 10:58:57 -07:00
Alexis Laferrière
90feb49d20 [Index] Remark on indexing system modules and use it in tests 2022-10-31 10:58:57 -07:00
Alexis Laferrière
b250f3e0e3 [Index] Skip indexing modules failing to build but write it down 2022-10-31 10:58:57 -07:00
Alexis Laferrière
c7de4b183d [Index] Move reloading from swiftinterface after the "up to date" check 2022-10-31 10:58:56 -07:00
Alexis Laferrière
01d35f3b85 [Index] Force indexing from the swiftinterfaces of system modules
Use setIgnoreAdjacentModules before indexing any resilient system module
to force indexing to read only from resilient modules.

rdar://100644036
2022-10-31 10:58:56 -07:00
Tony Allevato
4d16c43d21 Add -index-include-locals flag.
When this flag is passed, index store data emitted during a build
will also contain definitions/references for local symbols.
2022-07-11 14:08:07 -07:00
David Goldman
b9391289e1 Also fetch proper module name for system modules 2022-06-02 10:02:01 -04:00
David Goldman
65062c9261 Capture the proper module name even if the module indexing is disabled 2022-05-17 19:41:28 -04:00
David Goldman
63c88b97f1 Add -index-ignore-clang-modules flag
This flag avoids indexing import clang modules (pcms), behaving
similar to `-index-ignore-system-modules` except for PCMs.
2022-05-17 19:41:26 -04:00
David Goldman
c232ed2913 Support hermetic indexing information
Swiftc port of https://github.com/apple/llvm-project/pull/4207.

This introduces a new flag, `-file-prefix-map` which can be used
instead of the existing `-debug-prefix-map` and `-coverage-prefix-map`
flags, and also remaps paths in index information currently.
2022-05-16 11:00:14 -04:00
Ellie Shin
0ff713d44c [Indexing] Use module real name in case module aliasing is used
Resolves rdar://82896373
2021-12-07 02:22:36 -08:00
Brent Royal-Gordon
b440ab7331 [NFC] Move several types/functions to Import.h
To help consolidate our various types describing imports, this commit moves the following types and methods to Import.h:

* ImplicitImports
* ImplicitStdlibKind
* ImplicitImportInfo
* ModuleDecl::ImportedModule
* ModuleDecl::OrderImportedModules (as ImportedModule::Order)
* ModuleDecl::removeDuplicateImports() (as ImportedModule::removeDuplicates())
* SourceFile::ImportFlags
* SourceFile::ImportOptions
* SourceFile::ImportedModuleDesc

This commit is large and intentionally kept mechanical—nothing interesting to see here.
2020-10-09 18:57:07 -07:00
Varun Gandhi
d3369f7711 [NFC] Rename ImportFilterKind cases to be clearer. 2020-09-23 12:50:20 -07:00
Martin Boehme
d806ba53f6 Give OptionSet an initializer_list constructor.
This makes it easier to specify OptionSet arguments.

Also modify appropriate uses of ModuleDecl::ImportFilter to take
advantage of the new constructor.
2020-06-22 06:57:29 +02:00
Dave Lee
4e227f8a76 [Index] Apply -index-ignore-system-modules to clang modules 2020-05-26 10:45:52 -07:00
Daniel Sweeney
ea526c6383 Converting ModuleDecl::ImportedModule from std::pair to a dedicated struct. (#31360) 2020-04-30 20:26:03 -07:00
Nathan Hawes
25d531c842 Merge pull request #30856 from nathawes/dont-process-group-name
[Index] Don't replace ' ' or '-' in group names when mapping them to a 'module' name
2020-04-08 10:42:25 -07:00
Dan Zheng
c834696bfa Add SynthesizedFileUnit.
`SynthesizedFileUnit` is a container for synthesized declarations. Currently, it
only supports module-level declarations.

It is used by the SIL differentiation transform, which generates implicit struct
and enum declarations.
2020-04-07 18:29:26 -07:00
Nathan Hawes
4592c0ad2d [Index] Don't replace ' ' or '-' in group names when mapping them to a 'module' name.
The fake module names we create when indexing a Swift module divided into
'groups' (e.g. the stdlib) don't have to be valid Swift module names, but we
were manipulating them to be so. E.g. the stdlib "Lazy Views" group, became
"Lazy_Views". This name is displayed in some editors verbatim, and is also
used as input to some sourcekitd APIs (e.g. editor.open.interface) that only
work if the original group name is passed, rather than the processed one.

This patch stops mapping invalid module name characters like ' ' and '-' to '_'
so the original group name remains.

Resolves rdar://problem/57270811
2020-04-07 12:18:36 -07:00
Nathan Hawes
9f6f76308e Add method to ModuleDecl to lazily compute the underlying module for cross-import overlays
Also update code completion, indexing, interface generation and doc info to use it.
2020-04-03 16:04:32 -07:00
Nathan Hawes
d865b524f3 [Index] Make StringRef -> std::string conversions explicit.
Resovles rdar://problem/60944263
2020-03-27 11:11:11 -07:00
Nathan Hawes
a785fa6cee [Frontend][Index] Add frontend option to skip indexing the stdlib (for test performance)
Several tests related to indexing system modules were taking a considerable
amount of time (100+ seconds in the worst case) indexing the standard library.
This adds a frontend option to skip it and updates those tests to pass it.
2020-03-25 14:36:23 -07:00