Commit Graph

54 Commits

Author SHA1 Message Date
Ben Barham
5cdc7db420 [Index] Write stdlib records in alphabetical order
The order for writing records of the stdlib currently depends on
`StringMap` iteration (in a slightly roundabout manner). Sort these
alphabetically instead.
2023-08-18 19:38:26 -07:00
Alexis Laferrière
205a2edf38 [Sema] Intro set of import filters for general use
Calling getImportedModules requires to list the desired kind of imports.
With the new kind of imports this has become cumbersome. Let's simplify
it by offering common sets of imports. Advanced call sites can still
list the desired imports explicitly.
2023-03-17 16:05:44 -07:00
Ben Barham
eec2848508 [SourceKit] Stop using isSystemModule to represent "non-user" modules
Rather than using `ModuleDecl::isSystemModule()` to determine whether a
module is not a user module, instead check whether the module was
defined adjacent to the compiler or if it's part of the SDK.

If no SDK path was given, then `isSystemModule` is still used as a
fallback.

Resolves rdar://89253201.
2023-03-15 14:29:48 -07:00
Ben Barham
6269643b4d [Index] Prevent re-indexing system modules repeatedly
If a module was first read using the adjacent swiftmodule and then
reloaded using the swiftinterface, we would do an up to date check on
the adjacent module but write out the unit using the swiftinterface.
This would cause the same modules to be indexed repeatedly for the first
invocation using a new SDK. On the next run we would instead raad the
swiftmodule from the cache and thus the out of date check would match
up.

The impact of this varies depending on the size of the module graph in
the initial compilation and the number of jobs started at the same time.
Each SDK dependency is re-indexed *and* reloaded, which is a drain on
both CPU and memory. Thus, if many jobs are initially started and
they're all going down this path, it can cause the system to run out of
memory very quickly.

Resolves rdar://103119964.
2023-02-09 11:49:13 -08:00
Alex Lorenz
c352200d3e [interop] review fixes for symbolic interface emission 2023-02-07 13:42:10 -08:00
Alex Lorenz
bacd5ad67f [interop] emit symbolic interface files for re-exported clang module from a Swift module
This allows us to produce an interface for libc++ again
2023-02-02 18:46:46 -08:00
Alex Lorenz
0e60775e9a NFC, refactor importer::requiresCPlusPlus into reusable function 2023-01-31 14:58:25 -08:00
Alex Lorenz
d4aa18ab9f [cxx-interop][index] emit symbolic interface files for C++ modules 2023-01-31 14:58:19 -08:00
Alexis Laferrière
82486f2bfd [Index] Index any system module when there's no SDK 2022-12-13 11:37:20 -08:00
Alexis Laferrière
ac704e4e7f [Index] Prepare to support more distributed modules than the SDKs 2022-12-12 16:08:48 -08:00
Alexis Laferrière
14c639b549 [Index] Index only system modules in the SDK
Indexing a module from the swiftinterface puts the swiftinterface
version in the cache. Subsequent builds don't see the adjacent
swiftmodule file. This can break test clients that need the adjacent
swiftmodule information. To avoid this scenario, ensure that we only
index system modules in the SDK, not local versions.

rdar://102207620
2022-12-07 13:27:08 -08:00
Erik Eckstein
ab1b343dad use new llvm::Optional API
`getValue` -> `value`
`getValueOr` -> `value_or`
`hasValue` -> `has_value`
`map` -> `transform`

The old API will be deprecated in the rebranch.
To avoid merge conflicts, use the new API already in the main branch.

rdar://102362022
2022-11-21 19:44:24 +01:00
Alexis Laferrière
ae0077979b [Index] Don't reload modules already built from swiftinterface 2022-10-31 10:58:57 -07:00
Alexis Laferrière
3b63f4986a [Index] Do not reload the stdlib when indexing system modules 2022-10-31 10:58:57 -07:00
Alexis Laferrière
5ed1f1236d [Sema|Index] Ignore the cache when loading a system module for indexing 2022-10-31 10:58:57 -07:00
Alexis Laferrière
520537e42b [Index] Apply small comments from PR review 2022-10-31 10:58:57 -07:00
Alexis Laferrière
90feb49d20 [Index] Remark on indexing system modules and use it in tests 2022-10-31 10:58:57 -07:00
Alexis Laferrière
b250f3e0e3 [Index] Skip indexing modules failing to build but write it down 2022-10-31 10:58:57 -07:00
Alexis Laferrière
c7de4b183d [Index] Move reloading from swiftinterface after the "up to date" check 2022-10-31 10:58:56 -07:00
Alexis Laferrière
01d35f3b85 [Index] Force indexing from the swiftinterfaces of system modules
Use setIgnoreAdjacentModules before indexing any resilient system module
to force indexing to read only from resilient modules.

rdar://100644036
2022-10-31 10:58:56 -07:00
Tony Allevato
4d16c43d21 Add -index-include-locals flag.
When this flag is passed, index store data emitted during a build
will also contain definitions/references for local symbols.
2022-07-11 14:08:07 -07:00
David Goldman
b9391289e1 Also fetch proper module name for system modules 2022-06-02 10:02:01 -04:00
David Goldman
65062c9261 Capture the proper module name even if the module indexing is disabled 2022-05-17 19:41:28 -04:00
David Goldman
63c88b97f1 Add -index-ignore-clang-modules flag
This flag avoids indexing import clang modules (pcms), behaving
similar to `-index-ignore-system-modules` except for PCMs.
2022-05-17 19:41:26 -04:00
David Goldman
c232ed2913 Support hermetic indexing information
Swiftc port of https://github.com/apple/llvm-project/pull/4207.

This introduces a new flag, `-file-prefix-map` which can be used
instead of the existing `-debug-prefix-map` and `-coverage-prefix-map`
flags, and also remaps paths in index information currently.
2022-05-16 11:00:14 -04:00
Ellie Shin
0ff713d44c [Indexing] Use module real name in case module aliasing is used
Resolves rdar://82896373
2021-12-07 02:22:36 -08:00
Brent Royal-Gordon
b440ab7331 [NFC] Move several types/functions to Import.h
To help consolidate our various types describing imports, this commit moves the following types and methods to Import.h:

* ImplicitImports
* ImplicitStdlibKind
* ImplicitImportInfo
* ModuleDecl::ImportedModule
* ModuleDecl::OrderImportedModules (as ImportedModule::Order)
* ModuleDecl::removeDuplicateImports() (as ImportedModule::removeDuplicates())
* SourceFile::ImportFlags
* SourceFile::ImportOptions
* SourceFile::ImportedModuleDesc

This commit is large and intentionally kept mechanical—nothing interesting to see here.
2020-10-09 18:57:07 -07:00
Varun Gandhi
d3369f7711 [NFC] Rename ImportFilterKind cases to be clearer. 2020-09-23 12:50:20 -07:00
Martin Boehme
d806ba53f6 Give OptionSet an initializer_list constructor.
This makes it easier to specify OptionSet arguments.

Also modify appropriate uses of ModuleDecl::ImportFilter to take
advantage of the new constructor.
2020-06-22 06:57:29 +02:00
Dave Lee
4e227f8a76 [Index] Apply -index-ignore-system-modules to clang modules 2020-05-26 10:45:52 -07:00
Daniel Sweeney
ea526c6383 Converting ModuleDecl::ImportedModule from std::pair to a dedicated struct. (#31360) 2020-04-30 20:26:03 -07:00
Nathan Hawes
25d531c842 Merge pull request #30856 from nathawes/dont-process-group-name
[Index] Don't replace ' ' or '-' in group names when mapping them to a 'module' name
2020-04-08 10:42:25 -07:00
Dan Zheng
c834696bfa Add SynthesizedFileUnit.
`SynthesizedFileUnit` is a container for synthesized declarations. Currently, it
only supports module-level declarations.

It is used by the SIL differentiation transform, which generates implicit struct
and enum declarations.
2020-04-07 18:29:26 -07:00
Nathan Hawes
4592c0ad2d [Index] Don't replace ' ' or '-' in group names when mapping them to a 'module' name.
The fake module names we create when indexing a Swift module divided into
'groups' (e.g. the stdlib) don't have to be valid Swift module names, but we
were manipulating them to be so. E.g. the stdlib "Lazy Views" group, became
"Lazy_Views". This name is displayed in some editors verbatim, and is also
used as input to some sourcekitd APIs (e.g. editor.open.interface) that only
work if the original group name is passed, rather than the processed one.

This patch stops mapping invalid module name characters like ' ' and '-' to '_'
so the original group name remains.

Resolves rdar://problem/57270811
2020-04-07 12:18:36 -07:00
Nathan Hawes
9f6f76308e Add method to ModuleDecl to lazily compute the underlying module for cross-import overlays
Also update code completion, indexing, interface generation and doc info to use it.
2020-04-03 16:04:32 -07:00
Nathan Hawes
d865b524f3 [Index] Make StringRef -> std::string conversions explicit.
Resovles rdar://problem/60944263
2020-03-27 11:11:11 -07:00
Nathan Hawes
a785fa6cee [Frontend][Index] Add frontend option to skip indexing the stdlib (for test performance)
Several tests related to indexing system modules were taking a considerable
amount of time (100+ seconds in the worst case) indexing the standard library.
This adds a frontend option to skip it and updates those tests to pass it.
2020-03-25 14:36:23 -07:00
Nathan Hawes
8b03b05f1d [Index] Add index support for cross import overlays.
Resolves rdar://problem/59445445
2020-03-25 10:04:41 -07:00
Ben Langmuir
f30102e959 Fix Ubuntu 16.04 master-next build
Two more pair of string conversion fixes. With this, I was able to build
swift master-next on Ubuntu 16.04 (didn't try anything downstream of
swift, and not sure if the tests pass).
2020-03-19 09:51:39 -07:00
Michael Forster
345a9147ec Explicit SmallString->std::string conversion.
This is only needed temporarily until commit d7049213 from upstream
LLVM hits the swift/master branch.
2020-03-13 19:16:49 +01:00
Fred Riss
259d78a350 Adapt to llvm.org StringRef API change 2020-03-13 19:08:22 +01:00
Brent Royal-Gordon
f8df2f66fd Add mechanism for per-SourceFile overlays
When a “separately imported overlay” is added to a SourceFile, two things happen:

1. The direct import of the underlying module is removed from getImports*() by default. It is only visible if the caller passes ImportFilterKind:: ShadowedBySeparateOverlay. This means that non-module-scoped lookups will search _OverlayModule before searching its re-export UnderlyingModule, allowing it to shadow underlying declarations.

2. When you ask for lookupInModule() to look in the underlying module in that source file, it looks in the overlays instead. This means that UnderlyingModule.foo() can find declarations in _OverlayModule.
2020-02-18 11:07:12 -08:00
swift-ci
a453eda2dc Merge remote-tracking branch 'origin/master' into master-next 2019-09-18 10:09:53 -07:00
Jordan Rose
8d7f1b7c5d [AST] Separate SourceFile from FileUnit.h
Like the last commit, SourceFile is used a lot by Parse and Sema, but
less so by the ClangImporter and (de)Serialization. Split it out to
cut down on recompilation times when something changes.

This commit does /not/ split the implementation of SourceFile out of
Module.cpp, which is where most of it lives. That might also be a
reasonable change, but the reason I was reluctant to is because a
number of SourceFile members correspond to the entry points in
ModuleDecl. Someone else can pick this up later if they decide it's a
good idea.

No functionality change.
2019-09-17 17:54:41 -07:00
Jonas Devlieghere
b4d268e9e1 Migrate llvm::make_unique to std::make_unique
Now that we've moved to C++14, we no longer need the llvm::make_unique
implementation from STLExtras.h. This patch is a mechanical replacement
of (hopefully) all the llvm::make_unique instances in the swift repo.
2019-08-15 11:32:39 -07:00
Jonas Devlieghere
257dbcd49d Update Swift for the FileManager API change
Change Swift to work with the new FileManager API. It now returns and
ErrorOr<FileEntry*> instead of just the raw pointer.
2019-08-02 10:20:40 -07:00
Argyrios Kyrtzidis
3332b37d00 [Index/SourceKit] Remove the code related to calculating a module hash from the indexing walker
This has been an unnecessary code path for a long time now and should be removed particularly because it triggers wasteful `stat` calls.

rdar://51523161
2019-06-21 17:09:12 -07:00
Jordan Rose
0ba6c495ba Add @_implementationOnly
This is an attribute that gets put on an import in library FooKit to
keep it from being a requirement to import FooKit. It's not checked at
all, meaning that in this form it is up to the author of FooKit to
make sure nothing in its API or ABI depends on the implementation-only
dependency. There's also no debugging support here (debugging FooKit
/should/ import the implementation-only dependency if it's present).

The goal is to get to a point where it /can/ be checked, i.e. FooKit
developers are prevented from writing code that would rely on FooKit's
implementation-only dependency being present when compiling clients of
FooKit. But right now it's not.

rdar://problem/48985979
2019-03-28 15:57:53 -07:00
Jordan Rose
9ed3fe061d Change ModuleDecl::getImportedModules to take an option set
...in preparation for me adding a third kind of import, making the
existing "All" kind a problem. NFC, except that I did rewrite the
ClangModuleUnit implementation of getImportedModules to be simpler!
2019-03-28 14:44:41 -07:00
Adrian Prantl
d63debeb60 Experimental: Extend ClangImporter to import clang modules from DWARF
When debugging Objective-C or C++ code on Darwin, the debug info
collected by dsymutil in the .dSYM bundle is entirely
self-contained. It is possible to debug a program, set breakpoints and
print variables even without having the complete original source code
or a matching SDK available. With Swift, this is currently not the
case. Even though .dSYM bundles contain the binary .swiftmodule for
all Swift modules, any Clang modules that the Swift modules depend on,
still need to be imported from source to even get basic LLDB
functionality to work. If ClangImporter fails to import a Clang
module, effectively the entire Swift module depending on it gets
poisoned.

This patch is addressing this issue by introducing a ModuleLoader that
can ask queries about Clang Decls to LLDB, since LLDB knows how to
reconstruct Clang decls from DWARF and clang -gmodules producxes full
debug info for Clang modules that is embedded into the .dSYM budle.

This initial version does not contain any advanced functionality at
all, it merely produces an empty ModuleDecl. Intertestingly, even this
is a considerable improvement over the status quo. LLDB can now print
Swift-only variables in modules with failing Clang depenecies, and
becuase of fallback mechanisms that were implemented earlier, it can
even display the contents of pure Objective-C objects that are
imported into Swift. C structs obviously don't work yet.

rdar://problem/36032653
2018-12-05 13:54:13 -08:00