Use the std-equivalent names as the LLVM ones are now deprecated
(eventually `llvm::Optional` will disappear):
- `getValue` -> `value`
- `getValueOr` -> `value_or`
- `hasValue` -> `has_value`
Follow up from ab1b343dad and
7d8bf37e5e with some missing cases.
This enables the ability to cancel requests, which aren’t code completion requests, again.
Previous crashes in SILGen are prevented by disabling cancellation during the SIL stages. Instead, we add dedicated cancellation checkpoints before and after SIL.
rdar://98390926
#58786 (rdar://93030932) was failing because the `swift-frontend` invocations passed a `swiftExecutablePath` to `Invocation.parseArgs`. This caused the `ClangImporter` instance to point to a `clang` binary next to the `swift-frontend` executable while SourceKit used PATH to find `clang`. The clang executable next to `swift-frontend` doesn’t actually exist because `clang` lives in `llvm-linux-aarch64/bin` and `swift-frontend` lives in `swift-linux-aarch64/bin`.
So some checks for a minimum clang verison failed for the normal build (because the executable doesn’t actually exists) while they pass during the SourceKit build (which used `clang` from `PATH`). This in turn caused the `outline-atomics` to be enabled to the SourceKit clang compiler arguments but not the clang compiler arguments for a normal build and thus resulted in two separate module cache directories (which includes the enabled features in the module directory hash).
To fix this issue, also set the swift executable path for compiler invocations created from SourceKit.
Fixes#58786 (rdar://93030932)
We need to run SILGen for diagnostics (to actually get all diagnostics).
All non-completion requests share an AST and thus they too run SILGen.
Any lazy typechecking run in SILGen assumes that it succeeds.
Cancellation can cause typechecking to fail here though, since we simply
check the flag and error if it's set. This unfortunately has the ability
to cause any any number of crashes since various invariants in SILGen
are then broken.
Disable cancellation of in-flight non-completion requests for now until
we have a proper fix in place.
Resolves rdar://91251017.
Previously, `SwiftASTManager` and `SlowRequestSimulator` maintained their own list of in-progress cancellation tokens. With code completion cancellation coming up, there would need to be yet another place to track in-progress requests, so let’s centralize it.
While at it, also support cancelling requests before they are scheduled, eliminating the need for a `sleep` in a test case.
The current implementaiton leaks tiny amounts of memory if a request is cancelled after if finishes. I think this is fine because it is a pretty nieche case and the leaked memory is pretty small (a `std::map` entry pointing to a `std::function` + `bool`). Alternatively, we could require the client to always dispose of the cancellation token manually.
The key changes here are
- To keep track of cancellation tokens for all `ScheduledConsumer`s in `SwiftASTManager`
- Generate unique request handles for all incoming requests (`create_request_handle `), use these request handles as cancellation tokens and return them from the `sourcekitd_send_request` methods
- Implement cancellation with `sourcekitd_cancel_request` as the entry point and `SwiftASTManager::cancelASTConsumer` as the termination point
Everything else is just plumbing the cancellation token through the various abstraction layers.
rdar://83391505
This commit refactors the way ASTs are being built in SourceKit and how `SwiftASTConsumer`s are served by the built ASTs. `SwiftASTManager.h` should give an overview of the new design.
This commit does not change the cancellation paradigm in SourceKit (yet). That is, subsequent requests with the same `OncePerASTToken` still cancel previous requests with the same token. But while previously, we were only able to cancel requests that haven’t started an AST build yet, we can now also cancel the AST build of the to-be-cancelled requests.
With this change in place, we can start looking into explicit cancellation of requests or other cancellation paradigms.
We were only keeping track of `RawSyntax` node IDs to incrementally transfer a syntax tree via JSON. However, AFAICT the incremental JSON transfer option has been superceeded by `SyntaxParseActions`, which are more efficient.
So, let’s clean up and remove the `RawSyntax` node ID and JSON incremental transfer option.
In places that still need a notion of `RawSyntax` identity (like determining the reused syntax regions), use the `RawSyntax`’s pointer instead of the manually created ID.
In `incr_transfer_round_trip.py` always use the code path that uses the `SyntaxParseActions` and remove the transitional code that was still using the incremental JSON transfer but was never called.
Cursor info for a constructor would previously give the cursor info for
the containing type only. It now also adds cursor info for the
constructor itself in a "secondary_symbols" field.
Refactor `passCursorInfoForDecl` to use a single allocator rather than
keeping track of positions in a buffer and assigning everything at the
end of the function.
Refactor the various available refactoring gathering functions to take a
SmallVectorImpl and to not copy strings where they don't need to.
Resolves rdar://75385556
Adds a new 'key.retrieve_symbol_graph' option to the request. When set to 1 it
includes the JSON for a SymbolGraph containing a single node for the symbol at
the requested position.
This also extends the SymbolGraph library with a new entry point to get a graph
for a single symbol, and to additionally support type substitution to match the
existing CursorInfo behavior (e.g. so that when invoked on `first` in
`Array<Int>().first`, the type is given as `Int?` rather than `Element?`).
Resolves rdar://problem/70551509
The VFS tests were using Unix absolute paths, which does not play well
when Windows see them as relative to the current drive letter.
By using the temporal directory, both Windows and Unix can use the same
paths and avoid the problem.
Additionally, a couple of inputs have to be transformed into the native
path format, because sourcekitd-test compares the inputs as strings, and
they need to match exactly. So the source file and the name of the VFS
entries are transformed into native using the helper from LLVM support.
Previously, requests would fail silently by returning an empty struct
in the response.
With this change, responses will properly report fail with the internal
error.
Previously, requests would fail silently by returning an empty struct
in the response.
With this change, responses will properly report fail with the internal
error.
When the server shuts down we may still have outstanding async work that
can attempt to trigger a notification, so use a shared_ptr + weak_ptr
instead of unique_ptr + unowned references.
We were always returning true from those functions in SKEditorConsumer
and false in the test consumers. On the client side we would then ignore
the return value. So it's clearer to have the functions not return
anything.
The recommended way forward is to use the SyntaxClassifier on the Swift
side.
By removing the C++ SyntaxClassifier, we can also eliminate the
-force-libsyntax-based-processing option that was used to bootstrap
incremental parsing and would generate the syntax map from a syntax
tree.
The problem appears to be that cursor info is using an unsound mechanism
(canUseASTWithSnapshots) to run without rebuilding the AST and this test
was triggering it unintentionally. By changing the order of the edits we
force cursor info to notice the tokens have changed only after the code
is semantically correct again. In real code this issue is rare and
solves itself the next time you invoke the cursor info.
rdar://42247603
With more options coming for incremental syntax parsing, the list of
arguments will grow way to large and unhandy, so just extract them into
one common struct.
When adding to the AST consumer queue, keep track of the snapshots
expected and only run such AST consumers when a new enough AST is built.
Progress is ensured because we always run the AST consumer that
triggered the build.
This prevents cases where enqueuing a consumer during an AST build has
different behaviour than enqueuing it after the AST has finished.
rdar://40340631
The enhanced SourceKitd requests are EditorOpen and EdtiorReplaceText. In these two requests, the clients can specify a flag "key. enablesyntaxtree = 1" to get a serialize libSyntax tree with the response.
To help this integration, we added a function in SyntaxParsingContext to explicitly finalize the creation of a SourceFileSyntax to incorporate the fact that SourceKit needs the tree before its destroying the parser instance.
To test this integration, we diff the syntax tree serialized from the frontend action and the tree serialized from a SourceKitd response. They should be identical.
The OncePerASTToken machinery lets us automatically cancel "stale"
requests after a new one comes in. This avoid wasting time processing
requests that have been superceded, which is common for cursor-info, but
sometimes you really want to get results even later, so this commit adds
a way to opt out of the cancellation.
Incidentally, disable cancellation of name translation, which doesn't
really make sense and no one should be relying on that.
rdar://problem/31905379
- Add CompilerInvocation::getPCHHash
This will be used when creating a unique filename for a persistent
precompiled bridging header.
- Automatically generate and use a precompiled briding header
When we're given both -import-objc-header and -pch-output-dir
arguments, we will try to:
- Validate what we think the PCH filename should be for the bridging
header, based on the Swift PCH hash and the clang module hash.
- If we're successful, we'll just use it.
- If it's out of date or something else is wrong, we'll try to
emit it.
- This gives us a single filename which we can `stat` to check for the
validity of our code completion cache, which is keyed off of module
name, module filename, and module file age.
- Cache code completion results from imported modules
If we just have a single .PCH file imported, we can use that file as
part of the key used to cache declarations in a module. Because
multiple files can contribute to the __ObjC module, we've always given
it the phony filename "<imports>", which never exists, so `stat`-ing it
always fails and we never cache declarations in it.
This is extremely problematic for projects with huge bridging headers.
In the case where we have a single PCH import, this can bring warm code
completion times down to about 500ms from over 2-3s, so it can provide a
nice performance win for IDEs.
- Add a new test that performs two code-completion requests with a bridging header.
- Add some -pch-output-dir flags to existing SourceKit tests that import a bridging
header.
rdar://problem/31198982
