Enqueuing `SwiftASTConsumer`s might be expensive because `getBuildOperationForConsumer` consults the file system. Since all results from the AST build are processed asynchronously anyway, there’s no need to perform the enqueuing synchronously.
rdar://86289703
Previously, `SwiftASTManager` and `SlowRequestSimulator` maintained their own list of in-progress cancellation tokens. With code completion cancellation coming up, there would need to be yet another place to track in-progress requests, so let’s centralize it.
While at it, also support cancelling requests before they are scheduled, eliminating the need for a `sleep` in a test case.
The current implementaiton leaks tiny amounts of memory if a request is cancelled after if finishes. I think this is fine because it is a pretty nieche case and the leaked memory is pretty small (a `std::map` entry pointing to a `std::function` + `bool`). Alternatively, we could require the client to always dispose of the cancellation token manually.
The key changes here are
- To keep track of cancellation tokens for all `ScheduledConsumer`s in `SwiftASTManager`
- Generate unique request handles for all incoming requests (`create_request_handle `), use these request handles as cancellation tokens and return them from the `sourcekitd_send_request` methods
- Implement cancellation with `sourcekitd_cancel_request` as the entry point and `SwiftASTManager::cancelASTConsumer` as the termination point
Everything else is just plumbing the cancellation token through the various abstraction layers.
rdar://83391505
This commit refactors the way ASTs are being built in SourceKit and how `SwiftASTConsumer`s are served by the built ASTs. `SwiftASTManager.h` should give an overview of the new design.
This commit does not change the cancellation paradigm in SourceKit (yet). That is, subsequent requests with the same `OncePerASTToken` still cancel previous requests with the same token. But while previously, we were only able to cancel requests that haven’t started an AST build yet, we can now also cancel the AST build of the to-be-cancelled requests.
With this change in place, we can start looking into explicit cancellation of requests or other cancellation paradigms.
SwiftSourceInfo files provide source location information for decls coming from
loaded modules. For most IDE use cases it either has an undesirable impact on
performance with no benefit (code completion), results in stale locations being
used instead of more up-to-date indexer locations (cursor info), or has no
observable effect (live diagnostics, which are filtered to just those with a
location in the primary file).
For non-IDE clients of SourceKit though, cursor info providing declaration
locations for symbols from other modules is useful, so add a global
configuration option (and a new request to set it) to control whether
.swiftsourceinfo files are loaded or not based on use case (they are loaded by
default).
The invocation options are not an appropriate place to put this state,
since it can change between requests. This moves it to the editor
document, allowing us to change the specific VFS instance without
causing a rebuild (unless the contents/timestamps for a dependency
change).
When the server shuts down we may still have outstanding async work to
build an AST, so use a shared_ptr + weak_ptr instead of unique_ptr +
unowned references.
Stop parsing frontend arguments directly and use the driver instead. The
most intersting part of this change is that it forces us to consider
whether our compiler invocation will have inputs or not. We have
several kinds of requests that need to create a compiler instance, but
not parse any inputs (interface-generation, doc-info, and indexing when
operating on a module instead of source files).
Incidentally, add an error when trying to do doc-info on multiple source
files. This was already very broken (assertion failures and bogus source
locations), so add an error for it.
rdar://problem/17897287
... and add a few basic statistics about the number of requests, ASTs
built, etc. The Statistic type is loosely based on the one from LLVM,
but suitable for using without DEBUG macros and using SourceKit UIdents
to identify the statistic. The easiest way to add a new statistic is to
add it to SwiftStatistics.def in the SwiftLangSupport.
Cursor info requires access to the underlying AST, which is not
thread-safe. This manifest as crashes when performing concurrent
cursor-info requests on the same generated interface. We already
prevented concurrent cursor-infos on regular Swift files by using the
ASTManager, but generated interfaces use the InterfaceGenContext which
may use either an ASTUnit or its own internal CompilerInstance.
rdar://problem/27311624
The code goes into its own sub-tree under 'tools' but tests go under 'test',
so that running 'check-swift' will also run all the SourceKit tests.
SourceKit is disabled on non-darwin platforms.
