We were waiting for the semaphores outside of the `messageHandlingQueue`, which means that we didn’t actually block the message handling queue until the previous request received a result, effectively rendering `--sync` useless.
Unfortuantely, we have a few potential out-of-order exeuction possibilities while we migrate everything else to also be asyncronous. But those should be low-probability issues that we can fix in follow-up commits, so I think it’s fine for now. All of these places are marked with `FIXME: (async)`
This is the prerequisite for making `SourceKitServer` an actor, which will mean that the `handle` methods will be `async`.
The current paradigm of returning from `handle` once we can guarantee that there’s no out-of-order execution and then returning the actual result via the callback that’s attached to `Request` is a little weird still. I am hoping to change this paradigm to return the actual result and have a callback function that `handle` can call to indicate that it’s ready to accept another message while guaranteeing in-order execution, essentially flipping the role of the return value and the closure callback. But that’s something to be done after the entire stack has been asyncificied.
When we switch `SourceKitServer`, `SwiftLanguageServer` etc. to be actors, we can’t rely on them to provide ordering guarantees anymore because Swift concurrency doesn’t provide any ordering guarantees.
What we should thus do, is to handle all messages on a serial queue on the `Connection` level. This queue will be blocked from handling any new messages until a message has been sufficiently handled to avoid out-of-order handling of messages. For sourcekitd, this means that
a request has been sent to sourcekitd and for clangd, this means that we have forwarded the request to clangd.
Note that this serial queue is not the main thread, so we will continue accepting data over stdin, just the handling of those messages is blocked.
Since Musl is sufficiently different from Glibc (see https://wiki.musl-libc.org/functional-differences-from-glibc.html), it requires a different import, which now should be applied to files that have `import Glibc` in them.
Musl is a low footprint libc that's used in Linux distributions such as Alpine Linux, which allows producing fairly small container images. Additionally, unlike Glibc, musl allows full static linking, meaning apps can be easily distributed to an arbitrary Linux distribution that may have a version of Glibc incompatible with the one that Swift is usually built with or no Glibc installed at all.
Workaround for SR-16097 to avoid hitting SR-16095: VSCode will send shutdown requests without a 'params' key. On macOS getting the superDecoder for a non-existent container returns an empty decoder, on Linux it throws `DecodingError.keyNotFound`, causing a crash.
Perform a targeted fix: If we don't have 'params' and the request is a ShutdownRequest, manually create a `ShutdownRequest` without any parameters.
rdar://91288093
This adjusts the sourcekit-lsp build to use static linking for the
internal libraries. It is not currently possible to build
SourceKitLSP.dll as that requires re-exporting the interfaces from the
consumed modules. However, this allows us to reduce the overall size of
the distribution of SourceKit-LSP by ~1 MiB and reduces the
distributed file set. The values here assume partial static linking of
swift-package-manager, which helps reduce the total size.
Before:
228,352 BuildServerProtocol.dll
1,773,056 LanguageServerProtocol.dll
114,688 LanguageServerProtocolJSONRPC.dll
49,152 LSPLogging.dll
262,656 SKCore.dll
54,784 SKSupport.dll
80,896 SKSwiftPMWorkspace.dll
150,528 SourceKitD.dll
645,632 SourceKitLSP.dll
70,144 sourcekit-lsp.exe
3,429,888 bytes
After:
2,416,640 sourcekit-lsp.exe
2,416,640 bytes
This will add an additional link request for dispatch and Foundation
libraries. These are really required on non-Darwin targets, and should
be satisfied either by the library search path or by explicitly
indicating where the dependencies can be found.
We receive a `SIGPIPE` if we write to a pipe that points to a crashed process. This in particular happens if the target of a `JSONRPCConnection` has crashed and we try to send it a message.
On Darwin, `DispatchIO` ignores `SIGPIPE` for the pipes handled by it, but that features is not available on Linux.
Instead, globally ignore `SIGPIPE` on Linux to prevent us from crashing if the `JSONRPCConnection`'s target crashes.
Fixes rdar://75580936
This actually addresses the real issue that was ignored earlier about
pipes on Windows. The FileHandle cannot provide a non-owning file
descriptor (the returned file descriptor would need to be explicitly
`_close`'d by the receiver). Foundation now vends a `_handle` accessor
to the OS primitive handle. Use this to create the dispatch loop for
messaging. We now create the JSONRPCConnection from handles on Windows
which actually should help enable running some of the tests on Windows
as well.
* Replaces two logs since those don't result in response messages anymore
* Adds a func for sending messages synchronously, to get the two messages out before fatalError
This adjusts the use of Dispatch to build on Windows. Windows does not
provide `stdout_fileno` and `stderr_fileno`. However, it is possible to
use `fileno` to get the associated fileno from the descriptor.
Dispatch on Windows does not deal with fd's but rather with handles.
Convert the file descriptor to a handle and pass that off to dispatch.
The handle is a non-owning reference, and should not be closed.
Fortunately, dispatch does not close the handle when the DispatchIO is
closed.
Specifically, we care that all outstanding **writes** are finished
before we call the close handler, because otherwise we may (a) send
corrupted output during shutdown, or (b) drop notifications and replies
sent during the shutdown process. The former is a potential issue for
clients that are not robust about parse failures, and the latter is an
issue for reproducibility and robustness during testing/debugging - in
particular, we have some integration tests that send data without
waiting for individual replies and they need to finish outstanding
replies before exiting.
rdar://60159448
Since the connection and message handler have a reciprocal need to know
about each other, move the closeHandler so that it has the opportunity
to call a method on the message handler if desired.
We will be able to split the LSP modules off later. These LSP modules
will provide the ability to write custom LSP servers and clients in
Swift. The sourcekit-lsp repository will build on top of this new
package to provide an LSP implementation that creates a language server
for Swift and C-based-languages.
This code was doing a sync dispatch to a queue it was already running
on. Instead, go through the same path as other close calls. This would
hang, or if you're lucky crash when dispatch detects the issue.
We implicitly close the connection when the input file descriptor is
closed, or if it has an error. But we also close things down explicitly
when we see the exit notification. We need to protect against a race
triggering us to call the close handler multiple times. This was
resulting in `sourcekit-lsp` racing to call `exit(0)`, which triggered a
double-free and abort on Linux.
When testing the sourcekit-lsp binary, it is handy to be able to force
requests to be handled synchronously. This only affects the protocol
layer, not the implementation. This option is hidden from the help text
and should only be used for testing/debugging.
We are tied to using a SwiftPM that matches the toolchain, so upgrade
from 0.3.0 to .branch("master") and add a pins file to manage updating
that dependency.
This was driven by changes that broke loading packages being developed
with newer version of SwiftPM when using the 0.3.0 tag of libSwiftPM.
However, the changes seem to work when going in the other direction, so
using the newer libSwiftPM hasn't caused any known regressions for using
older toolchain versions (not guarantteed and not tested extensively
though).
This fixes using the latest (November 13) toolchain snapshot on macOS.
On Linux there are other issues not specific to LSP.
