Commit Graph

840 Commits

Author SHA1 Message Date
Alex Hoppen
729e0e23fc [libSyntax] Add logging for how long syntax tree serialization took 2018-08-02 14:32:12 -07:00
Alex Hoppen
85f1c5e7a1 [incrParse] Fix issue because of which the syntax tree always got serialized completely 2018-08-02 14:31:35 -07:00
Alex Hoppen
082a9878d1 [SourceKit] Reserve 4KB for the syntax tree buffer
Heuristics have shown that this prevents several new mallocs and speeds
incremental syntax tree transfer up.
2018-08-02 14:30:58 -07:00
Alex Hoppen
15b1559e06 [SourceKit] Move syntax tree serialization to separate function
This will allow us to experiment with other serialization formats in the
future more easily.
2018-08-02 14:30:58 -07:00
Alex Hoppen
99b1ce7ab7 Merge pull request #18416 from ahoppen/editorconsumer-return-values
[SourceKit] Change return value of functions in EditorConsumer to void
2018-08-02 09:53:44 -07:00
Alex Hoppen
38c62cf766 Merge pull request #18426 from ahoppen/sourcekit-raw-data
[sourcekitd] Add support for transferring raw data
2018-08-02 09:52:41 -07:00
Rintaro Ishizaki
335e913bed Merge pull request #18439 from rintaro/sourcekit-editoffset-rdar34206143
[SourceKit] Defensive guard for invalid offset and length in edit request
2018-08-02 10:19:33 +09:00
Rintaro Ishizaki
399fece7f5 [SourceKit] Dedicated error message for 'offset' + 'length' out of range 2018-08-02 09:29:28 +09:00
Alex Hoppen
9bbe441160 [SourceKit] Change return value of functions in EditorConsumer to void
We were always returning true from those functions in  SKEditorConsumer
and false in the test consumers. On the client side we would then ignore
the return value. So it's clearer to have the functions not return
anything.
2018-08-01 15:17:13 -07:00
Alex Hoppen
d920a664be [sourcekitd] Add support for transferring raw data
We will need this to transfer a binary encoded syntax tree in the future
2018-08-01 14:34:30 -07:00
Alex Hoppen
a38d7c525e [SourceKit] Fix compile error on Ubuntu 14 2018-08-01 10:30:16 -07:00
Rintaro Ishizaki
930d4342cd [SourceKit] Defensive guard for invalid offset and length in edit request
Otherwise, hits assertion, or crashes in no-assertion build.

Added 'EditableTextBuffer::getSize()' for getting size after previous updates
without actually applying them.

rdar://problem/34206143
2018-08-01 20:02:15 +09:00
Alex Hoppen
b48f740712 [SourceKit] Serialize the syntax tree in the EditorConsumer
This will allow us to switch to a more efficient serialization format in
the future.
2018-07-31 13:51:16 -07:00
Alex Hoppen
749a13a6fe [libSyntax] Remove the C++ SyntaxClassifier
The recommended way forward is to use the SyntaxClassifier on the Swift
side.

By removing the C++ SyntaxClassifier, we can also eliminate the
-force-libsyntax-based-processing option that was used to bootstrap
incremental parsing and would generate the syntax map from a syntax
tree.
2018-07-30 09:26:18 -07:00
Alex Hoppen
4d16a64651 Merge pull request #18238 from ahoppen/01b-fix-compile-error
[SourceKit] Fix compile error
2018-07-25 17:46:00 -07:00
Alex Hoppen
7ad81a88b9 [SourceKit] Fix compile error 2018-07-25 16:31:40 -07:00
Alex Hoppen
c8fc286428 Merge pull request #18152 from ahoppen/01-incr-tree-transfer
[libSyntax] Incremental syntax tree transfer
2018-07-25 15:26:57 -07:00
John McCall
7a4aeed570 Implement generalized accessors using yield-once coroutines.
For now, the accessors have been underscored as `_read` and `_modify`.
I'll prepare an evolution proposal for this feature which should allow
us to remove the underscores or, y'know, rename them to `purple` and
`lettuce`.

`_read` accessors do not make any effort yet to avoid copying the
value being yielded.  I'll work on it in follow-up patches.

Opaque accesses to properties and subscripts defined with `_modify`
accessors will use an inefficient `materializeForSet` pattern that
materializes the value to a temporary instead of accessing it in-place.
That will be fixed by migrating to `modify` over `materializeForSet`,
which is next up after the `read` optimizations.

SIL ownership verification doesn't pass yet for the test cases here
because of a general fault in SILGen where borrows can outlive their
borrowed value due to being cleaned up on the general cleanup stack
when the borrowed value is cleaned up on the formal-access stack.
Michael, Andy, and I discussed various ways to fix this, but it seems
clear to me that it's not in any way specific to coroutine accesses.

rdar://35399664
2018-07-23 18:59:58 -04:00
Alex Hoppen
57196f8902 [libSyntax] Enable serialization of syntax trees for incremental transfer 2018-07-23 12:32:49 -07:00
Slava Pestov
5213f80e7e IDE: Remove uses of AbstractFunctionDecl::getParameterLists() 2018-07-22 20:56:56 -07:00
Alex Hoppen
c8226d1507 [libSyntax] Make a typealias to unsigned to represent SyntaxNodeIds 2018-07-19 13:57:08 -07:00
Alex Hoppen
419ba044f1 [libSyntax] Record reused node IDs
This is cheaper than recording reused region offsets and the reused node
IDs will later be used to incrementally transfer the syntax to
SwiftSyntax.
2018-07-19 13:55:57 -07:00
Alex Hoppen
705f5b79a2 [libSyntax] Rename getAbsolutePosition-related methods for more clarity 2018-07-19 09:15:53 -07:00
Alex Hoppen
1b8840fcff [incrParse] Perform a full reparse of the file if needed for formatting 2018-07-18 13:35:11 -07:00
Alex Hoppen
57ccdd89b6 [incrParse] Add validation of incremental parsing
If enabled using the environment variable
SOURCEKIT_INCREMENTAL_PARSE_VALIDATION, the incrementally parsed syntax
tree will be compared to the from-scratch parsing syntax tree. If they
differ a warning is emitted and log files showing the difference written
to a temporary directory.
2018-07-18 13:35:11 -07:00
Alex Hoppen
6635be10ea [libSyntax] Add UserInfo to json::Output
This will allow us to customize the serialization of a syntax tree like
not serializing the node's IDs.
2018-07-18 13:35:10 -07:00
Alex Hoppen
7a750bf024 [libSyntax] Allow syntax cache reuse info to be passed back via SourceKit 2018-07-18 13:35:10 -07:00
Alex Hoppen
fcc5a6b424 [libSyntax] Enable incremental parsing for syntax tree based syntax coloring 2018-07-17 14:17:58 -07:00
Alex Hoppen
6bc1b5a389 [libSyntax] Add test variants for building the syntax map via libSyntax 2018-07-13 17:57:00 -07:00
Alex Hoppen
c8a3957ba2 [SourceKit] Add option to force the SyntaxMap to be generated via the syntax tree 2018-07-13 17:57:00 -07:00
Rintaro Ishizaki
7b12cfbfa6 [SourceKit] Respect -name option in sourcekitd-test 2018-07-05 16:11:51 +09:00
Rintaro Ishizaki
2b0728bc95 [SourceKit] Use "-" as fallback complier argument
Since compiler arguments are processed by Driver, arbitrary filename
passed as 'request.name' (e.g. "", "-1", etc.) may cause problems.
Using '-' guarantees successful initialization of 'CompilerInvocation'.

rdar://problem/40646921
rdar://problem/40955808
2018-07-05 16:11:51 +09:00
Alex Hoppen
6e12a634a1 [SourceKit] Move all options into a common options struct
With more options coming for incremental syntax parsing, the list of
arguments will grow way to large and unhandy, so just extract them into
one common struct.
2018-06-28 20:26:05 -07:00
Argyrios Kyrtzidis
4179ceb4f9 Add a comment for the AddSwiftSourceKit.cmake change 2018-06-26 10:57:01 -07:00
Argyrios Kyrtzidis
7f6afa0357 [SourceKit/CMake] Make sure the XPC service executable isn't linked with an absolute build directory rpath
rdar://40462455
2018-06-26 10:47:11 -07:00
Ben Langmuir
0b795c19db Merge pull request #17241 from benlangmuir/pop-less
[sourcekit] Do not dequeue AST consumers expecting newer snapshots
2018-06-19 10:33:06 -07:00
Vedant Kumar
d910e5a5c2 [SourceKit] Disable expensive SIL options (#17306)
Disable the sanitizers and code coverage when building a swift
invocation for the purpose of collecting diagnostics.

This should speed up diagnostic generation and reduce exposure to
compiler bugs.

rdar://40955900
2018-06-18 14:43:27 -07:00
Ben Langmuir
68b829a065 [sourcekit] Do not dequeue AST consumers expecting newer snapshots
When adding to the AST consumer queue, keep track of the snapshots
expected and only run such AST consumers when a new enough AST is built.
Progress is ensured because we always run the AST consumer that
triggered the build.

This prevents cases where enqueuing a consumer during an AST build has
different behaviour than enqueuing it after the AST has finished.

rdar://40340631
2018-06-18 13:01:23 -07:00
swift-ci
a700e235a6 Merge pull request #17191 from CodaFi/tuplet-rudiments 2018-06-15 15:19:21 -07:00
Robert Widmann
f208886a32 [Experiment] Drop null input type check 2018-06-15 11:48:36 -07:00
John McCall
e97fde0a32 Merge pull request #17212 from rjmccall/generalize-accessor-kinds
Rename accessor kinds from IsGetter -> IsGet, etc.
2018-06-14 18:06:39 -04:00
John McCall
9022b5152f Rename accessor kinds from IsGetter -> IsGet, etc.
Introduce some metaprogramming of accessors and generally prepare
for storing less-structured accessor lists.

NFC except for a change to the serialization format.
2018-06-14 17:08:55 -04:00
Jordan Rose
fb8c815c80 [SourceKit] Enable PrettyStackTrace printing (#17205)
In particular, this will result in PrettyStackTrace messages being
included in crash logs when SourceKit crashes on Apple platforms.
2018-06-14 13:54:58 -07:00
Robert Widmann
1beb75583a Push FunctionTypeRepr Input Invariants Up
Validation of the input side of FunctionTypeRepr was previously being done in Sema because of expression folding.  If we instead push the invariant that the input TypeRepr should always be a TupleTypeRepr into the AST a number of nice cleanups fall out:

- The SIL Parser no longer accepts Swift 2-style type declarations
-  Parse is more cleanly able to reject invalid FunctionTypeReprs
- Clients of the AST can be assured the input type is always a TupleType so we can flush Swift 2 hacks
2018-06-13 18:41:20 -07:00
Ben Langmuir
8902562e24 [sourcekit] Fix a race in sourcekitdInProc's notification handling
Setting the notification handler was racing with calling it; now both
happen on the same queue. Only affected the in-process sourcekitd.
2018-06-13 11:32:18 -07:00
Ben Langmuir
a88c73fe60 [sourcekit] Fix non-deterministic failure in CompileNotifications tests
This is a test-only change except for the introduction of a new request
that is only used by tests.

Our notifications are dispatched in the XPC event handler, which is not
synchronized with replies to explicit XPC send_message_with_reply calls.
This is fine for most users of sourcekitd, since the notifications are
already enqueued on the client side, but for testing we need a way to
guarantee that all notifications are passed to the client-side handler
before we exit. This commit introduces a new request for testing that
triggers a notification, allowing the client to wait on that
notification to ensure all previously posted notifications have been
handled.

Note: the non-deterministic test failures can be triggered by adding a
sleep of ~100 ms in the event handler before the notification is
dispatched to the main queue.

rdar://40311995
2018-06-13 11:32:18 -07:00
Xi Ge
e7e16ff12e Merge pull request #16927 from nkcsgexi/parse-buffer
SwiftSourceKit: add API to generate a syntax tree from a given source buffer.
2018-05-31 18:26:51 -07:00
Xi Ge
d4b74065f6 SwiftSourceKit: add an API to parse a given source buffer to generate a syntax tree. 2018-05-31 16:43:23 -07:00
Rintaro Ishizaki
66c71d1bf7 [SourceKit] Force print '?' for optional method call in code completion. (#16910)
Calling '@objc optional func' requires '?' or '!' after its name. When
completing method calls for them, 'key.sourcetext' should have '?'
whereas 'key.name' shouldn't.
Note that we deliberately do not use optional type name for
'key.typename'. This is consistent with optional chain '?.<propertyName>'
behavior.

rdar://problem/37904574
2018-05-31 16:06:23 -07:00
Ben Langmuir
c3e5d63384 [sourcekit] Send notification if compiler argument parsing fails in editor.open
Since we will not reach `createAST` where we would normally send the
notification, send one from `getInvocation` when it fails.

rdar://39225180
2018-05-29 15:24:20 -07:00