Commit Graph

168 Commits

Author SHA1 Message Date
John McCall
7a4aeed570 Implement generalized accessors using yield-once coroutines.
For now, the accessors have been underscored as `_read` and `_modify`.
I'll prepare an evolution proposal for this feature which should allow
us to remove the underscores or, y'know, rename them to `purple` and
`lettuce`.

`_read` accessors do not make any effort yet to avoid copying the
value being yielded.  I'll work on it in follow-up patches.

Opaque accesses to properties and subscripts defined with `_modify`
accessors will use an inefficient `materializeForSet` pattern that
materializes the value to a temporary instead of accessing it in-place.
That will be fixed by migrating to `modify` over `materializeForSet`,
which is next up after the `read` optimizations.

SIL ownership verification doesn't pass yet for the test cases here
because of a general fault in SILGen where borrows can outlive their
borrowed value due to being cleaned up on the general cleanup stack
when the borrowed value is cleaned up on the formal-access stack.
Michael, Andy, and I discussed various ways to fix this, but it seems
clear to me that it's not in any way specific to coroutine accesses.

rdar://35399664
2018-07-23 18:59:58 -04:00
Alex Hoppen
c8226d1507 [libSyntax] Make a typealias to unsigned to represent SyntaxNodeIds 2018-07-19 13:57:08 -07:00
Alex Hoppen
419ba044f1 [libSyntax] Record reused node IDs
This is cheaper than recording reused region offsets and the reused node
IDs will later be used to incrementally transfer the syntax to
SwiftSyntax.
2018-07-19 13:55:57 -07:00
Alex Hoppen
705f5b79a2 [libSyntax] Rename getAbsolutePosition-related methods for more clarity 2018-07-19 09:15:53 -07:00
Alex Hoppen
57ccdd89b6 [incrParse] Add validation of incremental parsing
If enabled using the environment variable
SOURCEKIT_INCREMENTAL_PARSE_VALIDATION, the incrementally parsed syntax
tree will be compared to the from-scratch parsing syntax tree. If they
differ a warning is emitted and log files showing the difference written
to a temporary directory.
2018-07-18 13:35:11 -07:00
Alex Hoppen
8430eff670 [libSyntax] Add syntax coloring based on the syntax tree 2018-07-13 17:48:47 -07:00
Alex Hoppen
9d59cd286b [incrParse] Add a stable id to the syntax nodes
The id is meant to be stable across incremental parses
2018-07-13 16:56:03 -07:00
John McCall
3247232aa3 Remove some gratutious uses of GCC extensions from the Syntax library.
Naming the bit-field structs is a significant readability improvement
because it's very clear that you shouldn't touch e.g. Bits.Token
without having checked/asserted that you're in a token case.

The assertions are all in statement context (which was obvious
because the NDEBUG versions all included semicolons), so there's no
reason not to use the traditional `do { } while (false)` trick instead
of a statement-expression.

This also clears up some warnings in atypical build configurations.
2018-06-30 03:59:49 -04:00
Robert Widmann
b7580493f5 [Gardening] 'throw' is not an expression keyword
Remove the workaround in CodeCompletion for this.
2018-06-15 13:25:06 -07:00
Alex Hoppen
15b2bae80a [libSyntax] Improve syntax related dump functions 2018-05-22 09:07:55 -07:00
Alex Hoppen
b2ebc96510 [incrParse] Reparse a node if the next leaf node has been modified 2018-05-22 09:07:55 -07:00
Alex Hoppen
a137e0da7d [libSyntax] Omit unknown nodes if they do not have any children 2018-05-22 08:52:40 -07:00
Alex Hoppen
e1a99efd57 [incrParse] Fix parsing of nodes covering no source text 2018-05-22 08:52:39 -07:00
Alex Hoppen
ec4a527c44 [incrParse] Reparse nodes if the next node's trailing trivia has changed 2018-05-22 08:52:36 -07:00
Alex Hoppen
92f8f34d22 [incrParse] Store reused regions and output them after parsing 2018-05-22 08:52:34 -07:00
Alex Hoppen
60d11d24f8 [incrParse] Reparse a node if the next leaf node has been modified 2018-05-22 08:52:33 -07:00
Alex Hoppen
de9737c946 [incrParse] Support incremental parsing for edited files 2018-05-22 08:52:33 -07:00
Alex Hoppen
f8cd1ca749 [libSyntax] Compute the text length of every node on the fly 2018-05-22 08:52:32 -07:00
Alex Hoppen
0eb16d6799 [libSyntax] Update space needed for raw syntax bits
The bit for ManualMemory was not taken care of previously
2018-05-22 08:52:32 -07:00
Alex Hoppen
f369d5d7d4 [libSyntax] Add static_asserts for the size of RawSyntaxBits 2018-05-22 08:52:31 -07:00
Alex Hoppen
d832a38fda [libSyntax] Documentation improvements 2018-05-22 08:52:31 -07:00
Xi Ge
5a8053e7ef libSyntax: add getAbsoluteEndPosition() method to syntax nodes.
This implementation uses sibling's absolute start position to help
populate caches while getting the end position.
2018-05-01 12:06:41 -07:00
Xi Ge
c45d659e1f Merge pull request #16264 from nkcsgexi/cache-absolute-pos-c-plus
libSyntax: cache absolute positions on SyntaxData.
2018-04-30 20:39:37 -07:00
Huon Wilson
763072dfae [Syntax] std::function -> llvm::function_ref for some non-escaping params. 2018-05-01 08:29:06 +10:00
Xi Ge
7b4218c2f7 libSyntax: cache absolute positions on SyntaxData.
Aligning with what we did for SwiftSyntax, this patch uses caches for
absolute position calculation on the C++ side.
2018-04-30 15:09:00 -07:00
Dexin Li
e0f8b27117 [Syntax]Add a deserializer that convert json to libSyntax tree (#15203) 2018-03-16 15:22:04 -07:00
Rintaro Ishizaki
c73065e698 Merge pull request #15257 from rintaro/parse-eliminate-square_lit
[Parse] Eliminate {l,r}_square_lit tokens.
2018-03-15 17:39:55 +09:00
Xi Ge
a121ce65ca Syntax: add APIs to help syntax tree serialization. NFC (#15241) 2018-03-14 13:12:31 -07:00
Rintaro Ishizaki
bdcd87754d [Parse] Eliminate POUND_OLD_OBJECT_LITERAL tokens
We don't want to handle them in libSyntax
2018-03-14 21:50:53 +09:00
Rintaro Ishizaki
7237875870 [Parse] Eliminate square_lit token 2018-03-14 21:50:53 +09:00
omochimetaru
420291bbad [Syntax] isComment in trivia 2018-03-12 23:08:37 +09:00
Xi Ge
94c3f55117 libSyntax: extract meta-information of trivia kinds to syntax_gyb_support. NFC
The existing libSyntax infrastructure uses external python
dictionaries to share logic between C++ and Swift implementations.
This patch teaches trivia kinds to adapt to this infrastructure
 as well.
2018-03-06 17:45:43 -08:00
Saleem Abdulrasool
2e77e6a8b1 build: tweak gyb line directives
Make the quoted parameters more portable.  This is needed in order to
support building swift on Windows.  Simplify the declaration of the
flags and do a small formatting clean up.  This ensures that the ninja
generation gets the command line invocation correct.
2018-03-01 12:21:54 -08:00
Harlan
2a3d4cb598 Initial infrastructure for documenting SwiftSyntax API (#14701) 2018-02-26 13:43:11 -05:00
Joe Groff
d365c153d4 SIL: Introduce sil_property declarations for property descriptors.
This provides SILGen a place to generate the key path component information for an exported property so that it can be linked to from other modules.
2018-02-23 14:57:45 -08:00
John Fairhurst
677491fc9b [SourceKit] Add syntaxtype for #error/#warning (#14742) 2018-02-21 14:48:18 -08:00
Harlan
5e02d2a877 Implement #warning and #error (#14048)
* Implement #warning and #error

* Fix #warning/#error in switch statements

* Fix AST printing for #warning/#error

* Add to test case

* Add extra handling to ParseDeclPoundDiagnostic

* fix dumping

* Consume the right paren even in the failure case

* Diagnose extra tokens on the same line after a diagnostic directive
2018-02-03 18:07:05 -05:00
Rintaro Ishizaki
2134f9f768 [Parse] Move SyntaxParsingContext to Parse (#14360)
This is only for the Parser.
Also, this resolves layering violation where libAST and libSyntax
depends on each other.
2018-02-03 10:43:52 +09:00
Rintaro Ishizaki
057254dbc1 [Syntax] Bump allocate and cache/reuse RawSyntax
Introduced SyntaxArena for managing memory and cache.

SyntaxArena holds BumpPtrAllocator as a allocation storage.
RawSyntax is now able to be constructed with normal heap allocation, or
by SyntaxArena. RawSyntax has ManualMemory flag which indicates it's managed by
SyntaxArena. If the flag is true, its Retain()/Release() is no-op thus it's
never destructed by IntrusiveRefCntPtr.
This speedups the memory allocation for RawSyntax.

Also, in Syntax parsing, "token" RawSyntax is reused if:
a) It's not string literal with >16 length; and
b) It doesn't contain random text trivia (e.g. comment).
This reduces the overall allocation cost.
2018-02-02 01:27:06 +09:00
Rintaro Ishizaki
6108c881be [Syntax] Use TrailingObjects for SyntaxData (#14301)
This should optimize memory usage for SyntaxData.
2018-01-31 21:50:04 +09:00
Rintaro Ishizaki
fced748790 [Syntax] Represent missing optioanl nodes as nullptr (#14300)
Allocating RawSyntax/SyntaxData for missing optional node is a waste of
resource.
2018-01-31 19:24:00 +09:00
Rintaro Ishizaki
941cfa80bb [Syntax] Don't construct complete Syntax nodes in parsing
Instead, directly use RawSyntax.
2018-01-31 17:13:00 +09:00
Rintaro Ishizaki
d8f6ff0019 [Syntax] Reserve capacity for vector when possible 2018-01-31 15:21:21 +09:00
Rintaro Ishizaki
98fc073e2e [Syntax] Don't rebuild Syntax with RawSyntax in SyntaxVisitor (#14057)
We don't guarantee RawSyntax nodes as unique object in the tree, while
we do that for SyntaxData. We shouldn't recreate SyntaxData once it was
built.
2018-01-22 14:24:20 -08:00
Rintaro Ishizaki
0780c529c4 [Syntax] Unify RawSyntax and RawTokenSyntax using union and TrailingObjects
It better matches with SwiftSyntax model.

Using TrailingObjects reduces the number of heap allocation which
gains 18% performance improvement.
2018-01-18 14:49:46 +09:00
Xi Ge
4c172037e8 [libSyntax][SourceKit] Integrating libSyntax representation of a source file with several SourceKitd syntax requests (#14000)
The enhanced SourceKitd requests are EditorOpen and EdtiorReplaceText. In these two requests, the clients can specify a flag "key. enablesyntaxtree = 1" to get a serialize libSyntax tree with the response.

To help this integration, we added a function in SyntaxParsingContext to explicitly finalize the creation of a SourceFileSyntax to incorporate the fact that SourceKit needs the tree before its destroying the parser instance.

To test this integration, we diff the syntax tree serialized from the frontend action and the tree serialized from a SourceKitd response. They should be identical.
2018-01-17 21:13:22 -08:00
Xi Ge
71af76a87e libSyntax: optionally emit diagnostics for unknown expressions and declarations. (#13973)
With more syntax nodes being specialized, we'd like this
straight-forward way to pinpoint unknown entities. This diagnostics
is only issued in -emit-syntax frontend action and swift-syntax-test
invocation.
2018-01-16 16:14:57 -08:00
Rintaro Ishizaki
d32b3425d2 [Syntax] Use single storage across all SyntaxParsingContexts 2018-01-16 12:34:34 -08:00
Harlan
aed3c47571 [gyb] Rework line directives to allow for C++-style directives (#13965)
This patch reworks the line-directive command line argument for gyb to
take in a python-style format string with `line` and `file` variables
to be substituted.
2018-01-16 14:42:48 -05:00
Rintaro Ishizaki
192434003d [Syntax] Introduce TOKEN macro in TokenKinds.def 2018-01-17 02:38:14 +09:00