Commit Graph

34 Commits

Author SHA1 Message Date
Harlan
bdfa6cd4d7 [SwiftSyntax] Add SyntaxRewriter.visitAny(_:) (#15212)
* [SwiftSyntax] Add SyntaxRewriter.visitAny(_:)

This function, when overridden, allows Syntax rewriters to perform custom dynamic visitation behavior. If a user wanted to, say, store a series of transformations accessible by metatype, they can override visitAny and do their own runtime dispatch.

If a non-nil result is returned from visitAny, the original specialized visitors are skipped.
2018-03-27 10:55:31 -04:00
Xi Ge
123ccc7602 SwiftSyntax: add an API to check whether a syntax node has underlying source. (#15389) 2018-03-20 18:16:54 -07:00
Denis Morozov
2866f0e0a6 [SwiftSyntax] Add replace method to SyntaxCollections (#15331)
* [SwiftSyntax] Add replace method SyntaxCollections

* Change replaceSubrange method call to subscript setter call

* [SwiftSyntax] Add test for SyntaxCollections

* Fix mistakes in SyntaxCollections tests

* Use syntax's raw value instead of syntax itself in replacing method

* Update SyntaxCollections tests
2018-03-20 14:58:16 -04:00
Xi Ge
1c9f1c48ed SwiftSyntax: allow any Syntax nodes to access their leading/trailing trivia. (#15278)
Although libSyntax is designed in a way that trivia is attached to
tokens, we shouldn't restrict clients to access trivia only from a token.
An example can be doc-comment, which conceptually attaches to a declaration rather
than the start token of a declaration, like at-attributes.
2018-03-15 15:31:08 -07:00
Xi Ge
af868778b6 SwiftSyntax: add an API to teach Trivia to calculate byte size. (#15251)
This missing piece will allow clients to customize the size of
SyntaxNode better.
2018-03-14 17:14:52 -07:00
Rintaro Ishizaki
ea5f93807f [Syntax] Add 'unknown' token to Token.py 2018-03-14 20:38:48 +09:00
Xi Ge
ee36dbeda3 [WIP] SwiftSyntax: Implement APIs for Syntax nodes to calculate absolute positions. (#15097)
Syntax nodes are designed as reusable blocks to construct Swift source
code. For this reason, we don't track absolute position in each node;
instead, the absolute position should be calculated on the fly when
needed. We recently found absolute positions are useful to bridge with
sourcekitd, which typically speaks in the language of line, column and
offset. Therefore, this patch tries to add a computed
property on SyntaxNode to get its absolute position.

To compute the absolute position of a SyntaxNode from scratch requires tree traversal.
However, getting the absolute position from these added APIs doesn't necessarily
mean we'll traverse since SyntaxData will actively cache the computed position.
Also, since we recursively compute the absolute position, all the position caches
for a SyntaxNode's previous siblings and ancestors will be populated as well during
the calculation of this SyntaxNode.
2018-03-12 15:42:46 -07:00
Harlan
e41a42b060 [SwiftSyntax] Ensure SyntaxRewriter respects nil children when rewriting (#15052) 2018-03-08 00:37:41 -05:00
Harlan
d3f6b8743d [SwiftSyntax] Publicize DiagnosticEngine.addConsumer and add hasErrors property (#14996)
* [SwiftSyntax] Publicize DiagnosticEngine.addConsumer and add hasErrors property

* [Syntax] Make DiagnosticEngine.hasErrors public
2018-03-06 00:40:24 -05:00
Xi Ge
2e03eac75c [WIP] Re-apply "SwiftSyntax: Teach SwiftSyntax to use SourceKitd to serialize syntax trees. (#14424)" (#14506)
After removing white space changes and attempting to configure dependencies
correctly.
2018-02-11 09:38:53 -08:00
Doug Gregor
774bee2294 Revert "Re-apply "SwiftSyntax: Teach SwiftSyntax to use SourceKitd to serialize syntax trees. (#14424)" (#14465)"
This reverts commit f8c77e17ce.
2018-02-08 22:58:45 -08:00
Doug Gregor
b060667877 Revert "[sourcekitd] Rename module SwiftSourceKit to SwiftLang. (#14497)"
This reverts commit 43435af7ee.
2018-02-08 22:58:43 -08:00
Xi Ge
43435af7ee [sourcekitd] Rename module SwiftSourceKit to SwiftLang. (#14497)
The sourcekitd client library provides parsing APIs for SwiftSyntax users.
The internal use of sourcekit service is an implementation detail that end users
shouldn't worry about.
2018-02-08 17:26:00 -08:00
Xi Ge
f8c77e17ce Re-apply "SwiftSyntax: Teach SwiftSyntax to use SourceKitd to serialize syntax trees. (#14424)" (#14465) 2018-02-08 15:11:31 -08:00
Rintaro Ishizaki
6c0af2a24f [Syntax] Introduce CodeBlockItem (#14458)
CodeBlockItem represents Decl, Stmt or Expr that optionally followed by
semi-colon.
SourceFile syntax holds a list of CodeBlockItem.
2018-02-08 10:31:01 +09:00
Xi Ge
50cde06cf0 Revert "SwiftSyntax: Teach SwiftSyntax to use SourceKitd to serialize syntax trees. (#14424)" 2018-02-06 23:20:42 -08:00
Xi Ge
871c9dac2a SwiftSyntax: Teach SwiftSyntax to use SourceKitd to serialize syntax trees. (#14424)
When using SwiftSyntax as a standalone tool, we invoke Swiftc
internally to get serialized syntax trees. This is not ideal for
several reasons: (1) we have to hard-code the relative path of swiftc
to invoke it; (2) we have to rely on standard input/output to pass the
tree across the process boundaries; and (3) we have to maintain two
different ways to get syntax tree (swiftc and sourcekitd).

This patch attempts to teach SwiftSyntax to use SourceKitd to get the
tree just like other clients. We first add a SourceKitd client library
written in Swift; and next teach SwiftSyntax to adopt this SourceKitd
client-side library. For platforms other than MacOS, we still use Swiftc
to get syntax trees. This client library also allows us to add 
SourceKitd tests in Swift.

This patch also re-enables several flaky tests.
2018-02-06 19:40:16 -08:00
Rintaro Ishizaki
fced748790 [Syntax] Represent missing optioanl nodes as nullptr (#14300)
Allocating RawSyntax/SyntaxData for missing optional node is a waste of
resource.
2018-01-31 19:24:00 +09:00
Xi Ge
1b5306158d [test] libSyntax: add a test to ensure the client side decodes syntax tree properly. NFC (#14203)
We need to expose a SwiftSyntax API to encode a source file into Json
format.
2018-01-26 16:53:20 -08:00
Harlan
a339c82c1e [SwiftSyntax] Make Syntax nodes structs instead of classes (#14122)
* [Experiment] Make Syntax nodes structs instead of classes

* [Experiment] Add Hashable conformance to concrete types

* Fix pep8 violation

* Remove AnySyntax, explicitly specialize SyntaxCollection nodes

* Refine the comment for SyntaxCollection nodes
2018-01-24 23:17:25 -05:00
Rintaro Ishizaki
9c4d6f4893 Revert "[SwiftSyntax] Add debug print when the JSONDecoder fails to decode JSON (#14056)" (#14101)
This reverts commit 840166c3ea.
2018-01-24 10:01:43 +09:00
Xi Ge
f0e906c7d2 [test] Disable a flaky test: SwiftSyntax/ParserFile. rdar://36740859 (#14095) 2018-01-23 16:40:45 -08:00
Rintaro Ishizaki
840166c3ea [SwiftSyntax] Add debug print when the JSONDecoder fails to decode JSON (#14056)
from the compiler's '-emit-syntax'.

Investigating: rdar://problem/36379512
2018-01-22 23:44:18 +09:00
Ben Langmuir
79c8695c5b [test] Disable SwiftSyntax/VisitorTest that fails occassionally in CI
rdar://36379512
2018-01-12 11:53:42 -08:00
Xi Ge
d41eedcbf6 SwiftSyntax: Add a read-only syntax visitor. (#13681)
Based on the feedbacks from our early adopters, separating syntax tree analysis
with transformation is a common pattern. Thus, we introduce a read-only
syntax tree visitor to help the analysis phase. This visitor never alters the
content of a tree being visited, in contrast to SyntaxRewriter which always does.
2018-01-02 16:52:30 -08:00
Harlan
9185ef5610 [Syntax] Add #selector and #keyPath tokens (#13615)
* [Syntax] Add #selector and #keyPath tokens

* [Syntax][test] Add #selector and #keyPath to syntax parsing tests

* [Syntax] Don't fatalError on an unknown token kind

* Fix test build error
2017-12-29 12:35:08 -05:00
Harlan
a0512cd48a re-enable SwiftSyntax tests (#13399) 2017-12-14 23:39:16 -05:00
Harlan
2cc756414b [SwiftSyntax] Swift Diagnostics API (#11511)
This patch provides an API and Serialization for clang-style diagnostics
from within Swift. Libraries can use this API to pop their own custom
diagnostics that can be serialized to JSON.
2017-12-12 15:55:22 -05:00
Harlan
d0339c2767 Add contextual_keyword and string_interpolation_anchor to SwiftSyntax (#13321) 2017-12-08 16:13:49 -05:00
Tony Allevato
1187890ca4 [SwiftSyntax] Fix SyntaxChildren iteration (#12319)
* [SwiftSyntax] Fix SyntaxChildren iteration

SyntaxChildren.Iterator stops the first time it sees a nil child;
since it traverses all indexes without considering presence, this
causes it to terminate iteration early if there is a missing child
in the list.

* [SwiftSyntax] Add tests for SyntaxChildren
2017-10-15 00:38:35 -04:00
Harlan
1d2d0a6f71 [CMake] Only build SwiftSyntax if building SDK overlays (#12141)
* [CMake] Only build SwiftSyntax if building SDK overlays

SwiftSyntax depends on Foundation, which depends on the SDK overlays
being built. However, the existing build configuration tried to build
SwiftSyntax even if the SDK overlays were not built. Ensure we're
building overlays before building SwiftSyntax, and guard tests with an
sdk_overlay test.

* Remove TODO comment
2017-10-04 15:30:02 -04:00
Harlan
fab6178f39 [CMake] Include stdlib/ before tools/ (#11752)
This is, unfortunately, the quickest workaround to get SwiftSyntax
building properly and generating a proper module. I'm going to take some
time to figure out what global mutable state exactly is touched inside
stdlib/ that suddenly makes this work, but for now, get the tests re-enabled.
2017-09-26 14:03:39 -04:00
Harlan Haskins
738508cad4 Disable SwiftSyntax tests (for now) 2017-08-14 18:08:48 -07:00
Harlan
ade67ca899 [Syntax] Swift libSyntax API (#11320)
* Create Swift libSyntax API

This patch is an initial implementation of the Swift libSyntax API. It
aims to provide all features of the C++ API but exposed to Swift.

It currently resides in SwiftExperimental and will likely exist in a
molten state for a while.

* Only build SwiftSyntax on macOS
2017-08-14 16:47:48 -07:00