Commit Graph

155 Commits

Author SHA1 Message Date
Rintaro Ishizaki
570ed9361f Revert "Merge pull request #26883 from rintaro/revert-revert-26478-gsoc-2019-parser-types"
This reverts commit faaa3a859d, reversing
changes made to 62f947d6ba.
2019-10-14 15:18:05 -07:00
Rintaro Ishizaki
8edea315cd [Syntax] Abolish 'backtick' trivia
- Stop producing 'backtick' trivia for escaping identifier token. '`'s
  are now parts of the token text
- Adjust and simplify C++ libSyntax APIs
- Add 'is_deprecated' property to Trivia.py to attribute SwiftSyntax
  APIs

rdar://problem/54810608
2019-09-09 11:49:25 -07:00
Rintaro Ishizaki
cb308b7e53 Revert "Revert "[Parser] Decouple the parser from AST creation (part 2)""
This reverts commit 8ad3cc8a82.
2019-08-27 14:36:41 -07:00
Rintaro Ishizaki
8ad3cc8a82 Revert "[Parser] Decouple the parser from AST creation (part 2)" 2019-08-27 12:28:48 -07:00
Jan Svoboda
77924c4b84 [Parser] Decouple the parser from AST creation (part 2)
Instead of creating the AST directly in the parser (and libSyntax or
SwiftSyntax via SyntaxParsingContext), make Parser to explicitly create
a tree of ParsedSyntaxNodes. Their OpaqueSyntaxNodes can be either
libSyntax or SwiftSyntax. If AST is needed, it can be generated from the
libSyntax tree.
2019-08-26 19:10:51 +02:00
Sam Lazarus
2a38b48eea Parse / AST: Allow let / var as argument labels with a warning.
The diagnostic is now a warning and the new message alerts the user that
though it is valid to have let and var as argument label names,
they are interpreted as argument labels, not keywords.
2019-04-26 04:08:01 -04:00
Joe Groff
be0140fe56 Switch __opaque to some, per the latest rev of the proposal 2019-04-17 14:46:21 -07:00
Joe Groff
5e1497967a Parse opaque types. 2019-04-17 14:43:32 -07:00
Argyrios Kyrtzidis
c7ac859310 [Parse] Optimize syntax parsing: Speed-up Lexer::lexTrivia()
Introduce ParsedTrivia which is a more efficient structure to use during lexing than syntax::Trivia.
2019-01-17 12:10:27 -08:00
Adrian Prantl
ff63eaea6f Remove \brief commands from doxygen comments.
We've been running doxygen with the autobrief option for a couple of
years now. This makes the \brief markers into our comments
redundant. Since they are a visual distraction and we don't want to
encourage more \brief markers in new code either, this patch removes
them all.

Patch produced by

      for i in $(git grep -l '\\brief'); do perl -pi -e 's/\\brief //g' $i & done
2018-12-04 15:45:04 -08:00
Rintaro Ishizaki
143f55a6e5 [Lexer] Add formStringLiteralToken dedicated for forming string literal 2018-09-19 18:58:54 +09:00
John Holdsworth
4da8cbe655 Implement SE-0200 (extended escaping in string literals)
Supports string literals like #"foo"\n"bar"#.
2018-09-06 15:19:52 -07:00
Brent Royal-Gordon
df22ea1bfb Revert "[Parse] Implementation for SE-200 (raw strings)" 2018-09-06 12:22:41 -07:00
John Holdsworth
999bb40294 New diagnostic for closing delimiter 2018-09-04 20:21:20 +01:00
John Holdsworth
dc96342368 Response to xwu's review 2018-09-02 11:37:02 +01:00
John Holdsworth
7866093ea5 Extend token boundary to include delimiter 2018-08-17 02:12:01 +01:00
John Holdsworth
2317048f44 Alternative implementation for raw strings 2018-07-09 18:38:54 +01:00
John Holdsworth
14213b84bd Revised implementation for raw strings 2018-07-02 12:54:06 +01:00
David Zarzycki
7174d14dad [Parsing] NFC: metaprogram contextual decl keywords into Attr.def 2018-04-07 18:00:12 -04:00
Slava Pestov
93c80da77c Parse: __shared and __owned should be contextual keywords
This was a source compatibility regression, someone actually had
an identifier named __shared.
2017-10-24 20:45:50 -07:00
Xi Ge
34e2aec662 Parser: use parser to generate a refined token stream to help syntax coloring. (#11809)
This patch allows Parser to generate a refined token stream to satisfy tooling's need. For syntax coloring, token stream from lexer is insufficient because (1) we have contextual keywords like get and set; (2) we may allow keywords to be used as argument labels and names; and (3) we need to split tokens like "==<". In this patch, these refinements are directly fulfilled through parsing without additional heuristics. The refined token vector is optionally saved in SourceFile instance.
2017-09-08 10:28:19 -07:00
Robert Widmann
b77f2c147c Parse Shared
Add parser support for __shared and __owned as type attributes.  Also, extend parser diagnostics and tests to account for the new type attributes.
2017-07-23 21:47:25 -07:00
John Holdsworth
981e706fd9 An implementation for 0168-multi-line-string-literals.md (#8813)
This adds support for SE-0168, multi-line string literals.

Extend the lexer to recognize the new literals. Test cases added.

There are still areas for future diagnostic improvement, such as fixits and notes as to why a multi-line string literal will be malformed. Multi-line literals are explicitly forbidden inside of string interpolation, though this may be relaxed in the future.
2017-04-25 18:13:03 -07:00
David Farler
7ee42994c8 Start the Syntax library and optional full token lexing
Add an option to the lexer to go back and get a list of "full"
tokens, which include their leading and trailing trivia, which
we can index into from SourceLocs in the current AST.

This starts the Syntax sublibrary, which will support structured
editing APIs. Some skeleton support and basic implementations are
in place for types and generics in the grammar. Yes, it's slightly
redundant with what we have right now. lib/AST conflates syntax
and semantics in the same place(s); this is a first step in changing
that to separate the two concepts for clarity and also to get closer
to incremental parsing and type-checking. The goal is to eventually
extract all of the syntactic information from lib/AST and change that
to be more of a semantic/symbolic model.

Stub out a Semantics manager. This ought to eventually be used as a hub
for encapsulating lazily computed semantic information for syntax nodes.
For the time being, it can serve as a temporary place for mapping from
Syntax nodes to semantically full lib/AST nodes.

This is still in a molten state - don't get too close, wear appropriate
proximity suits, etc.
2017-02-17 12:57:04 -08:00
practicalswift
6d1ae2a39c [gardening] 2016 → 2017 2017-01-06 16:41:22 +01:00
practicalswift
9d0b2abfc2 [gardening] Normalize end-of-namespace comments 2016-12-17 22:29:07 +01:00
practicalswift
797b80765f [gardening] Use the correct base URL (https://swift.org) in references to the Swift website
Remove all references to the old non-TLS enabled base URL (http://swift.org)
2016-11-20 17:36:03 +01:00
David Farler
f450f0ccdf Revert "Preserve whitespace and comments during lexing as Trivia"
This reverts commit d6e2b58382.
2016-11-18 13:23:31 -08:00
David Farler
d6e2b58382 Preserve whitespace and comments during lexing as Trivia
Store leading a trailing "trivia" around a token, such as whitespace,
comments, doc comments, and escaping backticks. These are syntactically
important for preserving formatting when printing ASTs but don't
semantically affect the program.

Tokens take all trailing trivia up to, but not including, the next
newline. This is important to maintain checks that statements without
semicolon separators start on a new line, among other things.

Trivia are now data attached to the ends of tokens, not tokens
themselves.

Create a new Syntax sublibrary for upcoming immutable, persistent,
thread-safe ASTs, which will contain only the syntactic information
about source structure, as well as for generating new source code, and
structural editing. Proactively move swift::Token into there.

Since this patch is getting a bit large, a token fuzzer which checks
for round-trip equivlence with the workflow:

fuzzer => token stream => file1
  => Lexer => token stream => file 2 => diff(file1, file2)

Will arrive in a subsequent commit.

This patch does not change the grammar.
2016-11-15 16:11:57 -08:00
Jordan Rose
043c2e0c71 [Parse] Parse SE-0117's contextual 'open' keyword. (#3875)
This will allow us to start writing 'open' in downstream projects
while John finishes the implementation. (This patch is extracted from
his full WIP.)
2016-07-29 20:15:18 -07:00
Chris Lattner
d522cd4270 Centralize the parsing logic for #identifiers and make it more similar to
the identifier parsing logic.
2016-02-03 22:37:28 -08:00
Denis Vnukov
3205d42144 Added getter for full (raw) text of the token 2016-01-27 13:43:40 -08:00
Doug Gregor
dccf3155f1 SE-0022: Implement parsing, AST, and semantic analysis for #selector. 2016-01-26 21:12:04 -08:00
Zach Panzarino
e3a4147ac9 Update copyright date 2015-12-31 23:28:40 +00:00
Doug Gregor
c8dd8d0661 Implement SE-0001: Allow (most) keywords as argument labels.
Allow all keywords except for parameter introducers (var/let/inout) to
be argument labels when declaring or calling a
function/initializer/subscript, e.g., this

  func touchesMatching(phase: NSTouchPhase, `in` view: NSView?) -> Set<NSTouch>

can now be expressed as

  func touchesMatching(phase: NSTouchPhase, in view: NSView?) -> Set<NSTouch>

and the call goes from

  event.touchesMatching(phase, `in`: view)

to

  event.touchesMatching(phase, in: view)

Fixes [SR-344](https://bugs.swift.org/browse/SR-344) /
rdar://problem/22415674.
2015-12-22 16:18:28 -08:00
Xi Ge
d569159ffb [Token] Trim surrounding whitespace when querying the ranges of tokens' associated comments.
Need this for rdar://21607719

Swift SVN r31880
2015-09-11 02:14:30 +00:00
Doug Gregor
c02cd1a424 Factor omit-needless-words logic out of the Clang importer.
Sink the actual logic for omitting needless words way down into
Basic, so we can re-use it elsewhere. Tie the Clang importer into that
logic, mapping Clang types down to strings appropriately. NFC

Swift SVN r31233
2015-08-13 23:39:29 +00:00
Doug Gregor
f76bc108fc Don't omit words down to "get", "set", or a Swift keyword.
Swift SVN r31189
2015-08-12 21:24:42 +00:00
Jordan Rose
f5b1efb354 Move client-affecting configuration options into a generated Config.h.
This way they can be used from other projects, like LLDB. The downside
is we now have to make sure the header is included consistently in all
the places we care about, but I think in practice that won't be a problem,
especially not with tests.

rdar://problem/22240127

Swift SVN r31173
2015-08-12 17:50:13 +00:00
Chris Willmore
5ea5ce0e70 Source range of backtick-escaped identifier token should include backticks.
<rdar://problem/21392294> Swift 2 Migration failures, redux

Swift SVN r30907
2015-08-01 05:07:32 +00:00
Chris Lattner
0001dc27bb remove support for the experiemental "character literals" feature.
Swift SVN r30509
2015-07-22 22:35:19 +00:00
Joe Groff
66ba6827bf Parse the 'indirect' attribute on enums and cases.
Swift SVN r29777
2015-06-28 21:52:59 +00:00
Devin Coughlin
8b5f6fec60 Rename '#os' to '#available'
The API review list found it confusing that if #os() and #if os() looked so similar, so
change the availability checking query to be spelled #available:

if #available(iOS >= 9.0, *) {
  ...
}

Swift SVN r26995
2015-04-04 23:33:13 +00:00
Chris Lattner
a0d02cdcdd Several minor changes:
- Strength reduce isAtStartOfBindingName() to just check for 
   identifier or _ and inline into its two callers.
 - Rename Token::isIdentifierOrNone to isIdentifierOrUnderscore.
 - Teach InVarOrLetPattern about matching patterns, so that the
   parser knows when it is parsing an expression as a matching
   pattern but is not yet inside a let/var pattern.
 - Use newfound knowledge of matching patterns to refine handling
   of unexpected let/var when parsing an expression, but not in a
   pattern context, slightly improving QoI in invalid cases.



Swift SVN r26172
2015-03-15 23:36:05 +00:00
Chris Lattner
65b432a495 split tok::oper_postfix into two token kinds, one that indicates that it was spaced
and one that indicates that it was unspaced.  NFC, needed by my next patch.


Swift SVN r25225
2015-02-12 05:19:18 +00:00
Dmitri Hrybenko
d225e496e6 Fix warnings about && nested in ||
Swift SVN r22326
2014-09-28 01:55:18 +00:00
Devin Coughlin
0d7996f4ee Add parsing of availability query expressions (#os(...))
This patch adds a new 'pound_os' token, a new case for it in parseExprPostfix, and parsing of platform version constraints, e.g., OSX >= 10.10. 

It also adds enough type checking and SILGen to get the parsing tests to run without triggering "Unimplemented" assertions.


Swift SVN r21865
2014-09-11 02:59:05 +00:00
Chris Lattner
8991456ff2 Switch infix/postfix/prefix to be declaration modifiers instead of attributes,
eliminating the @'s from them when used on func's.  This is progress towards
<rdar://problem/17527000> change operator declarations from "operator prefix" to "prefix operator" & make operator a keyword

This also consolidates rejection of custom operator definitions into one
place and makes it consistent, and adds postfix "?" to the list of rejected
operators.

This also changes the demangler to demangle weak/inout/postfix and related things
without the @.



Swift SVN r19929
2014-07-14 15:51:49 +00:00
Chris Lattner
8078fecd18 fix <rdar://problem/17625253> Remove the 'strong' context-sensitive keyword
Swift SVN r19819
2014-07-10 19:46:22 +00:00
Chris Lattner
5b49d59c57 Remove the @ from @final and @lazy, the last major piece of
rdar://17168115.

Also, reinstate the ARM driver change and testcase that I removed
in my last patch.


Swift SVN r19790
2014-07-10 06:23:27 +00:00