John Holdsworth
4da8cbe655
Implement SE-0200 (extended escaping in string literals)
...
Supports string literals like #"foo"\n"bar"#.
2018-09-06 15:19:52 -07:00
Brent Royal-Gordon
df22ea1bfb
Revert "[Parse] Implementation for SE-200 (raw strings)"
2018-09-06 12:22:41 -07:00
swift-ci
192587c98a
Merge pull request #17668 from johnno1962a/master
2018-09-06 11:47:36 -07:00
John Holdsworth
9208c5bca6
Final nits in comments.
2018-09-06 14:20:19 +01:00
John Holdsworth
f0f08e1e86
Remove zero width detection for now
2018-09-05 21:51:19 +01:00
John Holdsworth
999bb40294
New diagnostic for closing delimiter
2018-09-04 20:21:20 +01:00
John Holdsworth
02f7cd5db6
generated zero-width characters
2018-09-02 23:59:34 +01:00
John Holdsworth
dc96342368
Response to xwu's review
2018-09-02 11:37:02 +01:00
John Holdsworth
3fc43bcb80
Check for zero-width characters in delimiters
2018-09-01 21:54:54 +01:00
John Holdsworth
032d865fa1
Response to rintaro's 2nd review
2018-08-31 18:51:37 +01:00
Jordan Rose
63cd1258ea
Stop using SourceManager::getBufferIdentifierForLoc to find buffer IDs
...
The right way is findBufferContainingLoc. getBufferIdentifierForLoc is
both slower and wrong in the presence of #sourceLocation.
I couldn't come up with a test for the change in IDE/Utils.cpp because
refactoring still seems to be broken around #sourceLocation. I'll file
bugs for that.
2018-08-29 11:46:41 -07:00
John Holdsworth
9691076af0
Response to rintaro's review
2018-08-27 19:14:26 +01:00
John Holdsworth
4209b72a66
Delimiter specific diagnostic
2018-08-27 10:15:50 +01:00
John Holdsworth
6bd7cb884c
Pragmatic support of multiline/delimited in attributes
2018-08-20 08:43:53 +01:00
John Holdsworth
7866093ea5
Extend token boundary to include delimiter
2018-08-17 02:12:01 +01:00
Alex Hoppen
ac512d4341
[libSyntax] Add a reference counted version of OwnedString
...
We cannot use unowned strings for token texts of incrementally parsed
syntax trees since the source buffer to which reused nodes refer will
have been freed for reused nodes. Always copying the token text whenever
OwnedString is passed is too expensive. A reference counted copy of the
string allows us to keep the token's string alive across incremental
parses while eliminating unnecessary copies.
2018-08-13 15:37:53 -07:00
John Holdsworth
74dd71ca9b
Delimited/Raw strings inside interpolations
2018-08-10 17:27:38 +01:00
Jordan Rose
fc9ea1e329
Add Lexer::IsHashbangAllowed, drop SourceManager::getHashbangBufferID ( #18534 )
...
Having this be a single buffer hardcoded in the SourceManager and set
by all clients is silly. SourceFiles with the 'Main' kind are allowed
to have hashbang lines (`#!`), other files are not. And anyone
manually setting up a Lexer can decide for themselves.
No intended behavioral change.
2018-08-07 08:25:05 -07:00
Rintaro Ishizaki
1aacb8fefb
Merge pull request #17788 from rintaro/parse-identifier-drop3
...
[Parse] Drop Swift3 support for '$', 'throws', and 'rethrows' as identifier
2018-07-25 19:36:39 +09:00
Rintaro Ishizaki
21db9723e4
[Lexer] Don't include backtick length into comment length
...
Although backtick is a kind of trivia piece, Token::getCommentRange doesn't
take it into account. Invalid Token::getCommentRange used to cause
compiler crash.
rdar://problem/42492793
https://bugs.swift.org/browse/SR-8315
2018-07-24 04:37:07 +09:00
John Holdsworth
2317048f44
Alternative implementation for raw strings
2018-07-09 18:38:54 +01:00
Rintaro Ishizaki
a810908da1
[Parse] Drop Swift3 support for '$' as an identifier
...
Swift3 used to parse '$' as an identifier without digagnostics.
Related 6accc5989e
2018-07-06 18:04:26 +09:00
John Holdsworth
14213b84bd
Revised implementation for raw strings
2018-07-02 12:54:06 +01:00
Rintaro Ishizaki
da764e16f2
[Lexer][QoI] Diagnose and fix-it consecutive 'U+00A0's at once
2018-05-01 20:31:37 +09:00
Ahmad Alhashemi
036b2f534d
Minor style edits
2018-04-23 13:21:06 -04:00
Ahmad Alhashemi
e8c17b6686
Move non-breaking space handling to lexUnknown
2018-04-22 15:54:07 -04:00
Ahmad Alhashemi
1603ec2bee
[Parser] Detect nonbreaking space U+00A0 and fixit
2018-04-22 15:54:07 -04:00
Huon Wilson
00c32698e2
[Parse] Put indent-is-4-spaces assumption in one place.
...
This single location can, theoretically, be made more intelligent about
deducing indent from elsewhere and all the consumers will just work.
2018-04-04 10:34:33 +10:00
Rintaro Ishizaki
7237875870
[Parse] Eliminate square_lit token
2018-03-14 21:50:53 +09:00
omochimetaru
3c8057e13f
[Parse] Remove unnecessary Lexer fields
2018-03-12 23:35:13 +09:00
Rintaro Ishizaki
1dc3e17f52
[Lexer] Performance improvement for Lexer::kindOfIdentifier ( #15168 )
...
Don't check SIL_KEYWORDs when we're not in SIL mode.
Lexer::kindOfIdentifier is super hot-path because many of
tokens in a swift source file are keywords or identifiers.
2018-03-12 15:03:58 +09:00
omochimetaru
8592496b22
[Parse] Remove dead code in lexHash ( #15154 )
2018-03-12 08:47:54 +09:00
omochimetaru
5e555868fe
[Parse] Reduce branch by running lexTrivia always ( #15137 )
2018-03-10 21:26:54 +09:00
Sho Ikeda
74ba135008
Merge pull request #15040 from ikesyo/gardening-not-empty
...
[gardening] Use `!empty()` over `size() > 0`
2018-03-08 18:47:12 +09:00
omochimetaru
c4c5b7130e
[Parse] Fix error in unsigned char target ( #15068 )
2018-03-08 15:57:43 +09:00
omochimetaru
967a48cb77
[Parse] refactor Lexer initialization
2018-03-08 11:17:51 +09:00
Sho Ikeda
cea6c03eb2
[gardening] Use !empty() over size() > 0
2018-03-08 09:21:09 +09:00
Sho Ikeda
25cdc981c7
Merge pull request #14977 from ikesyo/gardening-use-empty
...
[gardening] Use `empty()` over `size() == 0`
2018-03-07 09:34:26 +09:00
omochimetaru
22cddbf033
[Syntax] Parse invalid chars as trivia
2018-03-06 08:25:09 -08:00
omochimetaru
b4192d80e9
[Parse] EmitDiagnosticsIfToken in lexUnknown
...
It is needed from lexTrivia update future.
2018-03-06 08:25:09 -08:00
omochimetaru
e6f42fc63d
[Parse] split lexUnknown function from lexImpl
2018-03-06 08:25:09 -08:00
omochimetaru
0a69bd7f8d
[NFC] readable control flow in lexImpl default case
...
There are 3 token flow and 2 trivia flow.
This commit makes it clear and more readable.
And apply clang-format.
2018-03-06 08:25:09 -08:00
omochimetaru
3e252c98a8
[NFC] Fix naming style in lexImpl default case
2018-03-06 08:25:09 -08:00
omochimetaru
190af6c063
[Syntax] support nul character as garbage trivia
2018-03-05 16:53:24 +09:00
omochimetaru
58857fa1cb
[Parse] refactor Lexer by made NulCharacterKind type
2018-03-05 16:53:18 +09:00
Sho Ikeda
26d650292f
[gardening] Use empty() over size() == 0
2018-03-05 14:43:13 +09:00
Rintaro Ishizaki
766774206b
[Lexer] Don't setEscapedIdentifier(true) for tok::eof at ArtificialEOF
...
https://bugs.swift.org/browse/SR-6926
This happens when the Parser re-lexing comment tokens that sets
ArtificialEOF at the end of comment range.
It used to cause an assertion failure:
(!value || Kind == tok::identifier) && "only identifiers can be escaped identifiers"
2018-02-12 14:58:12 +09:00
Rintaro Ishizaki
0780c529c4
[Syntax] Unify RawSyntax and RawTokenSyntax using union and TrailingObjects
...
It better matches with SwiftSyntax model.
Using TrailingObjects reduces the number of heap allocation which
gains 18% performance improvement.
2018-01-18 14:49:46 +09:00
Erik Eckstein
a680768971
SIL: In textual SIL allow global SIL names starting with '$'.
...
For example: @$S1m3fooyyF
It's needed to change the mangling prefix to $S.
The parser change only affects SIL (and not swift).
I didn't add test case because it will be fully tested when changing the mangling prefix.
2018-01-05 11:29:15 -08:00
omochimetaru
bc88330740
[Parse] Lexer build backtick trivia around espaced identifier token
2017-12-29 00:22:49 +09:00