These APIs return SourceLocs, and eventually the Parser should consume
tokens, which now include source trivia such as whitespace and comments,
and package them into a purely syntactic tree. Just a tiny step. NFC.
Store leading a trailing "trivia" around a token, such as whitespace,
comments, doc comments, and escaping backticks. These are syntactically
important for preserving formatting when printing ASTs but don't
semantically affect the program.
Tokens take all trailing trivia up to, but not including, the next
newline. This is important to maintain checks that statements without
semicolon separators start on a new line, among other things.
Trivia are now data attached to the ends of tokens, not tokens
themselves.
Create a new Syntax sublibrary for upcoming immutable, persistent,
thread-safe ASTs, which will contain only the syntactic information
about source structure, as well as for generating new source code, and
structural editing. Proactively move swift::Token into there.
Since this patch is getting a bit large, a token fuzzer which checks
for round-trip equivlence with the workflow:
fuzzer => token stream => file1
=> Lexer => token stream => file 2 => diff(file1, file2)
Will arrive in a subsequent commit.
This patch does not change the grammar.
If we found any error in a list, in most cases, we cannot expect that the
following tokens could construct a valid element. Skip them, instead of trying
to parse them as the next element. This significally reduces bogus diagnostics.
Bailout if seeing tok::eof or token that can never start a element, after
parsing an element. This silences superfluous "expected ',' separator" error,
or misleading expected declaration error. What we should emit is
"expected ')' in expression list, or "expected '}' in struct".
Use tok::NUM_TOKENS instead. tok::unknown can easily appear in source code.
For instance `skipUntil(tok::eof)` did not work as expected, because that was
`skipUntil(tok::eof, tok::unknown)` hence does stop at error tokens such as
`0xG` (invalid hex number literal).
Revert 2abc92bbb5, since that was
accidental side-effect of 45118037cc.
Forward references are not allowed actually.
If '>' could not be found, the parser should return the location of the
last token parsed, instead of the current token.
Previously, it may causes ASTVerifier error "child source range not contained
within its parent" in some cases.
This can be used as QoI for things like 'NSASCIIStringEncoding', which
is going to canonically be `String.Encoding.ascii` if/when SE-0086 is
accepted.
We can't actually handle this in NS_SWIFT_NAME (that is, we can't
/import/ something onto a nested type), but that's okay: we already
have stricter limitations on NS_SWIFT_NAME enforced by Clang.
rdar://problem/26352374
- Remove stray newline
- Adjust wording when recommending backticks for a keyword identifier
- Provide fix-it when encountering a keyword as an identifier
rdar://problem/25761380
When declaring a function like func repeat(){}, the diagnostic is
"expected an identifier" but 'repeat' looks like a reasonable
identifier at first glance, so actually say why it isn't.
rdar://problem/25761380
It should have the same form as the argument to NS_SWIFT_NAME
in Objective-C, except that it permits operators and (currently)
disallows instance members and properties. We do get to share the
same parsing code, at least.
This actually caught an error in the Foundation overlay!
Groundwork for SR-1008.
The swift_name string format now supports "getter:" and "setter:"
prefixes to indicate that a function is the getter or setter of a
Swift-synthesized property. Start parsing these DeclNames and make
sure they're reflected in the Swift name lookup tables.
[Clang update required]
A swift_name attribute on a global declaration can specify a dotted
name (e.g., SomeStruct.member) to map that global into a member of the
(Swift-)named type. Handle this mapping in DeclName parsing, plumb it
through importFullName, and cope with it in the Swift name lookup
tables (tested via the dump) and importing into a Swift DeclContext
(as-yet-untested). Part of SE-0033.
...because "build configuration" is already the name of an Xcode feature.
- '#if' et al are "conditional compilation directives".
- The condition is a "conditional compilation expression", or just
"condition" if it's obvious.
- The predicates are "platform conditions" (including 'swift(>=...)')
- The options set with -D are "custom conditional compilation flags".
(Thanks, Kevin!)
I left "IfConfigDecl" as is, as well as SourceKit's various "BuildConfig"
settings because some of them are part of the SourceKit request format.
We can change these in follow-up commits, or not.
rdar://problem/19812930
Expand the "skip" functions in the parser to be more careful about
not skipping passed a #endif or a code completion token, since they
are synchronization points for the parser.
The fix was to change Parser::delayParseFromBeginningToHere to use
a manual loop instead of skipUntil (which can stop early for non-EOF
tokens).
Swift parser splits tokens in few cases, but it swift::tokenize(...) does not know
about that. In order to reconstruct token stream as it was seen by the parser,
we need to collect the tokens it decided to split and use this information
in swift::tokenize(...).
Introduce a new attribute, swift3_migration, that lets us describe the
transformation required to map a Swift 2.x API into its Swift 3
equivalent. The only transformation understood now is "renamed" (to
some other declaration name), but there's a message field where we can
record information about other changes. The attribute can grow
somewhat (e.g., to represent parameter reordering) as we need it.
Right now, we do nothing but store and validate this attribute.
Allow all keywords except for parameter introducers (var/let/inout) to
be argument labels when declaring or calling a
function/initializer/subscript, e.g., this
func touchesMatching(phase: NSTouchPhase, `in` view: NSView?) -> Set<NSTouch>
can now be expressed as
func touchesMatching(phase: NSTouchPhase, in view: NSView?) -> Set<NSTouch>
and the call goes from
event.touchesMatching(phase, `in`: view)
to
event.touchesMatching(phase, in: view)
Fixes [SR-344](https://bugs.swift.org/browse/SR-344) /
rdar://problem/22415674.
This times each phase of compilation, so you can see where time is being
spent. This doesn't cover all of compilation, but does get all the major
work being done.
Note that these times are non-overlapping, and should stay that way.
If we add more timers, they should go in a different timer group, so we
don't end up double-counting.
Based on a patch by @cwillmor---thanks, Chris!
Example output, from an -Onone build using a debug compiler:
===-------------------------------------------------------------------------===
Swift compilation
===-------------------------------------------------------------------------===
Total Execution Time: 8.7215 seconds (8.7779 wall clock)
---User Time--- --System Time-- --User+System-- ---Wall Time--- --- Name ---
2.6670 ( 30.8%) 0.0180 ( 25.3%) 2.6850 ( 30.8%) 2.7064 ( 30.8%) Type checking / Semantic analysis
1.9381 ( 22.4%) 0.0034 ( 4.8%) 1.9415 ( 22.3%) 1.9422 ( 22.1%) AST verification
1.0746 ( 12.4%) 0.0089 ( 12.5%) 1.0834 ( 12.4%) 1.0837 ( 12.3%) SILGen
0.8468 ( 9.8%) 0.0171 ( 24.0%) 0.8638 ( 9.9%) 0.8885 ( 10.1%) IRGen
0.6595 ( 7.6%) 0.0142 ( 20.0%) 0.6737 ( 7.7%) 0.6739 ( 7.7%) LLVM output
0.6449 ( 7.5%) 0.0019 ( 2.6%) 0.6468 ( 7.4%) 0.6469 ( 7.4%) SIL verification (pre-optimization)
0.3505 ( 4.1%) 0.0023 ( 3.2%) 0.3528 ( 4.0%) 0.3530 ( 4.0%) SIL optimization
0.2632 ( 3.0%) 0.0005 ( 0.7%) 0.2637 ( 3.0%) 0.2639 ( 3.0%) SIL verification (post-optimization)
0.0718 ( 0.8%) 0.0021 ( 3.0%) 0.0739 ( 0.8%) 0.0804 ( 0.9%) Parsing
0.0618 ( 0.7%) 0.0010 ( 1.4%) 0.0628 ( 0.7%) 0.0628 ( 0.7%) LLVM optimization
0.0484 ( 0.6%) 0.0011 ( 1.5%) 0.0495 ( 0.6%) 0.0495 ( 0.6%) Serialization (swiftmodule)
0.0240 ( 0.3%) 0.0006 ( 0.9%) 0.0246 ( 0.3%) 0.0267 ( 0.3%) Serialization (swiftdoc)
0.0000 ( 0.0%) 0.0000 ( 0.0%) 0.0000 ( 0.0%) 0.0000 ( 0.0%) Name binding
8.6505 (100.0%) 0.0710 (100.0%) 8.7215 (100.0%) 8.7779 (100.0%) Total
This configuration clause will suppress lex diagnostics and skip parsing
altogether if the code under the clause isn't active - the compiler must
have a repository version greater than or equal to the version given to
_compiler_version.
This option is only meant to be used sparingly and not to track the
Swift *language* version.
Example, if using a compiler versioned 700.0.28:
#if _compiler_version("700.0.23")
print("This code will compile for versions 700.0.23 and later.")
#else
This + code + will + not + be + parsed
#endif
Included are new diagnostics for checking that the version is formatted
correctly and isn't empty.
New tests:
- Compiler version comparison unit tests
- Build configuration diagnostics
- Skipping parsing of code under inactive clauses
rdar://problem/22730282
Swift SVN r32195
Compute the hash of all interface tokens when parsing; write the
interface hash to the swiftdeps file, or if the -dump-interface-hash
option is passed to the frontend. This hash will be used in incremental
mode to determine whether a file's interface has changed, and therefore
whether dependent files need to be rebuilt in response to the change.
Committed on ChrisW's behalf while he gets his setup unborked.
rdar://problem/15352929
Swift SVN r30477
conjunction with .fixItInsert(). As such, introduce a helper named
.fixItInsertAfter() that does what we all want. Adopt this in various
places around the compiler. NFC.
Swift SVN r26147
Needed by fix in SourceKit which will be submitted later (rdar://problem/18945845,
SourceKit crashed in SourceKit::SwiftEditorDocument::replaceText()).
Swift SVN r23511
Use the same token-splitting technique we use to interpret '>>>' as a series of closing angle brackets to also extract '?' and '!' IUO sigils. Tweak it so that we properly reset the lexer state instead of just peeking the next token, because otherwise, when we chop a token like '>?>>', we'll see only the '?' (because '?' by itself is a question_postfix) and drop the '>>' on the floor. Remove the lexer hacks that pattern-matched specific >?>? sequences.
Swift SVN r22669
As part of this change, allow #line directives to extend to the end of the
file, rather than requiring a reset.
Note that #line regions that start or end within function bodies will not
behave correctly when in delayed parsing modes. This was true before and
after this commit. (#line regions contained entirely within a function and
not within any other #line regions should be fine.)
Swift SVN r20571
