589 Commits

Author SHA1 Message Date
Dmitri Hrybenko
0b8f72b9c1 Introduce an artificial EOF in the Lexer
This ensures that we don't go past the end of a subrange of a buffer when doing
delayed parsing.


Swift SVN r5952
2013-07-01 21:48:38 +00:00
Chris Lattner
5ef2ad7b3c rearrange this so the optimizer will jump thread it.
Swift SVN r5913
2013-06-30 21:21:46 +00:00
Chris Lattner
8d6eb930f6 autogenerate our lexer identifier table from our perfectly good .def file we already have.
Swift SVN r5911
2013-06-30 21:15:47 +00:00
Joe Groff
45a69154bd Parse: Parse switch statements (again).
Reimplement 'switch' parsing for our new AST representation, where cases contain patterns and 'where' guards, case blocks can have multiple cases, and 'default' is constrained to being the lone label of the last block if present. No type-checking or parsing of actual pattern productions yet.

Swift SVN r5834
2013-06-27 05:13:41 +00:00
Joe Groff
ba5d752cbb Parse: Reserve '_' as a keyword.
This makes parsing patterns a bit more straightforward and prevents us from inadvertently accepting '_' as an identifier in a place we shouldn't.

Swift SVN r5806
2013-06-26 00:45:13 +00:00
Dmitri Hrybenko
a2ddd4e488 Refactor lexer backtracking. Introduce opaque types that encapsulate lexer and
parser state.  Backtracking will be used a lot when we implement delayed
parsing for function bodies, and we don't want to leak lexer and parser state
details to AST classes when we store the state for the first and last token for
the function body.


Swift SVN r5759
2013-06-21 22:26:41 +00:00
Ted Kremenek
3341e11687 Initialize a variable to make the analyzer happy.
The analyzer can't reason about the following complicated
control-dependency:

    if (!isxdigit(BytesPtr[0]) || !isxdigit(BytesPtr[1]) ||
        !isxdigit(BytesPtr[2]) || !isxdigit(BytesPtr[3])
        continue;  // Ignore invalid escapes.
      
    StringRef(BytesPtr, 4).getAsInteger(16, CharValue);
    BytesPtr += 4;

In this case, we cap the number of bytes read at 4, and we have
verified that the bytes will always form an integer.  The analyzer
thinks there could be a false positive uninitialized value warning
here since 'getAsInteger()' does its own checking to see if the
parsed bytes actually compose an integer value.

Swift SVN r5566
2013-06-11 00:15:27 +00:00
Joe Groff
eb6651488c Consider ':' a separating character when lexing operators.
We would assert in the parser on something like [a:+b] because we were lexing '+' as a binary operator.

Swift SVN r5417
2013-05-31 02:25:02 +00:00
Chris Lattner
686efc45db allow naming local values in .sil files something meaningful, like %a.
Swift SVN r5346
2013-05-25 22:20:57 +00:00
Chris Lattner
b81adff06f update for mainline llvm change, hopefully fixing the buildbot.
Swift SVN r5319
2013-05-25 00:43:47 +00:00
Chris Lattner
98d708d1fc In a SIL function body, lex %42 as a new sil_local_name token.
Swift SVN r5261
2013-05-22 05:34:18 +00:00
Chris Lattner
22610d5db8 substantially improve error recovery for erroneous character and string literals.
Before we'd emit multiple diagnostics and confuse the lexer in some really common
cases (e.g. malformed escapes).


Swift SVN r5242
2013-05-20 23:26:42 +00:00
Chris Lattner
bde8f753da wire up minimal sil function parsing (still not validating or creating
a SIL function).  This introduces contextual lexing state for SIL function 
bodies for SIL-specific lexing rules.


Swift SVN r5179
2013-05-16 18:46:46 +00:00
Joe Groff
b818405034 Replace direct use of [[clang::fallthrough]] with a macro.
Add a SWIFT_FALLTHROUGH macro that expands to [[clang::fallthrough]] for Clang and nothing for other compilers. No functionality change.

Swift SVN r5043
2013-05-05 18:54:03 +00:00
Joe Groff
a2341a4cde Add Unicode symbol characters to operator charset.
Remove '@' from the operator character set, but add the math, symbol, punctuation, arrow, and line- and box-drawing characters. Also allow operators to contain (but not start with) combining characters--this should be safe, because identifiers can't start with combining characters either, and we don't allow them anywhere else.

Swift SVN r5019
2013-05-01 23:13:48 +00:00
Dave Zarzycki
a87df3d5dc Robustness: parse incomplete hex in char escapes
Swift SVN r5011
2013-05-01 06:36:08 +00:00
Chris Lattner
5b4c31dc94 reland r4968, with a bugfix to avoid breaking the lexer measuring token lengths.
Original message:
SIL Parsing: add plumbing to know when we're parsing a .sil file
Enhance the lexer to lex "sil" as a keyword in sil mode.


Swift SVN r4988
2013-04-30 00:28:37 +00:00
Chris Lattner
aafe3bdbdc revert r4968, it apparently breaks the world. I'll recommit it when I have time to investigate.
Swift SVN r4971
2013-04-29 16:58:43 +00:00
Chris Lattner
b503206bec SIL Parsing: add plumbing to know when we're parsing a .sil file
Enhance the lexer to lex "sil" as a keyword in sil mode.


Swift SVN r4970
2013-04-29 05:42:59 +00:00
Joe Groff
1013781bcc Parse: Allow Unicode identifier characters.
Extend the character set for identifiers according to WG14 N1518, which recommends an extended character set for identifier start and continuation characters in C. Mangle identifiers containing non-ASCII characters by borrowing the Punycode encoding used for international domain names.

No Unicode operators just yet.

Swift SVN r4968
2013-04-28 21:39:02 +00:00
Jordan Rose
790248d8b4 Diagnostics: use builder pattern instead of streaming for ranges/fix-its.
Per Chris's feedback and suggestions on the verbose fix-it API, convert
diagnostics over to using the builder pattern instead of Clang's streaming
pattern (<<) for fix-its and ranges. Ranges are included because
otherwise it's syntactically difficult to add a fix-it after a range.

New syntax:

  diagnose(Loc, diag::warn_problem)
    .highlight(E->getRange())
    .fixItRemove(E->getLHS()->getRange())
    .fixItInsert(E->getRHS()->getLoc(), "&")
    .fixItReplace(E->getOp()->getRange(), "++");

These builder functions only exist on InFlightDiagnostic; while you can
still modify a plain Diagnostic, you have to do it with plain accessors
and a raw DiagnosticInfo::FixIt.

Swift SVN r4894
2013-04-24 23:15:53 +00:00
Jordan Rose
3486446e65 Add fix-its for stray NUL characters.
Swift SVN r4814
2013-04-18 23:34:23 +00:00
Jordan Rose
9db3e070e8 Add fix-its for unterminated comments and missing EOF newlines.
Note that we can't actually display the latter in textual format (because
the fix-it contains a newline), nor can we verify it (because the fixit
contains a newline).

Swift SVN r4813
2013-04-18 23:34:10 +00:00
Joe Groff
bfd2f85b5c Parse 'fallthrough' statements.
Create a new FallthroughStmt, which transfers control from a 'case' or 'default' block to the next 'case' or 'default' block within a switch. Implement parsing and sema for FallthroughStmt, which syntactically consists of a single 'fallthrough' keyword. Sema verifies that 'fallthrough' actually appears inside a switch statement and that there is a following case or default block to pass control to.

SILGen/IRGen support forthcoming.

Swift SVN r4653
2013-04-10 17:30:42 +00:00
Joe Groff
ef6d33039e Mark fallthroughs with [[clang::fallthrough]].
Use [[clang::fallthrough]] instead of informal /*fallthrough*/ comments.

Swift SVN r4574
2013-04-02 20:51:38 +00:00
Joe Groff
88d4284178 Parser: Relax whitespace rules for ( and [.
Now that we enforce semicolon or newline separation between statements, we can relax the whitespace requirements on '(' and '[' tokens. A "following" token is now just a token that isn't at the start of a line, and any token can be a "starting" token. This allows for:

  a(b)
  a (b)
  a[b]
  a [b]

to parse as applications and subscripts, and:

  a
  (b)
  a
  [b]

to parse as an expr followed by a tuple or an expr followed by a container literal.

Swift SVN r4573
2013-04-02 16:43:20 +00:00
Chris Lattner
d53efbba38 fix character escape comments.
Swift SVN r4522
2013-03-28 22:03:03 +00:00
Joe Groff
9667bda089 Implement 'as' syntax for coercions and casts.
Provide distinct syntax 'a as T' for coercions and 'a as! T' for unchecked downcasts, and add type-checker logic specialized to coercions and downcasts for these expressions. Change the AST representation of ExplicitCastExpr to keep the destination type as a TypeLoc rather than a subexpression, and change the names of the nodes to UncheckedDowncast and UncheckedSuperToArchetype to make their unchecked-ness explicit and disambiguate them from future checked casts.

In order to keep the changes staged, this doesn't yet affect the T(x) constructor syntax, which will for the time being still perform any construction, coercion, or cast.

Swift SVN r4498
2013-03-27 22:27:11 +00:00
Joe Groff
89e1b7e773 Parser: Give IfExpr traditional ternary syntax.
Swift SVN r4489
2013-03-26 01:17:34 +00:00
Joe Groff
4c09ef61e3 Add conditional expressions.
Implement the syntax 'if x then y else z', which evaluates to 'y' if 'x' is true or 'z' if 'x' is false. 'x' must be a valid logic value, and 'y' and 'z' must be implicitly convertible to a common type.

Swift SVN r4407
2013-03-16 20:28:58 +00:00
Doug Gregor
bd61ca5927 Diagnose multiple statements/declarations on the same line that are not separated by a semicolon.
Swift SVN r4364
2013-03-13 01:32:30 +00:00
Joe Groff
062ad267c4 Value-only switch statements.
Implement switch statements with simple value comparison to get the drudge work of parsing and generating switches in place. Cases are checked using a '=~' operator to compare the subject of the switch to the value in the case. Unlike a C switch, cases each have their own scope and don't fall through. 'break' and 'continue' apply to an outer loop rather to the switch itself. Multiple case values can be specified in a comma-separated list, as in 'case 1, 2, 3, 4:'. Currently no effort is made to check for duplicate cases or to rank cases by match strength; cases are just checked in source order, and the first one wins (aside from 'default', which is branched to if all cases fail).

Swift SVN r4359
2013-03-12 04:43:01 +00:00
Dave Zarzycki
c4259e05c8 Make literal parsing more robust
Swift SVN r4339
2013-03-08 17:16:05 +00:00
Dave Zarzycki
0f2f031a67 Typo
Swift SVN r4266
2013-03-02 17:55:36 +00:00
Dave Zarzycki
68c6d4c6e3 13324820 Change tuple accessor syntax to 'tuple.0'
Swift SVN r4265
2013-03-02 17:51:03 +00:00
Dave Zarzycki
23cd39137b Do not allow embedded ASCII control characters in string/char literals
We might need/want to make an exception for '\t'.

Swift SVN r4257
2013-03-01 22:36:21 +00:00
Dave Zarzycki
9d777dfad7 Revert \a, \b, \f
This branch should have been committed a long time ago. Sorry.

Swift SVN r4256
2013-03-01 22:36:19 +00:00
Dave Zarzycki
f043cdca5a Revert "wipUnifyFormToken"
This reverts commit 39e8ace65c29d9e83f4fbebc24be9c9896e9a3e1.

Sorry!

Swift SVN r4185
2013-02-25 03:13:14 +00:00
Dave Zarzycki
06d3c09d97 wipUnifyFormToken
Swift SVN r4183
2013-02-25 03:11:11 +00:00
Dave Zarzycki
9b1a46c910 Treat zero length files as an empty line
<rdar://problem/13211640> "Missing newline at end of file" when input file is zero-length (truly "empty")

Swift SVN r4171
2013-02-24 01:24:11 +00:00
Joe Groff
ddb7ead55c REPL: Contextual completions.
If the completion prefix has a '.' behind it, guesstimate a context expression by lexing backward through an identifier(.identifier)* dotted path, then attempt to parse and typecheck that expression to decide on a base type in which to find completions.

Swift SVN r4063
2013-02-16 20:07:50 +00:00
Dave Zarzycki
177e243ad5 Tokens should not be named after their use
This fixes an old TODO item based on feedback from Chris and John.

Swift SVN r4038
2013-02-13 21:40:43 +00:00
Dave Zarzycki
5b92dae801 Make '=' consistent with the assignment operators
It seems wrong that '=' behaves differently than the compound assignment
operators and comparison operators.

I don't feel strongly about this behavior in and of itself. We can always
revert this change later if we want to rationalize why '=' should have
grammatically different rules than say '+='.

Swift SVN r4027
2013-02-13 06:02:10 +00:00
Dave Zarzycki
bf3c98a0e6 Revert \v from r3968
Nobody can find any modern use for \v.

Swift SVN r3990
2013-02-08 18:19:58 +00:00
Dave Zarzycki
a282c35e6a Make floating-point number parsing more strict
The range operators (".." and someday "...") make constructs like .24...42
ambiguous. Therefore, we will enforce that programmers either place digits
on both sides of the decimal place, or use of exponent based notation.

Swift SVN r3989
2013-02-08 15:48:02 +00:00
Dave Zarzycki
1ca414e1c9 Part two of r3968
Other subsystems need to be in sync with r3968.

Swift SVN r3978
2013-02-07 00:12:51 +00:00
Dave Zarzycki
9ad1e4c950 C/C++/ObjC Interop: string/char escapes
If we're going to import C/C++/ObjC code, we ought to "just work" with
their escape patterns when reasonable. (No error-prone octal escapes or
trigraph support). Also, update LangRef to document what the escapes do.

This patch DOES fix a bug with lexing operators in the case of (already
warned about) embedded NUL bytes within a source file.

This patch DOES NOT change what we consider to be valid whitespace in
the language.

Swift SVN r3970
2013-02-06 17:51:18 +00:00
Dave Zarzycki
89865a0dd5 Warn when missing a newline at EOF
While we are not C, we should not ignore the strong Unix command-line
tool conventions that motivated the C standard to make this be a
requirement of the language and a warning in clang/gcc.

Swift SVN r3946
2013-02-05 01:30:50 +00:00
Joe Groff
081787c3d5 Lexer: Lex C99-style hexadecimal float literals.
APFloat's parser gives us the parsing for free. Unlike C99 we require at least one digit on both sides of the hexadecimal point in order to allow '0x1.method()' expressions, similar to Dave's proposed change to float lexing. Also, we were requiring a sign after 'e' in the exponent, which is inconsistent with C, C++, and the Java regex we claim to follow, so I made the exponent sign optional.

Swift SVN r3940
2013-02-03 19:06:08 +00:00
Dave Zarzycki
cd3f31fa9d Formalize "starting" vs "following" token generation
If we generalize John's insight about l_(paren|square) being about
"starting" and "following" tokens, then we can detect many statement
or declaration boundaries that are lacking either white space or a
semicolon.

Ensuring some amount of whitespace between statements and declarations
is good for future proofing.

Swift SVN r3914
2013-01-31 18:57:53 +00:00