Commit Graph

309 Commits

Author SHA1 Message Date
Joe Pamer
988a5877f2 Some updates:
- Respond to Doug's code review feedback
   - Stop hacking around with scopes and use "emplace" to work around RAII in the inactive config case
   - Limit use of StringRef on the front-end, in favor of std::string
   - Use ArrayRef rather than SmallVector within IfConfigDecl
   - Reorder new property declarations on BraceStmt to prevent unnecessary alignment issues
- Update ParseBraceItems to better capture top-level declarations, rather than using token lookahead

Swift SVN r14306
2014-02-24 18:16:49 +00:00
Joe Pamer
f83f94d9d8 Support build and target configurations
These changes add support for build and target configurations in the compiler.
Build and target configurations, combined with the use of #if/#else/#endif allow
for conditional compilation within declaration and statement contexts.

Build configurations can be passed into the compiler via the new '-D' flag, or
set within the LangOptions class. Target configurations are implicit, and
currently only "os" and "arch" are supported.

Swift SVN r14305
2014-02-24 18:16:48 +00:00
Dmitri Hrybenko
6871e96df7 Lexer: don't use locale-dependent ctype.h functions
The dependency on Clang is unfortunate here, hopefully these functions will be
moved to LLVM.


Swift SVN r14288
2014-02-23 18:47:15 +00:00
Dmitri Hrybenko
ecd798b9fd Comment parsing: attaching comments to declarations
We can attach comments to declarations.  Right now we only support comments
that precede the declarations (trailing comments will be supported later).

The implementation approach is different from one we have in Clang.  In Swift
the Lexer attaches the comments to the next token, and parser checks if
comments are present on the first token of the declaration.  This is much
cleaner, and faster than Clang's approach (where we perform a binary search on
source locations and do ad-hoc fixups afterwards).

The comment <-> decl correspondence is modeled as "virtual" attributes that can
not be spelled in the source.  These attributes are not serialized at the
moment -- this will be implemented later.


Swift SVN r14031
2014-02-18 09:04:37 +00:00
Chris Lattner
3404f27237 remove 'let' from the parser.
Swift SVN r13995
2014-02-17 16:56:54 +00:00
John McCall
10ac15ed0d Lex $notAllDigits as an identifier and diagnose it in the lexer
outside of debugger-support mode.  Rip out the existing special-case
code when parsing expr-identifier.

This means that the Lexer needs a LangOptions.  Doug and I
talked about just adding that as a field of SourceMgr, but
decided that it was worth it to preserve the possibility of
parsing different dialects in different source files.

By design, the lexer doesn't tokenize fundamentally differently
in different language modes; it might decide something is invalid,
or it might (eventually) use a different token kind for the
same consumed text, but we don't want it deciding to consume more or
less of the stream per token.

Note that SIL mode does make that kind of difference, and that
arguably means that various APIs for tokenizing need to take a
"is SIL mode" flag, but we're getting away with it because we
just don't really care about fidelity of SIL source files.

rdar://14899000

Swift SVN r13896
2014-02-14 01:54:17 +00:00
Chris Lattner
6072e1d40d parse the 'val' keyword. For now it is a synonym for let. I will rip
'let' out (and continue migrating terminology in the compiler) as an
ongoing project.


Swift SVN r13821
2014-02-12 06:29:15 +00:00
Dave Zarzycki
5556d24f3a Parser: Remove "too many '.'s" error
This only made sense when '...' was reserved. In other words, if we
allow '..' and '...' as formal operators, then why not '....'?

Swift SVN r13673
2014-02-08 02:15:17 +00:00
Dave Zarzycki
1e3fd1a5b1 Parser: allow '...' to be an operator for ranges
<rdar://problem/16018151> Allow me to declare an operator spelled "..."

Swift SVN r13670
2014-02-08 02:10:37 +00:00
Manman Ren
729d30fb54 SILParser: first step towards parsing sil_witness_table.
Only one entry kind out of four is supported right now.


Swift SVN r12936
2014-01-24 21:58:45 +00:00
Jordan Rose
5bbdb23910 Fix up "invalid UTF-8" error, and add a test case.
Swift SVN r12447
2014-01-17 01:08:32 +00:00
Jordan Rose
11008f0ed1 Split diagnostics out into separate files.
Thanks to the way we've set up our diagnostics engine, there's not actually
a reason for /everything/ to get rebuilt when /one/ diagnostic changes.
I've split them up into five categories for now: Parse, Sema, SIL, IRGen,
and Frontend, plus a set of "Common" diagnostics that are used in multiple
areas of the compiler. We can massage this later.

No functionality change, but should speed up compile times!

Swift SVN r12438
2014-01-17 00:15:12 +00:00
Dave Zarzycki
adf26ccd2a 15836688 Fail gracefully on UTF-16 input files
Swift SVN r12420
2014-01-16 21:34:55 +00:00
Argyrios Kyrtzidis
e244f51229 [Lexer] Add some const goodness to the SourceManager that the Lexer uses.
No functionality change.

Swift SVN r12182
2014-01-11 01:09:30 +00:00
Dmitri Hrybenko
1363140c81 Correct assertion message
Swift SVN r10869
2013-12-05 18:30:38 +00:00
Dave Abrahams
7ab9d369aa [stdlib] Rename Char => UnicodeScalar
Swift SVN r10864
2013-12-05 17:30:37 +00:00
Manman Ren
05899b7a8d SILParser: parse sil_global and sil_global_addr.
rdar://15493552


Swift SVN r10823
2013-12-05 00:46:40 +00:00
Chris Lattner
ef93c81ffb Introduce a new SIL-level "undef" value, useful for SIL transformations.
IRGen support is missing, Joe volenteers to implement it.


Swift SVN r9776
2013-10-30 00:58:09 +00:00
Dmitri Hrybenko
882b770983 Portability: replace isnumber() with isdigit()
Swift SVN r9578
2013-10-22 05:05:21 +00:00
Manman Ren
d503c65036 SILParser: parse sil_vtable
rdar://15165644


Swift SVN r9342
2013-10-15 00:52:40 +00:00
Doug Gregor
611a5cce4b Replace the library-defined postfix '!' with an expr-postfix production.
As with the monadic '?', we treat any left-bound '!' as a postfix
operator. Currently, it extracts the value of its optional
subexpression, failing at run-time if the optional is empty.


Swift SVN r8948
2013-10-06 23:09:58 +00:00
Joe Groff
2728b7d23c Improve lexer error for multiple-Unicode-scalar "character" literals.
Swift SVN r8945
2013-10-05 22:22:24 +00:00
Chris Lattner
af509afe67 make '@' be a valid token, even when not in sil mode. No functionality change.
Swift SVN r8898
2013-10-04 00:18:51 +00:00
John McCall
298577676e Introduce the monadic ? operator.
A ? operator is interpreted as this if it's left-bound,
so ternary operators now have to be spaced on the left.

Swift SVN r8832
2013-10-02 01:27:45 +00:00
Argyrios Kyrtzidis
5db368ce7b [Lexer] Introduce Lexer::getLocForStartOfToken() that returns the location at the start of the token that a given offset points to.
Swift SVN r8281
2013-09-16 18:41:16 +00:00
Jordan Rose
d40f74f2c1 [Parse] Treat ! as a postfix operator when both left- and right-bound.
For most operators, an expression like "a@b" is lexed with '@' as a binary
operation on 'a' and 'b'. However, we want to make sure we can support
calling optional functions using the syntax "a!()". Consequently, if there
are no spaces around a single '!' token, it is treated as a postfix operator
instead of an infix operator. The infix interpretation can still be requested
by using spaces on both sides of the operator.

Swift SVN r7651
2013-08-28 00:18:19 +00:00
Dmitri Hrybenko
8a8b97985b Lexer: fix a bug where restoring lexer state could produce duplicate
code_complete tokens


Swift SVN r7636
2013-08-27 22:34:29 +00:00
Jordan Rose
a62cfd3c30 [Parse] Parse the '@' in "x@.y" as a postfix op.
...but make sure plain "@.y" still parses as a prefix op.

This allows our planned postfix ! operator for Optional types to parse
correctly when a member is accessed. There's probably still QoI work here:
the following member may have been intended to be a free member as in
"x @ .y", where '@' is a binary operator.

Swift SVN r7513
2013-08-23 19:03:30 +00:00
Dmitri Hrybenko
dc439adb59 Lexer: improve recovery for invalid character literals and invalid escape
sequences in charater literals

Now we return tok::character_literal with REPLACEMENT CHARACTER U+FFFD instead
of tok::unknown.


Swift SVN r7475
2013-08-22 20:56:47 +00:00
Dmitri Hrybenko
3bee330e50 Lexer: don't inf loop on an unterminated string literal near EOF
Swift SVN r7474
2013-08-22 20:40:34 +00:00
Dmitri Hrybenko
39d40c54dd Lexer: don't emit a diagnostic if it was not requested
Swift SVN r7472
2013-08-22 19:05:19 +00:00
Dmitri Hrybenko
c4775b8287 Lexer: unterminated/invalid character literals: recover by skipping to a single
quote, newline, or EOF, whichever is first


Swift SVN r7470
2013-08-22 18:51:17 +00:00
Dmitri Hrybenko
4ba6197a9c Lexer: add a specific diagnostic "unterminated character literal" instead of a
generic one "invalid character literal"


Swift SVN r7467
2013-08-22 18:36:04 +00:00
Dmitri Hrybenko
4536a90421 Lexer: fix a crash on an unterminated character literal starting right before EOF
Swift SVN r7463
2013-08-22 18:22:57 +00:00
Dmitri Hrybenko
c98719dc03 Cleanup includes and forward declarations
Swift SVN r7194
2013-08-13 03:36:47 +00:00
Dmitri Hrybenko
70f2b64ad9 Add CharSourceRange -- a half-open character range, which will be used in IDE
integration

Motivation: libIDE clients should be simple, and they should not have to
translate token-based SourceRanges to character locations.

This also allows us to remove the dependency of DiagnosticConsumer on the
Lexer.  Now the DiagnosticEngine translates the diagnostics to CharSourceRanges
and passes character-based ranges to the DiagnosticConsumer.


Swift SVN r7173
2013-08-12 20:15:51 +00:00
Dmitri Hrybenko
13070a31bd SourceManager: make findBufferContainingLoc() always succeed.
Every valid source location corresponds to a source buffer.  There should be no
cases where we create a source location for a random string.  Thus,
findBufferContainingLoc() always succeeds.


Swift SVN r7120
2013-08-10 01:48:04 +00:00
Dmitri Hrybenko
8f7a437d44 Wrap a few functions from LLVM SourceMgr in preparation of making
SourceLoc::Value private


Swift SVN r7114
2013-08-09 23:15:59 +00:00
Dmitri Hrybenko
db08a32a95 Factor out SourceManager::getLocOffsetInBuffer()
and remove some abuse of SourceLoc::Value::getPointer()


Swift SVN r7105
2013-08-09 21:53:32 +00:00
Dmitri Hrybenko
888d9b4e58 Tighten the contract of Lexer::getEncodedCharacterLiteral():
treat the token as a token (instead of a string) and check that it has the
correct kind


Swift SVN r7094
2013-08-09 20:15:37 +00:00
Dmitri Hrybenko
de59d8dcd4 Remove unneeded llvm:: qualifier for llvm::StringRef and llvm::SmallVector
Swift SVN r7089
2013-08-09 18:41:46 +00:00
Dmitri Hrybenko
d1453b3f9d Lexer constuctors: allow to create a lexer that scans a subrange without having
to create a parent lexer first.


Swift SVN r7085
2013-08-09 18:19:55 +00:00
Dmitri Hrybenko
49e15b33e7 Remove the public Lexer::getBufferEnd() function which breaks encapsulation
of the source buffer


Swift SVN r7084
2013-08-09 18:10:29 +00:00
Dmitri Hrybenko
056923e802 Lexer constructors: inline Lexer::initLexer into the principal constructor.
Introducing it was just a refactoring step.


Swift SVN r7081
2013-08-09 17:46:31 +00:00
Dmitri Hrybenko
8c15b7d1cf Lexer constructors: remove redundant code: Lexer::initLexer will query the
SourceManager and set the code completion point correctly, no need to do this
elsewhere.


Swift SVN r7080
2013-08-09 17:41:55 +00:00
Dmitri Hrybenko
ea710cd62a Lexer constructors: simplify and establish an invariant:
Lexer::BufferID is the single point of truth, Lexer::BufferStart and
Lexer::BufferEnd is just a cache -- they always point to the beginning and end
of the buffer, even in a sublexer.


Swift SVN r7079
2013-08-09 17:40:17 +00:00
Dmitri Hrybenko
c92fd7cbdd Lexer: refactor constructors to bring the parameter count back to a sane number
Now we have a clear separation between a primary lexer, which scans the whole
buffer, and a sublexer, which can be created from a primary lexer to scan a
part of the buffer.


Swift SVN r7077
2013-08-09 02:06:50 +00:00
Dmitri Hrybenko
e5923592e1 Refactor Lexer::State to store a SourceLoc intead of a pointer into the buffer
Swift SVN r7065
2013-08-08 23:51:03 +00:00
Argyrios Kyrtzidis
d6b048dfe0 [Lexer] Refactor lexing of interpolated strings.
Decouple splitting an interpolated string to segments, from encoding the string segments.
This allows us to tokenize or re-lex a string literal without having to allocate memory for
encoding the string segments when we don't need them encoded.

Swift SVN r6940
2013-08-06 14:59:01 +00:00
Argyrios Kyrtzidis
cda7a109bd [Lexer] Remove the special hack in the Lexer that handles lexing an expression in an interpolated string.
-Parse the expression using a begin/end state sub-Lexer instead of a general StringRef Lexer
-Introduce a Lexer constructor accepting a BufferID and a range inside the buffer, and use it for swift::tokenize.

Swift SVN r6938
2013-08-06 14:58:59 +00:00