Change the lexing of '?' to be similar to '!', where we special-case the postfix case for the intrinsic postfix optional operator, but fall back to lexing as an operator when it isn't left-bound. For now, only accept '??' as an operator name--we could easily generalize this, but that warrants discussion first.
Swift SVN r20591
swift x.swift
x.swift:3:15: error: unicode escapes changed, please use the \u{42} syntax instead
var x = "foo \x42 \u1234 \U12345678"
^~~~
u{42}
x.swift:3:20: error: unicode escapes changed, please use the \u{1234} syntax instead
var x = "foo \x42 \u1234 \U12345678"
^~~~~~
u{1234}
x.swift:3:27: error: unicode escapes changed, please use the \u{12345678} syntax instead
var x = "foo \x42 \u1234 \U12345678"
^~~~~~~~~~
u{12345678}
No testcase since this is all temporary anyway.
Swift SVN r19449
This consolidates the \x, \u, and \U escape sequences into one \u{abc} escape sequence.
For now we still parse and cleanly reject the old forms with a nice error message, this
will eventually be removed in a later beta (tracked by rdar://17527814)
Swift SVN r19435
This patch extends the syntax with a new #line directive that is inspired
by the homonymous CPP directive. It can be specified in all locations a #if
is legal (Stmt, Decl).
Semantics
---------
#line 42 "file.swift"
This makes diagnostics and debug information behave as if the subsequent
lines came from file.swift+42.
#line // without arguments
This switches back to the main source file and the switches back to the
normal line numbering. Any previous #line directives will result in gaps
in the main file.
Rationale
---------
LLDB and the REPL need this for making expressions that are entered into
the expression evaluator or REPL debugable. For more info see
<rdar://problem/17441710> Need #line directive or something similar so we can enhance the debugging of expressions and REPL
Also, I believe the stdlib would benefit from this and it would allow us
to get rid of the line-directive wrapper script.
Swift SVN r19384
In UTF-8 decoder:
- implement U+FFFD insertion according to the recommendation given in the
Unicode spec. This required changing the decoder to become stateful, which
significantly increased complexity due to the need to maintain an internal
buffer.
- reject invalid code unit sequences properly instead of crashing rdar://16767868
- reject overlong sequences rdar://16767911
In stdlib:
- change APIs that assume that UTF decoding can never fail to account for
possibility of errors
- fix a bug in UnicodeScalarView that could cause a crash during backward
iteration if U+8000 is present in the string
- allow noncharacters in UnicodeScalar. They are explicitly allowed in the
definition of "Unicode scalar" in the specification. Disallowing noncharacters
in UnicodeScalar prevents actually using these scalar values as internal
special values during string processing, which is exactly the reason why they
are reserved in the first place.
- fix a crash in String.fromCString() that could happen if it was passed a null
pointer
In Lexer:
- allow noncharacters in string literals. These Unicode scalar values are not
allowed to be exchanged externally, but it is totally reasonable to have them
in literals as long as they don't escape the program. For example, using
U+FFFF as a delimiter and then calling str.split("\uffff") is completely
reasonable.
This is a lot of changes in a single commit; the primary reason why they are
lumped together is the need to change stdlib APIs to account for the
possibility of UTF decoding failure, and this has long-reaching effects
throughout stdlib where these APIs are used.
Swift SVN r19045
...in preparation for non-source locations, i.e. locations that don't come
frome source buffers.
No functionality change, but a fair bit of SourceManager API and idioms have
changed.
Swift SVN r18942
I didn't want to rip this logic out wholesale. There is a possibility
the character lexing can be reborn/revisited later, and
disabling it in the parser was easy.
Swift SVN r18102
Part of <rdar://problem/16742001>. At the moment, this is just a
parsing thing, because argument names are still API by default
anyway.
Swift SVN r16991
This restructures IfConfigDecl/Stmt to be a list of clauses controlled
by a condition. This makes it straight-forward to drop in #elseif support.
While I'm in here, this patch moves checking for extraneous stuff at the
end of the #if line from the lexer to the parser. This means that you can
now put a comment on the same line as a #if/#else/#elseif/#endif.
Swift SVN r16912
This is meant to be utilized for a narrow set of scenarios specific to dogfooding our pre-1.0 compiler, so please do not take any dependencies on this. In fact, I'll be removing this in the next milestone. (See rdar://problem/16380797.)
Also included - improve error recovery when parsing broken build configuration clauses.
Swift SVN r15694
* replaced yet another variant of isWhitespace with the version from
clang/Basic/CharInfo.h. The major difference is that our variant used to
consider '\0' whitespace.
* made sure that we don't construct StringRefs that point after the end of the
buffer. If the buffer ends with "#", then MemoryBuffer will only guarantee
that there is one additional NUL character. memcmp(), OTOH, is allowed to
access the complete span of the provided memory. I colud not actually get
this to crash on OSX 10.10, but I do remember similar crashes we fixed in Clang.
* added checks to reject extra tokens at the end of the build configuration
directive -- see tests, that code used to compile without diagnostics. The
lexer tried to do this, but in a self-referential way -- by checking the
NextToken variable (which is actually the previous token, when viewed from
the point of lexImpl()). The checks I added are a little too strict, they
reject comments at the end of the directive, but at least we don't accept
strange constructs. Allowing comments would not be hard, just requires
factoring out lexer's routines to skip comments so that they accept a pointer
to the buffer and return the comment end point. Filed
<rdar://problem/16301704> Allow comments at the end of bulid configuration directives
for that.
Found by inspection... I was grepping the codebase for 'isWhitespace'.
Swift SVN r14959
Track whether an identifier token is an escaped identifier token so that 'isContextualKeyword' can say "no" when an identifier is escaped.
Swift SVN r14712
Lex a backtick-enclosed `[:identifier_start:][:identifier_cont:]+` as an identifier, even if it's a Swift keyword. For now, require that the escaped name still be a valid identifier, keyword collisions notwithstanding. (We could in theory allow an arbitrary string, but we'd have to invent a mangling for non-identifier characters and do other tooling which doesn't seem productive.)
Swift SVN r14671
- Respond to Doug's code review feedback
- Stop hacking around with scopes and use "emplace" to work around RAII in the inactive config case
- Limit use of StringRef on the front-end, in favor of std::string
- Use ArrayRef rather than SmallVector within IfConfigDecl
- Reorder new property declarations on BraceStmt to prevent unnecessary alignment issues
- Update ParseBraceItems to better capture top-level declarations, rather than using token lookahead
Swift SVN r14306
These changes add support for build and target configurations in the compiler.
Build and target configurations, combined with the use of #if/#else/#endif allow
for conditional compilation within declaration and statement contexts.
Build configurations can be passed into the compiler via the new '-D' flag, or
set within the LangOptions class. Target configurations are implicit, and
currently only "os" and "arch" are supported.
Swift SVN r14305
We can attach comments to declarations. Right now we only support comments
that precede the declarations (trailing comments will be supported later).
The implementation approach is different from one we have in Clang. In Swift
the Lexer attaches the comments to the next token, and parser checks if
comments are present on the first token of the declaration. This is much
cleaner, and faster than Clang's approach (where we perform a binary search on
source locations and do ad-hoc fixups afterwards).
The comment <-> decl correspondence is modeled as "virtual" attributes that can
not be spelled in the source. These attributes are not serialized at the
moment -- this will be implemented later.
Swift SVN r14031
outside of debugger-support mode. Rip out the existing special-case
code when parsing expr-identifier.
This means that the Lexer needs a LangOptions. Doug and I
talked about just adding that as a field of SourceMgr, but
decided that it was worth it to preserve the possibility of
parsing different dialects in different source files.
By design, the lexer doesn't tokenize fundamentally differently
in different language modes; it might decide something is invalid,
or it might (eventually) use a different token kind for the
same consumed text, but we don't want it deciding to consume more or
less of the stream per token.
Note that SIL mode does make that kind of difference, and that
arguably means that various APIs for tokenizing need to take a
"is SIL mode" flag, but we're getting away with it because we
just don't really care about fidelity of SIL source files.
rdar://14899000
Swift SVN r13896
Thanks to the way we've set up our diagnostics engine, there's not actually
a reason for /everything/ to get rebuilt when /one/ diagnostic changes.
I've split them up into five categories for now: Parse, Sema, SIL, IRGen,
and Frontend, plus a set of "Common" diagnostics that are used in multiple
areas of the compiler. We can massage this later.
No functionality change, but should speed up compile times!
Swift SVN r12438
As with the monadic '?', we treat any left-bound '!' as a postfix
operator. Currently, it extracts the value of its optional
subexpression, failing at run-time if the optional is empty.
Swift SVN r8948
