Also, remove calls to isSwiftReservedName in
ClangImporter::Implementation::importName(), since 'true' and 'false'
are now keywords and rdar://problem/13187570 is no longer a problem.
rdar://problem/18797808
Swift SVN r23244
When a subclass is missing a required initializer, produce an error
within the subclass that mentions the required initializer along with
a Fix-It that provides an initializer stub, e.g.,
required init(coder aDecoder: NSCoder!) {
fatalError("init(coder:) has not been implemented")
}
We take care to insert this stub in the main class, after all of the
initializers (if there are any) or near the beginning of the class (if
there aren't any initializers), and try to match the existing
indentation. If this works out, we should handle unsatisfied protocol
requirements the same way. <rdar://problem/17923210>
Swift SVN r21055
backtracking, because we didn't restore the lexer state to before the comment, preventing
it from being attached to the token that followed it after backtracking was restored.
This is super obscure right now, but causes two tests to fail with my forthcoming patch,
lets nip this in the bud.
Swift SVN r19553
This consolidates the \x, \u, and \U escape sequences into one \u{abc} escape sequence.
For now we still parse and cleanly reject the old forms with a nice error message, this
will eventually be removed in a later beta (tracked by rdar://17527814)
Swift SVN r19435
* replaced yet another variant of isWhitespace with the version from
clang/Basic/CharInfo.h. The major difference is that our variant used to
consider '\0' whitespace.
* made sure that we don't construct StringRefs that point after the end of the
buffer. If the buffer ends with "#", then MemoryBuffer will only guarantee
that there is one additional NUL character. memcmp(), OTOH, is allowed to
access the complete span of the provided memory. I colud not actually get
this to crash on OSX 10.10, but I do remember similar crashes we fixed in Clang.
* added checks to reject extra tokens at the end of the build configuration
directive -- see tests, that code used to compile without diagnostics. The
lexer tried to do this, but in a self-referential way -- by checking the
NextToken variable (which is actually the previous token, when viewed from
the point of lexImpl()). The checks I added are a little too strict, they
reject comments at the end of the directive, but at least we don't accept
strange constructs. Allowing comments would not be hard, just requires
factoring out lexer's routines to skip comments so that they accept a pointer
to the buffer and return the comment end point. Filed
<rdar://problem/16301704> Allow comments at the end of bulid configuration directives
for that.
Found by inspection... I was grepping the codebase for 'isWhitespace'.
Swift SVN r14959
Lex a backtick-enclosed `[:identifier_start:][:identifier_cont:]+` as an identifier, even if it's a Swift keyword. For now, require that the escaped name still be a valid identifier, keyword collisions notwithstanding. (We could in theory allow an arbitrary string, but we'd have to invent a mangling for non-identifier characters and do other tooling which doesn't seem productive.)
Swift SVN r14671
We can attach comments to declarations. Right now we only support comments
that precede the declarations (trailing comments will be supported later).
The implementation approach is different from one we have in Clang. In Swift
the Lexer attaches the comments to the next token, and parser checks if
comments are present on the first token of the declaration. This is much
cleaner, and faster than Clang's approach (where we perform a binary search on
source locations and do ad-hoc fixups afterwards).
The comment <-> decl correspondence is modeled as "virtual" attributes that can
not be spelled in the source. These attributes are not serialized at the
moment -- this will be implemented later.
Swift SVN r14031
outside of debugger-support mode. Rip out the existing special-case
code when parsing expr-identifier.
This means that the Lexer needs a LangOptions. Doug and I
talked about just adding that as a field of SourceMgr, but
decided that it was worth it to preserve the possibility of
parsing different dialects in different source files.
By design, the lexer doesn't tokenize fundamentally differently
in different language modes; it might decide something is invalid,
or it might (eventually) use a different token kind for the
same consumed text, but we don't want it deciding to consume more or
less of the stream per token.
Note that SIL mode does make that kind of difference, and that
arguably means that various APIs for tokenizing need to take a
"is SIL mode" flag, but we're getting away with it because we
just don't really care about fidelity of SIL source files.
rdar://14899000
Swift SVN r13896
I tried hard find all references to 'func' in documentation, comments and
diagnostics, but I am sure that I missed a few. If you find something, please
let me know.
rdar://15346654
Swift SVN r9886
Lexer::BufferID is the single point of truth, Lexer::BufferStart and
Lexer::BufferEnd is just a cache -- they always point to the beginning and end
of the buffer, even in a sublexer.
Swift SVN r7079
Now we have a clear separation between a primary lexer, which scans the whole
buffer, and a sublexer, which can be created from a primary lexer to scan a
part of the buffer.
Swift SVN r7077
Replace uses of it with the newly introduced constructor that accepts a buffer ID.
The StringRef constructor was rather unsafe since it had the implicit requirement that the StringRef
was null-terminated.
Swift SVN r6942
Decouple splitting an interpolated string to segments, from encoding the string segments.
This allows us to tokenize or re-lex a string literal without having to allocate memory for
encoding the string segments when we don't need them encoded.
Swift SVN r6940
-Parse the expression using a begin/end state sub-Lexer instead of a general StringRef Lexer
-Introduce a Lexer constructor accepting a BufferID and a range inside the buffer, and use it for swift::tokenize.
Swift SVN r6938
Also centralizes the knowledge about whether the hashbang is allowed in the
SourceManager. This fixes a bug in tokenize() because previously it just had
to guess.
Swift SVN r6822
around everywhere
Fixes:
rdar://14585108 Code completion does not work at the beginning of the file
rdar://14592634 Code completion returns zero results at EOF in a function
without a closing brace
Swift SVN r6820
completion token
This is required to handle cases like fooObject.#^A^#.bar where code completion
is invoked inside the ".." token. Previously, the token would not be split and
the lexer would produce an incorrect tokenization for this case. Now we
produce ".", tok::code_complete, ".".
Swift SVN r6635
-Introduce PersistentParserState to represent state persistent among multiple parsing passes.
The advantage is that PersistentParserState is independent of a particular Parser or Lexer object.
-Use PersistentParserState to keep information about delayed function body parsing and eliminate parser-specific
state from the AST (ParserTokenRange).
-Introduce DelayedParsingCallbacks to abstract out of the parser the logic about which functions should be delayed
or skipped.
Many thanks to Dmitri for his valuable feedback!
Swift SVN r6580
* Added a mode in swift-ide-test to test code completion. Unlike c-index-test,
the code completion token in tests is a real token -- we don't need to
count lines and columns anymore.
* Added support in lexer to produce a code completion token.
* Added a parser interface to code completion. It is passed down from the
libFrontend to the parser, but its functions are not called yet.
* Added a sketch of the interface of code completion consumer and code
completion results.
Note: all this is not doing anything useful yet.
Swift SVN r6128
This is to make overload resolution for the protected constructor call sites more explicit.
Otherwise, adding a parameter in the protected constructor and a default parameter in the public one
can inadvertently result in the protected constructor call sites switching to picking the public one.
Swift SVN r6007
When we are interpreting escape sequences in the lexer, we copy the string
literal bytes to ASTContext instead of storing a pointer to the source buffer.
But then we used to try to get a source location for that string in the heap,
which is not a valid source buffer. It succeeds during parsing, but breaks
when we try to print a diagnostic using this location.
Added a verifier check for this.
Also added a real source range for StringLiteralExpr, instead of a source range
with begin == end, produced from the beginning location.
Swift SVN r5961
In order to do this, we need to save and restore parser state easily. The
important pieces of state are:
* lexer position;
* lexical scope stack.
Lexer position can be saved/restored easily. We don't need to store the tokens
for the function body because swift does not have a preprocessor and we can
easily re-lex everything we need. We just store the lexer state for the
beginning and the end of the body.
To save the lexical scope stack, we had to change the underlying data
structure. Originally, the parser used the ScopedHashTable, which supports
only a stack of scopes. But we need a *tree* of scopes. I implemented
TreeScopedHashTable based on ScopedHashTable. It has an optimization for
pushing/popping scopes in a stack fashion -- these scopes will not be allocated
on the heap. While ‘detached’ scopes that we want to re-enter later, and all
their parent scopes, are moved to the heap.
In parseIntoTranslationUnit() we do a second pass over the 'structural AST'
that does not contain function bodies to actually parse them from saved token
ranges.
Swift SVN r5886
