Commit Graph

727 Commits

Author SHA1 Message Date
Dmitri Hrybenko
13070a31bd SourceManager: make findBufferContainingLoc() always succeed.
Every valid source location corresponds to a source buffer.  There should be no
cases where we create a source location for a random string.  Thus,
findBufferContainingLoc() always succeeds.


Swift SVN r7120
2013-08-10 01:48:04 +00:00
Dmitri Hrybenko
ca5351e4b6 Fix bad recovery in Parser::parseList() and Parser::parseDeclUnionElement()
Now parseList() always sets RBLoc, which fixes the crash.
parseDeclUnionElement() now skips to the next decl, instead of ":", which may
not appear in the file at all.


Swift SVN r7117
2013-08-10 00:44:09 +00:00
Dmitri Hrybenko
8f7a437d44 Wrap a few functions from LLVM SourceMgr in preparation of making
SourceLoc::Value private


Swift SVN r7114
2013-08-09 23:15:59 +00:00
Dmitri Hrybenko
dc655eb1cb Remove more abuse of SourceLoc::Value::getPointer()
Swift SVN r7109
2013-08-09 22:35:33 +00:00
Dmitri Hrybenko
db08a32a95 Factor out SourceManager::getLocOffsetInBuffer()
and remove some abuse of SourceLoc::Value::getPointer()


Swift SVN r7105
2013-08-09 21:53:32 +00:00
Dmitri Hrybenko
48faba29f3 Simplify parser constructors
Constructor delegation in parser was useless, because the code was split
between the constructors arbitrarily.

There was no need to pass down IsMainModule because the parser could figure
that out on its own.  Also rename it to allowTopLevelCode() to better describe
what it actually affects.


Swift SVN r7098
2013-08-09 20:43:21 +00:00
Dmitri Hrybenko
de59d8dcd4 Remove unneeded llvm:: qualifier for llvm::StringRef and llvm::SmallVector
Swift SVN r7089
2013-08-09 18:41:46 +00:00
Dmitri Hrybenko
d1453b3f9d Lexer constuctors: allow to create a lexer that scans a subrange without having
to create a parent lexer first.


Swift SVN r7085
2013-08-09 18:19:55 +00:00
Dmitri Hrybenko
49e15b33e7 Remove the public Lexer::getBufferEnd() function which breaks encapsulation
of the source buffer


Swift SVN r7084
2013-08-09 18:10:29 +00:00
Dmitri Hrybenko
c92fd7cbdd Lexer: refactor constructors to bring the parameter count back to a sane number
Now we have a clear separation between a primary lexer, which scans the whole
buffer, and a sublexer, which can be created from a primary lexer to scan a
part of the buffer.


Swift SVN r7077
2013-08-09 02:06:50 +00:00
Dmitri Hrybenko
2bbef342f0 Code completion: refactor code completion callbacks so that actual name lookups
happen after delayed parsing is finished.  This ensures that the AST for
delayed parsed code (for example, function body) is constructed.  This is
required for partial type checking of function bodies.


Swift SVN r7010
2013-08-07 22:42:36 +00:00
Argyrios Kyrtzidis
4908da8361 [Lexer] Remove the public Lexer constructor that accepts a StringRef.
Replace uses of it with the newly introduced constructor that accepts a buffer ID.
The StringRef constructor was rather unsafe since it had the implicit requirement that the StringRef
was null-terminated.

Swift SVN r6942
2013-08-06 14:59:03 +00:00
Argyrios Kyrtzidis
4e297d044b Move tokenization of interpolated strings in swift::tokenize().
Swift SVN r6941
2013-08-06 14:59:02 +00:00
Argyrios Kyrtzidis
47c04e760a [Parser] Remove a constructor that is not used anymore.
Swift SVN r6939
2013-08-06 14:59:00 +00:00
Argyrios Kyrtzidis
cda7a109bd [Lexer] Remove the special hack in the Lexer that handles lexing an expression in an interpolated string.
-Parse the expression using a begin/end state sub-Lexer instead of a general StringRef Lexer
-Introduce a Lexer constructor accepting a BufferID and a range inside the buffer, and use it for swift::tokenize.

Swift SVN r6938
2013-08-06 14:58:59 +00:00
Dmitri Hrybenko
7684d6dc2a Add a good diagnostic for a hashbang line when it is not allowed
Also centralizes the knowledge about whether the hashbang is allowed in the
SourceManager.  This fixes a bug in tokenize() because previously it just had
to guess.


Swift SVN r6822
2013-08-01 23:07:43 +00:00
Dmitri Hrybenko
5cea4ebf41 Code completion: put CodeCompletionOffset on SourceManager instead of passing
around everywhere

Fixes:
rdar://14585108 Code completion does not work at the beginning of the file
rdar://14592634 Code completion returns zero results at EOF in a function
                without a closing brace


Swift SVN r6820
2013-08-01 22:08:58 +00:00
Dmitri Hrybenko
e1c4ae3174 Wrap llvm::SourceMgr in swift::SourceManager so that we can add new members
to the source manager.


Swift SVN r6815
2013-08-01 20:39:22 +00:00
Dmitri Hrybenko
015cea67a5 Move hashbang line detection to the Lexer implementation, instead of requiring
the Lexer client to skip it before constructing the Lexer.


Swift SVN r6801
2013-08-01 02:18:25 +00:00
Dmitri Hrybenko
e98ab3d83a swift::parseCompletionContextExpr() is dead code now, remove it.
Swift SVN r6794
2013-08-01 01:22:13 +00:00
Dmitri Hrybenko
bec5e3eea7 swift::tokenize: create the Lexer on the stack
Swift SVN r6718
2013-07-29 22:37:51 +00:00
Dmitri Hrybenko
72ae1fd842 swift::tokenize: don't include tok::eof, per feedback from Argyrios
Swift SVN r6716
2013-07-29 22:23:31 +00:00
Dmitri Hrybenko
5142210a3e Code completion: delay parsing of top-level var decls
This allows us to do code completion inside top-lever var initializers.

As a part of implementation, we make sure that error recovery does not
implicitly skip over the code completion token.  This also fixes a bug that
prevented us from doing code completion inside function argument list, added
tests for that.


Swift SVN r6708
2013-07-29 21:31:44 +00:00
Dmitri Hrybenko
a779b658e3 Don't dereference the text of tok::eof, because it is usually empty. Use
string comparison instead.


Swift SVN r6692
2013-07-29 17:43:46 +00:00
Dmitri Hrybenko
6beee6a62e Code completion: implement delayed parsing and code completion for
TopLevelCodeDecls


Swift SVN r6638
2013-07-26 01:44:23 +00:00
Dmitri Hrybenko
497ec93ec9 Code completion: generate a tok::code_complete token during the first pass of
parsing

No functionality change, this will be used by next commits.


Swift SVN r6637
2013-07-26 01:40:47 +00:00
Dmitri Hrybenko
464df1cc11 Lexer: make tok::eof length equal to 0
It used to be equal to 1, which makes Lexer::getLocForEndOfToken() return
an out-of-bounds location for tok::eof.


Swift SVN r6626
2013-07-26 00:14:09 +00:00
Argyrios Kyrtzidis
51990109e5 If the parser needs priming, let the parser handle it.
Swift SVN r6597
2013-07-25 14:42:10 +00:00
Argyrios Kyrtzidis
803ba0e2be [Parser] Introduce PersistentParserState::ParserPos to encapsulate a Parser/Lexer independent info for restoring parsing from a certain token location.
Swift SVN r6593
2013-07-25 14:42:07 +00:00
Argyrios Kyrtzidis
cfd3e09e1e [Parser] Make PersistentParserState parameter optional and remove it from the constructor that accepts a StringRef fragment.
Swift SVN r6592
2013-07-25 14:42:06 +00:00
Argyrios Kyrtzidis
66d1e516c8 Refactor how multiple parsing passes and delayed parsing works.
-Introduce PersistentParserState to represent state persistent among multiple parsing passes.
  The advantage is that PersistentParserState is independent of a particular Parser or Lexer object.
-Use PersistentParserState to keep information about delayed function body parsing and eliminate parser-specific
  state from the AST (ParserTokenRange).
-Introduce DelayedParsingCallbacks to abstract out of the parser the logic about which functions should be delayed
  or skipped.

Many thanks to Dmitri for his valuable feedback!

Swift SVN r6580
2013-07-25 01:40:16 +00:00
Chris Lattner
500d90995c Change the sentinel used by the parser to represent unprimed state to be something the lexer can't produce.
Swift SVN r6447
2013-07-22 14:48:44 +00:00
Doug Gregor
50d6fd0455 Lock down on the use of default values in patterns and types.
Per previous discussions, we only want to allow default values for
uncurried 'func' and 'constructor' parameters, and not for return
types or arbitrary tuple types. Introduce this restriction, fixing
part of <rdar://problem/13372694>. 



Swift SVN r6156
2013-07-11 17:53:05 +00:00
Dmitri Hrybenko
3c5b12fc0f Code completion: add a lot of infrastructure code
* Added a mode in swift-ide-test to test code completion.  Unlike c-index-test,
  the code completion token in tests is a real token -- we don't need to
  count lines and columns anymore.

* Added support in lexer to produce a code completion token.

* Added a parser interface to code completion.  It is passed down from the
  libFrontend to the parser, but its functions are not called yet.

* Added a sketch of the interface of code completion consumer and code
  completion results.

Note: all this is not doing anything useful yet.


Swift SVN r6128
2013-07-10 20:53:40 +00:00
Doug Gregor
85231a5d16 Remove || closures.
Swift SVN r6119
2013-07-10 17:40:52 +00:00
Dmitri Hrybenko
cb98234d67 Allow the parser to persist after parseIntoTranslationUnit() returns
Swift SVN r6102
2013-07-10 00:25:37 +00:00
Dmitri Hrybenko
b92a157bc0 Remove the llvm::MemoryBuffer member from Parser, it is not used anywhere.
Swift SVN r6098
2013-07-09 22:43:39 +00:00
Dmitri Hrybenko
1c0233efb1 Move lib/Parse/{Parser.h, Scope.h} -> include/swift/Parse/
Swift SVN r6062
2013-07-08 20:36:40 +00:00
Dmitri Hrybenko
c7421211a3 Factor out logic that creates and configures all objects required for
compilation into a 'Frontend' library.

This library is still not in its best shape, but now it can be reused in
swift-index-test.


Swift SVN r6040
2013-07-06 00:46:19 +00:00
Argyrios Kyrtzidis
6e7d0490f7 Allow optionally to produce comment tokens when lexing and add a tokenize() utility function.
Swift SVN r6008
2013-07-05 15:02:42 +00:00
Dmitri Hrybenko
2311e4ec16 Don't try to do delayed body parsing for a function that does not have a body.
Also actually check that we call getBodyTokenRange() only when the correct
union member is active.


Swift SVN r5965
2013-07-02 23:06:08 +00:00
Chris Lattner
20229fbf85 Introduce infrastructure for maintaining per-translation unit SIL parser state across
invocations of the parser.



Swift SVN r5906
2013-06-30 18:44:59 +00:00
Dmitri Hrybenko
f73d866d91 Implement delayed parsing for function bodies
In order to do this, we need to save and restore parser state easily.  The
important pieces of state are:

* lexer position;
* lexical scope stack.

Lexer position can be saved/restored easily.  We don't need to store the tokens
for the function body because swift does not have a preprocessor and we can
easily re-lex everything we need.  We just store the lexer state for the
beginning and the end of the body.

To save the lexical scope stack, we had to change the underlying data
structure.  Originally, the parser used the ScopedHashTable, which supports
only a stack of scopes.  But we need a *tree* of scopes.  I implemented
TreeScopedHashTable based on ScopedHashTable.  It has an optimization for
pushing/popping scopes in a stack fashion -- these scopes will not be allocated
on the heap.  While ‘detached’ scopes that we want to re-enter later, and all
their parent scopes, are moved to the heap.

In parseIntoTranslationUnit() we do a second pass over the 'structural AST'
that does not contain function bodies to actually parse them from saved token
ranges.


Swift SVN r5886
2013-06-28 22:38:10 +00:00
Joe Groff
10f136f7d3 Parse: Remove 'OtherThanAssignment' from 'parseStmt'.
Assignment isn't a Stmt anymore.

Swift SVN r5832
2013-06-27 05:13:33 +00:00
Chris Lattner
1040bfec6a In the REPL, allow access to the Builtin module if explicitly imported,
there is no reason to deny it and it could be theoretically useful.


Swift SVN r5779
2013-06-24 16:07:56 +00:00
Chris Lattner
bb82daca8f generalize parseToken to take an arbitrary argument list for the diagnostic
when it fails.

Teach the SIL parser to parse and validate (but still discard) tuple and
return instructions.


Swift SVN r5318
2013-05-25 00:40:46 +00:00
Chris Lattner
bd217c9227 extend the general form of parseIdentifier to optionally return a SourceLoc,
and use this throughout the parser.


Swift SVN r5314
2013-05-25 00:07:56 +00:00
Chris Lattner
ce314e59c1 - introduce a form of Parser::parseIdentifier that captures a token location,
requiring us to make the variadic templates a bit more specific.

- Make SIL parser tests use updated syntax (still not parsed).


Swift SVN r5312
2013-05-24 23:59:21 +00:00
Chris Lattner
efea0bc2f9 Simplify the Parser ctors to just take the TranslationUnit being parsed into, save this as an ivar.
Swift SVN r5248
2013-05-21 05:33:42 +00:00
Chris Lattner
f2e3af3cf6 Eliminate the "global" list of unresolved identifier types maintained
by TranslationUnit.  This list existed solely to allow name lookup of
an unbound IdentifierType to know its DeclContext.  Instead of indirecting
through this list, just store the DeclContext in the IdentifierType in its
uninitialized state.

This eliminates a really terrible performance fixme about scanning the list,
eliminates the management fiddling around with this list in the parser, and
is generally much cleaner.


Swift SVN r5246
2013-05-21 05:27:37 +00:00