Commit Graph

317 Commits

Author SHA1 Message Date
Dmitri Hrybenko
2bbef342f0 Code completion: refactor code completion callbacks so that actual name lookups
happen after delayed parsing is finished.  This ensures that the AST for
delayed parsed code (for example, function body) is constructed.  This is
required for partial type checking of function bodies.


Swift SVN r7010
2013-08-07 22:42:36 +00:00
Argyrios Kyrtzidis
4908da8361 [Lexer] Remove the public Lexer constructor that accepts a StringRef.
Replace uses of it with the newly introduced constructor that accepts a buffer ID.
The StringRef constructor was rather unsafe since it had the implicit requirement that the StringRef
was null-terminated.

Swift SVN r6942
2013-08-06 14:59:03 +00:00
Argyrios Kyrtzidis
4e297d044b Move tokenization of interpolated strings in swift::tokenize().
Swift SVN r6941
2013-08-06 14:59:02 +00:00
Argyrios Kyrtzidis
47c04e760a [Parser] Remove a constructor that is not used anymore.
Swift SVN r6939
2013-08-06 14:59:00 +00:00
Argyrios Kyrtzidis
cda7a109bd [Lexer] Remove the special hack in the Lexer that handles lexing an expression in an interpolated string.
-Parse the expression using a begin/end state sub-Lexer instead of a general StringRef Lexer
-Introduce a Lexer constructor accepting a BufferID and a range inside the buffer, and use it for swift::tokenize.

Swift SVN r6938
2013-08-06 14:58:59 +00:00
Dmitri Hrybenko
7684d6dc2a Add a good diagnostic for a hashbang line when it is not allowed
Also centralizes the knowledge about whether the hashbang is allowed in the
SourceManager.  This fixes a bug in tokenize() because previously it just had
to guess.


Swift SVN r6822
2013-08-01 23:07:43 +00:00
Dmitri Hrybenko
5cea4ebf41 Code completion: put CodeCompletionOffset on SourceManager instead of passing
around everywhere

Fixes:
rdar://14585108 Code completion does not work at the beginning of the file
rdar://14592634 Code completion returns zero results at EOF in a function
                without a closing brace


Swift SVN r6820
2013-08-01 22:08:58 +00:00
Dmitri Hrybenko
e1c4ae3174 Wrap llvm::SourceMgr in swift::SourceManager so that we can add new members
to the source manager.


Swift SVN r6815
2013-08-01 20:39:22 +00:00
Dmitri Hrybenko
015cea67a5 Move hashbang line detection to the Lexer implementation, instead of requiring
the Lexer client to skip it before constructing the Lexer.


Swift SVN r6801
2013-08-01 02:18:25 +00:00
Dmitri Hrybenko
e98ab3d83a swift::parseCompletionContextExpr() is dead code now, remove it.
Swift SVN r6794
2013-08-01 01:22:13 +00:00
Dmitri Hrybenko
bec5e3eea7 swift::tokenize: create the Lexer on the stack
Swift SVN r6718
2013-07-29 22:37:51 +00:00
Dmitri Hrybenko
72ae1fd842 swift::tokenize: don't include tok::eof, per feedback from Argyrios
Swift SVN r6716
2013-07-29 22:23:31 +00:00
Dmitri Hrybenko
5142210a3e Code completion: delay parsing of top-level var decls
This allows us to do code completion inside top-lever var initializers.

As a part of implementation, we make sure that error recovery does not
implicitly skip over the code completion token.  This also fixes a bug that
prevented us from doing code completion inside function argument list, added
tests for that.


Swift SVN r6708
2013-07-29 21:31:44 +00:00
Dmitri Hrybenko
a779b658e3 Don't dereference the text of tok::eof, because it is usually empty. Use
string comparison instead.


Swift SVN r6692
2013-07-29 17:43:46 +00:00
Dmitri Hrybenko
6beee6a62e Code completion: implement delayed parsing and code completion for
TopLevelCodeDecls


Swift SVN r6638
2013-07-26 01:44:23 +00:00
Dmitri Hrybenko
497ec93ec9 Code completion: generate a tok::code_complete token during the first pass of
parsing

No functionality change, this will be used by next commits.


Swift SVN r6637
2013-07-26 01:40:47 +00:00
Dmitri Hrybenko
464df1cc11 Lexer: make tok::eof length equal to 0
It used to be equal to 1, which makes Lexer::getLocForEndOfToken() return
an out-of-bounds location for tok::eof.


Swift SVN r6626
2013-07-26 00:14:09 +00:00
Argyrios Kyrtzidis
51990109e5 If the parser needs priming, let the parser handle it.
Swift SVN r6597
2013-07-25 14:42:10 +00:00
Argyrios Kyrtzidis
803ba0e2be [Parser] Introduce PersistentParserState::ParserPos to encapsulate a Parser/Lexer independent info for restoring parsing from a certain token location.
Swift SVN r6593
2013-07-25 14:42:07 +00:00
Argyrios Kyrtzidis
cfd3e09e1e [Parser] Make PersistentParserState parameter optional and remove it from the constructor that accepts a StringRef fragment.
Swift SVN r6592
2013-07-25 14:42:06 +00:00
Argyrios Kyrtzidis
66d1e516c8 Refactor how multiple parsing passes and delayed parsing works.
-Introduce PersistentParserState to represent state persistent among multiple parsing passes.
  The advantage is that PersistentParserState is independent of a particular Parser or Lexer object.
-Use PersistentParserState to keep information about delayed function body parsing and eliminate parser-specific
  state from the AST (ParserTokenRange).
-Introduce DelayedParsingCallbacks to abstract out of the parser the logic about which functions should be delayed
  or skipped.

Many thanks to Dmitri for his valuable feedback!

Swift SVN r6580
2013-07-25 01:40:16 +00:00
Chris Lattner
500d90995c Change the sentinel used by the parser to represent unprimed state to be something the lexer can't produce.
Swift SVN r6447
2013-07-22 14:48:44 +00:00
Doug Gregor
50d6fd0455 Lock down on the use of default values in patterns and types.
Per previous discussions, we only want to allow default values for
uncurried 'func' and 'constructor' parameters, and not for return
types or arbitrary tuple types. Introduce this restriction, fixing
part of <rdar://problem/13372694>. 



Swift SVN r6156
2013-07-11 17:53:05 +00:00
Dmitri Hrybenko
3c5b12fc0f Code completion: add a lot of infrastructure code
* Added a mode in swift-ide-test to test code completion.  Unlike c-index-test,
  the code completion token in tests is a real token -- we don't need to
  count lines and columns anymore.

* Added support in lexer to produce a code completion token.

* Added a parser interface to code completion.  It is passed down from the
  libFrontend to the parser, but its functions are not called yet.

* Added a sketch of the interface of code completion consumer and code
  completion results.

Note: all this is not doing anything useful yet.


Swift SVN r6128
2013-07-10 20:53:40 +00:00
Doug Gregor
85231a5d16 Remove || closures.
Swift SVN r6119
2013-07-10 17:40:52 +00:00
Dmitri Hrybenko
cb98234d67 Allow the parser to persist after parseIntoTranslationUnit() returns
Swift SVN r6102
2013-07-10 00:25:37 +00:00
Dmitri Hrybenko
b92a157bc0 Remove the llvm::MemoryBuffer member from Parser, it is not used anywhere.
Swift SVN r6098
2013-07-09 22:43:39 +00:00
Dmitri Hrybenko
1c0233efb1 Move lib/Parse/{Parser.h, Scope.h} -> include/swift/Parse/
Swift SVN r6062
2013-07-08 20:36:40 +00:00
Dmitri Hrybenko
c7421211a3 Factor out logic that creates and configures all objects required for
compilation into a 'Frontend' library.

This library is still not in its best shape, but now it can be reused in
swift-index-test.


Swift SVN r6040
2013-07-06 00:46:19 +00:00
Argyrios Kyrtzidis
6e7d0490f7 Allow optionally to produce comment tokens when lexing and add a tokenize() utility function.
Swift SVN r6008
2013-07-05 15:02:42 +00:00
Dmitri Hrybenko
2311e4ec16 Don't try to do delayed body parsing for a function that does not have a body.
Also actually check that we call getBodyTokenRange() only when the correct
union member is active.


Swift SVN r5965
2013-07-02 23:06:08 +00:00
Chris Lattner
20229fbf85 Introduce infrastructure for maintaining per-translation unit SIL parser state across
invocations of the parser.



Swift SVN r5906
2013-06-30 18:44:59 +00:00
Dmitri Hrybenko
f73d866d91 Implement delayed parsing for function bodies
In order to do this, we need to save and restore parser state easily.  The
important pieces of state are:

* lexer position;
* lexical scope stack.

Lexer position can be saved/restored easily.  We don't need to store the tokens
for the function body because swift does not have a preprocessor and we can
easily re-lex everything we need.  We just store the lexer state for the
beginning and the end of the body.

To save the lexical scope stack, we had to change the underlying data
structure.  Originally, the parser used the ScopedHashTable, which supports
only a stack of scopes.  But we need a *tree* of scopes.  I implemented
TreeScopedHashTable based on ScopedHashTable.  It has an optimization for
pushing/popping scopes in a stack fashion -- these scopes will not be allocated
on the heap.  While ‘detached’ scopes that we want to re-enter later, and all
their parent scopes, are moved to the heap.

In parseIntoTranslationUnit() we do a second pass over the 'structural AST'
that does not contain function bodies to actually parse them from saved token
ranges.


Swift SVN r5886
2013-06-28 22:38:10 +00:00
Joe Groff
10f136f7d3 Parse: Remove 'OtherThanAssignment' from 'parseStmt'.
Assignment isn't a Stmt anymore.

Swift SVN r5832
2013-06-27 05:13:33 +00:00
Chris Lattner
1040bfec6a In the REPL, allow access to the Builtin module if explicitly imported,
there is no reason to deny it and it could be theoretically useful.


Swift SVN r5779
2013-06-24 16:07:56 +00:00
Chris Lattner
bb82daca8f generalize parseToken to take an arbitrary argument list for the diagnostic
when it fails.

Teach the SIL parser to parse and validate (but still discard) tuple and
return instructions.


Swift SVN r5318
2013-05-25 00:40:46 +00:00
Chris Lattner
bd217c9227 extend the general form of parseIdentifier to optionally return a SourceLoc,
and use this throughout the parser.


Swift SVN r5314
2013-05-25 00:07:56 +00:00
Chris Lattner
ce314e59c1 - introduce a form of Parser::parseIdentifier that captures a token location,
requiring us to make the variadic templates a bit more specific.

- Make SIL parser tests use updated syntax (still not parsed).


Swift SVN r5312
2013-05-24 23:59:21 +00:00
Chris Lattner
efea0bc2f9 Simplify the Parser ctors to just take the TranslationUnit being parsed into, save this as an ivar.
Swift SVN r5248
2013-05-21 05:33:42 +00:00
Chris Lattner
f2e3af3cf6 Eliminate the "global" list of unresolved identifier types maintained
by TranslationUnit.  This list existed solely to allow name lookup of
an unbound IdentifierType to know its DeclContext.  Instead of indirecting
through this list, just store the DeclContext in the IdentifierType in its
uninitialized state.

This eliminates a really terrible performance fixme about scanning the list,
eliminates the management fiddling around with this list in the parser, and
is generally much cleaner.


Swift SVN r5246
2013-05-21 05:27:37 +00:00
Chris Lattner
b4eee19287 Switch the SIL parse to parse the file in a model similar to "immediate"
mode for normal .swift files.  We basically parse batches of non-sil function
decls, type check them as a batch, then process any SIL functions.  This allows
us to have mutually recursive types and other things that are fully sema'd and
that are referenced by SIL functions, without involving SIL functions too
intimately with type checking.

This does mean that SIL functions can't forward reference types, oh well.



Swift SVN r5243
2013-05-21 03:27:27 +00:00
Chris Lattner
a697922f68 fix a logic error in parseList where it could drive off the end of the file
and infinitely loop because it didn't realize that skipUntil doesn't always
consume something.

This fixes an infinite loop on this testcase:

func isSpace(c : Char) -> Bool {
  return (c == '\v' ||
          c == '\f')
}

which comes from rdar://11936003.  I'm not adding it because it only works
as the last thing in a file, and isn't likely to regress.  The diagnostics
produced are also still really really awful for this.


Swift SVN r5241
2013-05-20 22:53:22 +00:00
Doug Gregor
fab984aeae Test and improve parser error recovery for closures.
Swift SVN r5203
2013-05-17 16:39:20 +00:00
Doug Gregor
6e64ca66f0 Treat '|' as a delimiter while parsing the signature of a closure.
'|' is part of the character set for operators, but within the
signature of a closure we need to treat the first non-nested '|' as
the closing delimiter for the closure parameter list. For example,

  { |x = 1| 2 + x}

parses with the default value of '1' for x, with the body 2 + x. If
the '|' operator is needed in the default value, it can be wrapped in
parentheses:

  { |x = (1|2)| x }

Note that we have problems with both name binding and type checking
for default values in closures (<rdar://problem/13372694>), so they
aren't actually enabled. However, this allows us to parse them and
recover better in their presence.



Swift SVN r5202
2013-05-17 16:02:44 +00:00
Doug Gregor
8372f46fb4 Factor parsing of pattern-tuple-element into its own routine.
This simple refactor makes pattern-tuple-element available for re-use
in closure expressions. As a drive-by, diagnose non-final ellipses via
diagnose() rather than via assert(), the latter being considered
rather unfriendly. Also, take pains to restore AST invariants after
such an error.


Swift SVN r5163
2013-05-13 21:32:33 +00:00
Chris Lattner
5b4c31dc94 reland r4968, with a bugfix to avoid breaking the lexer measuring token lengths.
Original message:
SIL Parsing: add plumbing to know when we're parsing a .sil file
Enhance the lexer to lex "sil" as a keyword in sil mode.


Swift SVN r4988
2013-04-30 00:28:37 +00:00
Chris Lattner
aafe3bdbdc revert r4968, it apparently breaks the world. I'll recommit it when I have time to investigate.
Swift SVN r4971
2013-04-29 16:58:43 +00:00
Chris Lattner
b503206bec SIL Parsing: add plumbing to know when we're parsing a .sil file
Enhance the lexer to lex "sil" as a keyword in sil mode.


Swift SVN r4970
2013-04-29 05:42:59 +00:00
Jordan Rose
790248d8b4 Diagnostics: use builder pattern instead of streaming for ranges/fix-its.
Per Chris's feedback and suggestions on the verbose fix-it API, convert
diagnostics over to using the builder pattern instead of Clang's streaming
pattern (<<) for fix-its and ranges. Ranges are included because
otherwise it's syntactically difficult to add a fix-it after a range.

New syntax:

  diagnose(Loc, diag::warn_problem)
    .highlight(E->getRange())
    .fixItRemove(E->getLHS()->getRange())
    .fixItInsert(E->getRHS()->getLoc(), "&")
    .fixItReplace(E->getOp()->getRange(), "++");

These builder functions only exist on InFlightDiagnostic; while you can
still modify a plain Diagnostic, you have to do it with plain accessors
and a raw DiagnosticInfo::FixIt.

Swift SVN r4894
2013-04-24 23:15:53 +00:00
Jordan Rose
f7cd3a6ec6 Add fix-its for missing and spurious separators.
Swift SVN r4789
2013-04-18 00:42:59 +00:00