This patch changes the syntax map data structure it uses to be offset based
rather than line/col based in order to avoid calling getLineAndColumn for the
start and end offset of every token. This removes the 30% of time spent in
getLineAndColumn for this request in large files (rdar://problem/28965123).
The logic for returning the affected range and the token ranges to highlight
following an edit also made several assumptions that no longer hold. This
patch changes it to compare the syntax maps from before and after the edit,
find the first mismtaching tokens from the start and end of the syntax maps
and return the tokens in that range (adjusted to line boundaries). This fixes
syntax highlighting issues with interpolated multi-line strings
(rdar://problem/32148117) and block comments.
With the above changes the per-keystroke time spent for syntax highlighting
(with sematic info disabled) dropped from ~80ms to just under 50ms for a
12KLOC file.
This adds some heuristics so we can emit a fixit to remove extraneous
whitespace after a . and diagnose the case where a member just hasn't
been written yet better. This also improves handling of tok::unknown
throughout the parser a bit.
This is a re-commit of ff4ea54 with an update for a SourceKit test.
The code goes into its own sub-tree under 'tools' but tests go under 'test',
so that running 'check-swift' will also run all the SourceKit tests.
SourceKit is disabled on non-darwin platforms.
