Xi Ge 34e2aec662 Parser: use parser to generate a refined token stream to help syntax coloring. (#11809) ...

This patch allows Parser to generate a refined token stream to satisfy tooling's need. For syntax coloring, token stream from lexer is insufficient because (1) we have contextual keywords like get and set; (2) we may allow keywords to be used as argument labels and names; and (3) we need to split tokens like "==<". In this patch, these refinements are directly fulfilled through parsing without additional heuristics. The refined token vector is optionally saved in SourceFile instance.

2017-09-08 10:28:19 -07:00

2.2 KiB

Raw Blame History

View Raw