David Farler 330c2d96e6 Make the lexer UTF-8 RFC 3629 correct re: prefix octets ...

RFC 2279 states that, in UTF-8:
"The octet values FE and FF never appear."

RFC 3629 states that, in UTF-8:
"The octet values C0, C1, F5 to FF never appear."

Generalize the check to advance past invalid starting bytes for
a UTF-8 sequence to fix a crash in the lexer.

2016-12-05 17:21:17 -08:00

101 B

Raw Blame History

View Raw