It was originally designed for faster trasmission of syntax trees from
C++ to SwiftSyntax, but superceded by the CLibParseActions. There's no
deserializer for it anymore, so let's just remove it.
The argument handling for `subprocess.check_output` differs between
Python 2.7, [3.0, 3.6], [3.7).
With Python 2.7, the list is implicitly decoded and we cannot use the
`encode` method as it will attempt to decode it as ASCII which will fail
with unicode data.
Python [3.0, 3.6] will attempt to encode the arguments before executing,
but will do so based on the system locale. In the case that the locale
is `POSIX`, the encoding will be attempted in ASCII, which again fails
when unicode data is used.
Python [3.7) will accept the encoded strings and pass them through. It
will implicitly encode the arguments using the file system encoding
(normally UTF-8), which will allow it to actually work with the content
encoded or decoded.
In order to provide maximal compatibility, pre-encode the arguments as
UTF-8 when using python 3. Rely on the interpreter encoding the
argument implicitly on Python 2. This allows running the
incrParse.simple test with python 2.7, python 3.6, python 3.7, python
3.8 with `LC_ALL=POSIX` as is the default on Ubuntu Server.
Thanks to @tbkka for the details on how to reproduce this issue and the
help with the python conversion!
The last set of changes to make it backwards compatible with Python 2
required converting the arguments. That is not compatible on Python 3
unfortunately. Only perform that on Python 2 to make the utility
compatible with 2 and 3.
This uses `io.open` to allow `test_util` to open with the encoding and
newline handling across python 2 and python 3. With this change, the
test suite failures are within the single digits locally.
Use `io.open` to open the files with the correct encoding and to allow
proper newline specification. This allows the script to be python 2 and
python 3 compatible.
Enhances `swift-syntax-test` to output the parser diagnostics so we can verify that
if the incremental parse resulted in parser diagnostics, those diagnostics were also emitted during the full parse.
Add a shared library with a C API that provides access to the syntactic parser with callbacks for the inference of raw syntax nodes.
This is primarily intended to be used by SwiftSyntax to speed-up source code parsing for it.
Use the python library `difflib` which is part of a standard python
installation to generate the diff. This allows us to run the tool on
Windows wihtout needing external tools installed.
