Fixes:
* multiple statements on one line (colon) (E701)
* missing whitespace around arithmetic operator (E226)
* missing whitespace around operator (E225)
* closing bracket does not match visual indentation (E124)
* blank line contains whitespace (W293)
* continuation line missing indentation or outdented (E122)
* continuation line over-indented for hanging indent (E126)
* missing expected blank line (E301)
* trailing whitespace (W291)
* unexpected spaces around keyword / parameter equals (E251)
* whitespace after '(', '[' or '{' (E201)
* whitespace before ')', ']' or '}' (E202)
* whitespace before ',' or ':' (E203)
Python 3 on Linux reads the system locale information to determine what
it should use as the default encoding for strings read from files (this
is different from OS X which is always UTF-8 by default [1]).
Since all the Swift gyb templates are UTF-8 encoded there is effectively
no reason to parse them as anything else. This patch forces the gyb
template parser to read the template using UTF-8 encoding. It accounts
for both reading and writing to a file as well as reading from stdin and
writing to stdout.
Two changes of note are that it now includes a __future__ import that
should make Python 2 behave a little closer to Python 3 in terms of
unicode support. Additionally Python 2 can no longer use cStringIO
because it does not support unicode [2].
To test this patch I ran these commands before and after the patch.
Note: that before the patch if the locale was set to something other
than UTF-8, ASCII for instance, the Python 3 runs would fail. See [3]
for example failure message.
Without stdin/stdout:
$ python2 utils/gyb -o Arrays.2.7.swift stdlib/public/core/Arrays.swift.gyb
$ python3 utils/gyb -o Arrays.3.5.swift stdlib/public/core/Arrays.swift.gyb
$ diff -u Arrays.2.7.swift Arrays.3.5.swift
With stdin/stdout:
$ cat stdlib/public/core/Arrays.swift.gyb | python2 utils/gyb > Arrays.2.7.stdin.stdout.swift
$ cat stdlib/public/core/Arrays.swift.gyb | python3 utils/gyb > Arrays.3.5.stdin.stdout.swift
$ diff -u Arrays.2.7.stdin.stdout.swift Arrays.3.5.stdin.stdout.swift
[1] https://docs.python.org/3/howto/unicode.html#unicode-filenames
[2] https://docs.python.org/2/library/stringio.html#cStringIO.StringIO
[3] https://lists.swift.org/pipermail/swift-dev/Week-of-Mon-20160111/000780.html
In Python 2 the syntax to catch exceptions was:
except (Exception1, Exception2), target:
In Python 3 the syntax to catch exceptions is:
except (Exception1, Exception2) as target:
This newer Python 3 syntax is also available in 2.6 and 2.7. Therefore,
it is preferred for compatibility reasons. Additionally, the target no
longer can be a tuple. This patch refactors the exception handeling code
to be the newer Python 3 exception catch syntax.
The `StringIO` and `cStringIO` modules are gone in Python 3. The
recommendation is to import the `io` module on Python 3. Therefore, this
patch first attempts to import the Python 2 module, `cStringIO`, and if
that fails then attempts to import the Python 3 module, `io`.
**NOTE**: There are still other Python 3.x fixes necessary to make `gyb`
run on a Python 3.x interpreter. This is just one small incremental
patch on the way there.
[PEP 3114](https://www.python.org/dev/peps/pep-3114/) renamed
`iterator.next()` (Python 2) to `iterator.__next__()` (Python 3). The
recommended solution to make the code work in both Python 2 and 3 is to
call the global `next` function.
To use this recommended global `next` function this patch uses a lambda
function to supply `tokenize.generate_tokens` a callable function as it was previously.
This should be functionally equivalent to the old code with the added
benefit of working on both Python 2 and 3.
Rather than using `f = open(path).read()`, which leaves the file open
for an indeterminate period of time, switch to the `with open(path) as f`
idiom, which ensures the file is always closed correctly.
In Python 3, 'print' was changed from a statement to a function. Using
the __future__ module allows scripts to use print function whether
running with Python 2.6+ or Python 3.x. This commit changes as many
instances of print as I could find to use the print function and the
__future__ module.
This allows Python or .gyb files to do "import gyb" and use the basic
components of the system. For example, we could create a Python
function to generate the boilerplate for a Mirror. That function could
use gyb.parseTemplate() to load an inline template and
gyb.executeTemplate() to generate the code. Then calls to that function
can be embedded in ${...} in a .gyb file.
Swift SVN r19490
