Because utils/gyb is just a launcher, `doctest.testmod()` without
explicit module doesn't perform doctest of `gyb` module.
It used to perform doctest of `util/gyb`, the `__main__` module.
From the Python documentation:
* NotImplemented: "Special value which can be returned by the 'rich comparison'
special methods (__eq__(), __lt__(), and friends), to indicate that the
comparison is not implemented with respect to the other type."
* NotImplementedError: "This exception is derived from RuntimeError. In user
defined base classes, abstract methods should raise this exception when they
require derived classes to override the method."
* E101: indentation contains mixed spaces and tabs
* E111: indentation is not a multiple of four
* E128: continuation line under-indented for visual indent
* E302: expected 2 blank lines, found 1
* W191: indentation contains tabs
The repo contains roughly 80 Python scripts. "snake_case" naming is used for
local variables in all those scripts. This is the form recommended by the PEP 8
naming recommendations (Python Software Foundation) and typically associated
with idiomatic Python code.
However, in nine of the 80 scripts there were at least one instance of
"camelCase" naming prior to this commit.
This commit improves consistency in the Python code base by making sure that
these nine remaining files follow the variable naming convention used for
Python code in the project.
References:
* PEP 8: https://www.python.org/dev/peps/pep-0008/
* pep8-naming: https://pypi.python.org/pypi/pep8-naming
Fixes:
* multiple statements on one line (colon) (E701)
* missing whitespace around arithmetic operator (E226)
* missing whitespace around operator (E225)
* closing bracket does not match visual indentation (E124)
* blank line contains whitespace (W293)
* continuation line missing indentation or outdented (E122)
* continuation line over-indented for hanging indent (E126)
* missing expected blank line (E301)
* trailing whitespace (W291)
* unexpected spaces around keyword / parameter equals (E251)
* whitespace after '(', '[' or '{' (E201)
* whitespace before ')', ']' or '}' (E202)
* whitespace before ',' or ':' (E203)
Python 3 on Linux reads the system locale information to determine what
it should use as the default encoding for strings read from files (this
is different from OS X which is always UTF-8 by default [1]).
Since all the Swift gyb templates are UTF-8 encoded there is effectively
no reason to parse them as anything else. This patch forces the gyb
template parser to read the template using UTF-8 encoding. It accounts
for both reading and writing to a file as well as reading from stdin and
writing to stdout.
Two changes of note are that it now includes a __future__ import that
should make Python 2 behave a little closer to Python 3 in terms of
unicode support. Additionally Python 2 can no longer use cStringIO
because it does not support unicode [2].
To test this patch I ran these commands before and after the patch.
Note: that before the patch if the locale was set to something other
than UTF-8, ASCII for instance, the Python 3 runs would fail. See [3]
for example failure message.
Without stdin/stdout:
$ python2 utils/gyb -o Arrays.2.7.swift stdlib/public/core/Arrays.swift.gyb
$ python3 utils/gyb -o Arrays.3.5.swift stdlib/public/core/Arrays.swift.gyb
$ diff -u Arrays.2.7.swift Arrays.3.5.swift
With stdin/stdout:
$ cat stdlib/public/core/Arrays.swift.gyb | python2 utils/gyb > Arrays.2.7.stdin.stdout.swift
$ cat stdlib/public/core/Arrays.swift.gyb | python3 utils/gyb > Arrays.3.5.stdin.stdout.swift
$ diff -u Arrays.2.7.stdin.stdout.swift Arrays.3.5.stdin.stdout.swift
[1] https://docs.python.org/3/howto/unicode.html#unicode-filenames
[2] https://docs.python.org/2/library/stringio.html#cStringIO.StringIO
[3] https://lists.swift.org/pipermail/swift-dev/Week-of-Mon-20160111/000780.html
In Python 2 the syntax to catch exceptions was:
except (Exception1, Exception2), target:
In Python 3 the syntax to catch exceptions is:
except (Exception1, Exception2) as target:
This newer Python 3 syntax is also available in 2.6 and 2.7. Therefore,
it is preferred for compatibility reasons. Additionally, the target no
longer can be a tuple. This patch refactors the exception handeling code
to be the newer Python 3 exception catch syntax.
The `StringIO` and `cStringIO` modules are gone in Python 3. The
recommendation is to import the `io` module on Python 3. Therefore, this
patch first attempts to import the Python 2 module, `cStringIO`, and if
that fails then attempts to import the Python 3 module, `io`.
**NOTE**: There are still other Python 3.x fixes necessary to make `gyb`
run on a Python 3.x interpreter. This is just one small incremental
patch on the way there.
[PEP 3114](https://www.python.org/dev/peps/pep-3114/) renamed
`iterator.next()` (Python 2) to `iterator.__next__()` (Python 3). The
recommended solution to make the code work in both Python 2 and 3 is to
call the global `next` function.
To use this recommended global `next` function this patch uses a lambda
function to supply `tokenize.generate_tokens` a callable function as it was previously.
This should be functionally equivalent to the old code with the added
benefit of working on both Python 2 and 3.
Rather than using `f = open(path).read()`, which leaves the file open
for an indeterminate period of time, switch to the `with open(path) as f`
idiom, which ensures the file is always closed correctly.
