The `__future__` we relied on is now, where the 3 specific things are
all included [since Python 3.0](https://docs.python.org/3/library/__future__.html):
* absolute_import
* print_function
* unicode_literals
* division
These import statements are no-ops and are no longer necessary.
The way we already gather numbers for this test is that we run two runs of
`Benchmark_O $TEST` with num-samples=2, iters={2,3}. Under the assumption that
the only difference in counter numbers can be caused by that extra iteration,
subtracting the group of counts for 2,3 gives us the number of counts in that
iteration.
In certain cases, I have found that a small subset of the benchmarks are
producing weird output and I haven't had the time to look into why. That being
said, I do know what these weird results look like, so in this commit we do some
extra validation work to see if we need to fail a test due to instability.
The specific validation is that:
1. We perform another run with num-samples=2, iter=5 and subtract the iter=3
counts from that. Under the assumption that overall work should increase
linearly with iteration size in our benchmarks, we check if the counts are
actual 2x.
2. If either `result[iter=3] - result[iter=2]` or `result[iter=5] -
result[iter=3]` is negative. All of the counters we gather should never decrease
with iteration count.
Otherwise, one can get results that seem to imply more rr traffic when in
reality, one was not tracking {retain,release}_n that as a result of better
optimization become just simple retain, release.
Replace the project global linting rule excludes (as defined in .pep8) with
fine-grained "# noqa" annotations.
By using noqa annotation the excludes are made on a per line basis instead of
globally.
These annotations are used where we make deliberate deviations from the standard
linting rules.
To lint the Python code in the project:
$ flake8
To install flake8:
$ pip install flake8
See https://flake8.readthedocs.org/en/latest/ for details.
To enable checking of the PEP-8 naming conventions, install the optional
extension pep8-naming:
$ pip install pep8-naming
To enable checking of blind "except:" statements, install the optional
extension flake8-blind-except:
$ pip install flake8-blind-except
To enable checking of import statement order, install the optional
extension flake8-import-order:
$ pip install flake8-import-order
* E101: indentation contains mixed spaces and tabs
* E111: indentation is not a multiple of four
* E128: continuation line under-indented for visual indent
* E302: expected 2 blank lines, found 1
* W191: indentation contains tabs
