[benchmark] Strangler run_benchmarks

Replaced guts of the `run_benchmarks` function with implementation from `BenchmarDriver`. There was only single client which called it with `verbose=True`, so this parameter could be safely removed.

Function `instrument_test` is replaced by running the `Benchmark_0` with `--memory` option, which implements the MAX_RSS measurement while also excluding the overhead from the benchmarking infrastructure. The incorrect computation of standard deviation was simply dropped for measurements of more than one independent sample. Bogus aggregated `Totals` statistics were removed, now reporting only the total number of executed benchmarks.
This commit is contained in:
Pavol Vaskovic
2018-08-17 00:08:21 +02:00
parent a84db83062
commit 076415f969
4 changed files with 73 additions and 69 deletions

View File

@@ -181,6 +181,7 @@ class TestPerformanceTestResult(unittest.TestCase):
def test_init(self):
log_line = '1,AngryPhonebook,20,10664,12933,11035,576,10884'
r = PerformanceTestResult(log_line.split(','))
self.assertEquals(r.test_num, '1')
self.assertEquals(r.name, 'AngryPhonebook')
self.assertEquals(
(r.num_samples, r.min, r.max, r.mean, r.sd, r.median),
@@ -346,7 +347,7 @@ Totals,269
log = ("""
# TEST SAMPLES MIN(μs) MAX(μs) MEAN(μs) SD(μs) MEDIAN(μs) MAX_RSS(B)
3 Array2D 20 2060 2188 2099 0 2099 20915200
Totals 281 2693794 2882846 2748843 0 0 0""")
Totals 281""")
parser = LogParser()
results = parser.parse_results(log.splitlines()[1:]) # without 1st \n
self.assertTrue(isinstance(results[0], PerformanceTestResult))