averello/swift-mirror
1
0
Fork 0
mirror of https://github.com/apple/swift.git synced 2025-12-14 20:36:38 +01:00
Code Issues Packages Projects Releases Wiki Activity
184,973 Commits 622 Branches 2,942 Tags
maxd/async-build-script
Commit Graph

28 Commits

Author SHA1 Message Date
Erik Eckstein
2960f472a7 fix the swift library code size comparison in the run_smoke_bench script 
In some configurations the script mixed up the build architectures and accidentally reported the code size difference between the x86 and arm.
 2024-08-05 11:03:24 +02:00
Tim Kientzle
dfe8284462 pylint fixes 2022-11-07 14:45:59 -08:00
Tim Kientzle
b0ce365b53 A better way to adapt to -num-samples 2022-11-05 16:05:26 -07:00
Tim Kientzle
c3a727486f Make --num-samples actually work 2022-11-05 14:18:41 -07:00
Tim Kientzle
5c14017bba For size comparisons, build the result objects directly with sample data 2022-11-05 14:17:34 -07:00
Tim Kientzle
e1ab70a4b0 Use results consistently 2022-11-05 13:29:52 -07:00
Josh Soref
fa3ff899a9 Spelling benchmark (#42457) 
* spelling: approximate

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: available

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: benchmarks

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: between

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: calculation

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: characterization

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: coefficient

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: computation

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: deterministic

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: divisor

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: encounter

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: expected

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: fibonacci

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: fulfill

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: implements

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: into

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: intrinsic

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: markdown

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: measure

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: occurrences

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: omitted

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: partition

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: performance

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: practice

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: preemptive

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: repeated

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: requirements

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: requires

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: response

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: supports

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: unknown

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: utilities

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: verbose

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

Co-authored-by: Josh Soref <jsoref@users.noreply.github.com>
 2022-04-25 09:02:06 -07:00
Erik Eckstein
fb65284995 benchmarks: fix run_smoke_bench after upgrading to python3 
Need to decode result of `subprocess.check_output`
 2022-04-19 13:59:55 +02:00
Daniel Duan
3dfc40898c [NFC] Remove Python 2 imports from __future__ (#42086) 
The `__future__` we relied on is now,  where the 3 specific things are
all included [since Python 3.0](https://docs.python.org/3/library/__future__.html):

* absolute_import
* print_function
* unicode_literals
* division

These import statements are no-ops and are no longer necessary.
 2022-04-13 14:01:30 -07:00
Mishal Shah
ddabee30e2 Add arch info to benchmark report 2021-06-01 09:59:20 -07:00
Erik Eckstein
abcae7bfa1 benchmarks: fix smoke test run by setting the dynamic library path 
This is a workaround for rdar://78584073
 2021-05-31 15:10:08 +02:00
Erik Eckstein
a46cda8c51 benchmarks: fix run_smoke_bench to support new benchmark executable naming scheme 
Find the right benchmark executable with a glob pattern.
Also, add an option "-arch" to select between executables for different architectures.
 2020-07-07 11:01:49 +02:00
Meghana Gupta
911ac8e45e Fix code size reporting when input directory is missing a trailing '/' 
run_smoke_bench script fails to report code size changes if you have a
trailing '/' in <old_build_dir> but not <new_build_dir>.

This change appends a separator if it is missing
 2020-05-14 13:49:35 -07:00
Ross Bayer
b1961745e0 [Python: black] Reformatted the benchmark Python sources using utils/python_format.py. 2020-02-08 15:32:44 -08:00
Erik Eckstein
81a5c0f479 run_smoke_bench: make num_retries configurable 2019-10-14 11:37:42 +02:00
Pavol Vaskovic
73b31006ee [benchmark] Fix help printing for run_smoke_bench 2019-05-23 21:40:44 +02:00
Pavol Vaskovic
3f179f39e0 Increase # of independent samples for changes. 
Multimodal benchmarks with significant delta between the modes can report false performance changes when we gather too few independent samples. This increases the minimal number of independent samples from 5 to 10.
Fix for https://bugs.swift.org/browse/SR-9907
 2019-02-12 11:42:51 +01:00
Pavol Vaskovic
85ba83191e [benchmark] Remove unused function get_results 
Remove the `get_results` function, which is no longer used after the refactoring that rebased the benchmark measurements on `BenchmarDriver` class in #21684.
 2019-02-04 10:11:57 +01:00
Pavol Vaskovic
d854f0f898 [benchmark] test_performance with BenchmarkDriver 
Refactored `test_perfomance` function to use existing  BenchmarkDriver and TestComparator.

This replaces hand-rolled parser and comparison logic with library functions which already have full unit test coverage.
 2019-01-08 00:22:00 +01:00
Pavol Vaskovic
cd4886aa2b [Gardening] Move imports and DriverArgs to top 2019-01-07 20:59:47 +01:00
Pavol Vaskovic
d6392cc014 Merge pull request #20807 from palimondo/and-dreadfully-distinct 
[benchmark] Added Benchmark Check Report
 2018-11-29 05:35:18 +01:00
Pavol Vaskovic
cd47f32350 [benchmark] Don’t emit report for no added tests 2018-11-28 23:57:23 +01:00
Pavol Vaskovic
94e86c043e [benchmark] Call test_opt_levels directly 
Simplify the code by removing indirection:
After removal of -check-added option, only single function remains.
 2018-11-28 22:47:02 +01:00
Pavol Vaskovic
98d6f22a7d [benchmark] run_smoke_bench -skip-check-added 
Removed the option to `-check-added`, as it’s now run by default. Replaced with option to skip checking added benchmarks: `-skip-check-added`.
 2018-11-28 21:47:57 +01:00
Pavol Vaskovic
cbd882119c [benchmark] Added Benchmark Check Report 
Produce Markdown formatted report, analyzing the quality of newly added benchmarks.
 2018-11-27 23:01:39 +01:00
Pavol Vaskovic
4e5f51d2f6 [benchmark] run_smoke tests with new naming 
Allow for running of test matching the naming convention proposed in #20334.
 2018-11-17 22:35:58 +01:00
Pavol Vaskovic
af719248e6 [benchmark] run_smoke_bench deterministic hashing 
Tests that used hashing were being unnecessarily tested multiple times, because this environment variable was missing.
 2018-11-17 21:38:46 +01:00
Erik Eckstein
040aa06fec benchmarks: combine everything which is needed into run_smoke_bench 
Now, run_smoke_bench runs the benchmarks, compares performance and code size and reports the results - on stdout and as a markdown file.
No need to run bench_code_size.py and compare_perf_tests.py separately.

This has two benefits:
- It's much easier to run it locally
- It's now more transparent what's happening in '@swiftci benchmark', because now all the logic is in run_smoke_bench rather than in the not visible script on the CI bot.

I also remove the branch-arguments from ReportFormatter in ompare_perf_tests.py. They were not used anyway.

For a smooth rollout in CI, I created a new script rather than changing the existing one. Once everything is setup in CI, I'll delete the old run_smoke_test.py and bench_code_size.py.
 2018-11-01 16:41:39 -07:00