Commit Graph

38 Commits

Author SHA1 Message Date
Egor Zhdan
c75507511c [docs] Remove duplicated paragraph in the Testing guide 2024-12-03 19:29:15 +00:00
Daniel Rodríguez Troitiño
2f62bf4e04 [docs] Add a couple of paragraphs about swift_feature in testing.md
Quickly explain the feature, its intended usage and a warning about not
longer needing `REQUIRES: asserts` anymore.`
2024-11-02 11:46:47 -07:00
Rintaro Ishizaki
1d2fd4223f [Test] Add %host_triple and %host_sdkroot substitutions
Macro tests need to build host libraries/tools. We can't use %target-*
substitutions for that.

rdar://107398734
2023-03-31 07:41:41 -07:00
Becca Royal-Gordon
04f82b424c [NFC] Emit descriptive string constant names
Modify IRGen to emit “anonymous” string constants with names of the form `@.str.<len>.<contents>` (with a special mangling for internal `\0` characters). This makes it much easier to write IRGen tests that check for the contents of strings, because matching the constant name also implies that the constant has the expected content.
2022-09-21 11:40:15 -07:00
Alexis Laferrière
d1e36b0a12 [Sema|lit] Intro lit substitution for generating the 2 swiftinterfaces
Intro the lit substitution
`%target-swift-emit-module-interfaces(Main.swiftinterface,
Main.private.swiftinterface` to generate both the public and private
swiftinterface. And use it in spi-only-import-exportability.swift.

To verify the generated interfaces I would recommend to keep two calls
to `%target-swift-typecheck-module-from-interface` as each is a
different invocation likely to fail.
2022-09-09 09:20:51 -07:00
Allan Shortlidge
d3dbe75419 ModuleInterface: Introduce the %target-swift-emit-module-interface and %target-swift-typecheck-module-from-interface lit substitutions and adopt them in some tests that involve emitting a swiftinterface. The substitutions specify the additional flags -swift-version 5 -enable-library-evolution. There are a few motivations for adding these substitutions:
- Tests that depend on emitted interfaces should generally use flags that are typical for modules that have textual interfaces (e.g. `-enable-library-evolution`).
- If a test is intended to produce a valid `swiftinterface` then it should verify that interface. This will help prevent interface printing regressions caused by compiler changes.
- Having commonly used substitutions for tests that emit interfaces makes it easy to experiment with compiler flags that might effect interface printing.

Resolves rdar://91634358
2022-04-14 15:09:36 -07:00
Alastair Houghton
cd99995d84 [Tests] Re-add round-trip type test.
This was reverted in 12fa024 because it broke the iOS simulator build.

rdar://37170485
2021-08-17 09:51:22 +01:00
Mishal Shah
f45195454f Revert "[Tests] Add a test to round-trip types through mangled names." 2021-08-16 14:03:39 -07:00
Alastair Houghton
1cc1036ec9 Merge pull request #38649 from al45tair/problem/37170485
[Tests] Add a test to round-trip types through mangled names.
2021-08-16 14:21:38 +01:00
Alastair Houghton
8e12cd0b62 [Tests] Add a test to round-trip types through mangled names.
Added a new test to the test suite that round trips Swift types through
mangled names and checks that we get the same type back that we started
with.

rdar://37170485
2021-08-16 11:22:21 +01:00
Alastair Houghton
8918a09595 [Tests] Fix RemoteMirror/interop.swift to work with ASAN enabled.
In order to work for an ASAN build, parts of this test need themselves to be
built with ASAN turned on.

rdar://81274645
2021-07-29 18:27:11 +01:00
Ben Rimmington
0e2e11d1d2 [docs] Update Testing.md (#37399) 2021-05-17 19:42:52 +01:00
Varun Gandhi
7df40080af [docs] Note LIT_FILTER in Testing.md. 2021-04-06 16:23:39 -07:00
David Zarzycki
acfdbc94df [Docs] Fix testing target suffixes
The underlying change happened almost five years ago: 457f2b901c.
2021-02-02 08:46:00 -05:00
Mishal Shah
40024718ac Update doc and links to support new main branch 2020-09-22 23:53:29 -07:00
Meghana Gupta
b34791a0a0 Update code as per Apple Style Guide
whitelist -> allowlist
blacklist -> denylist
2020-07-24 11:37:15 -07:00
Dmitri Gribenko
870ca6da4c Add instructions on how to create a 32-bit iOS simulator device
iPhone 5 device does not come preconfigured in the latest Xcode
(11.4.1).
2020-05-21 09:46:58 +02:00
Ben Rimmington
0a3d2d5070 [docs] Update Testing.md 2020-05-18 20:42:52 +01:00
Dan Zheng
2f50c859c0 [test] Add config.target_run_simple_swiftgyb_parameterized.
Support `%target-run-simple-swiftgyb(...)` taking compiler arguments.
Copies existing support for `%target-run-simple-swift(...)`.
2020-04-09 01:47:53 -07:00
ematejska
85b7b4bedf Updating Testing instructions with utils/run-test
Updating Testing instructions with utils/run-test.  I find this utility useful and others might too.
2020-03-17 16:41:04 -07:00
Owen Voorhees
f749e8551f [Docs] Add a section to Diagnostics.md on -verify mode 2019-09-30 17:14:32 -07:00
Joe Shajrawi
ad59d902b9 Set up testing with the standard libraries coming from the OS
We  default Swift tests to use the just-built libraries
See radars rdar://problem/35163663 and rdar://problem/42176864
2019-03-27 11:16:48 -07:00
Mathew Polzin
3d05edd239 Noticed while reading the Testing README that there were now five categories so I updated the word four to five. 2019-02-11 09:01:39 -08:00
Andreas Jönsson
0a3f38792a [Doc] Updated testing lit.py example to use macosx environment as iphonesimulator doesn't work out of the box 2018-12-23 00:03:42 +01:00
Mike Ash
1131507578 [Testing] Add a substitution for %target-is-simulator. 2018-10-04 12:32:50 -04:00
Jordan Rose
8c48e39e42 [test] Add --param remote_run_extra_args (#18885)
This goes with 810b240354: if we want to pass extra arguments to SSH
when using remote-run, there has to be a way to get them to remote-run
when using lit.
2018-08-22 13:19:03 -07:00
Vedant Kumar
9eec249a85 [docs] Document the swift-lldb test whitelist (#18730) 2018-08-15 16:48:03 -07:00
Jordan Rose
290bf08655 [docs] Update Testing.md for remote-run support 2018-08-14 11:17:08 -07:00
David Zarzycki
283713d61d [Testing] Formalize stress tests
Stress tests are, by definition, stressful. They intentionally burn a
lot of resources by using randomness to hopefully surface state machine
bugs. Additionally, many stress tests are multi-threaded these days and
they may attempt to use all of the available CPUs to better uncover
bugs. In isolation, this is not a problem, but the test suite as a whole
assumes that individual tests are single threaded and therefore running
multiple stress tests at once can quickly spiral out of control.

This change formalizes stress tests and then treats them like long
tests, i.e. tested via 'check-swift-all' and otherwise opt-in.

Finally, with this change, the CI build bots might need to change if
they are still only testing 'validation' instead of all of the tests.
I see three options:

1) Run all of the tests. -- There are very few long tests left these
   days, and the additional costs seems small relative to the cost of
   the whole validation test suite before this change.
2) Continue checking 'validation', now sans stress tests.
3) Check 'validation', *then* the stress tests. If the former doesn't
   pass, then there is no point in the latter, and by running the stress
   tests separately, they stand a better chance of uncovering bugs and
   not overwhelming build bot resources.
2018-03-20 21:45:28 -04:00
Graydon Hoare
2536bbb788 [docs] Mention lit.py -vv option, helpful for diagnosis. 2017-10-20 15:38:07 -07:00
Greg Parker
b7f744c302 [test] Replace CPU=i386_or_x86_64 with CPU=i386 || CPU=x86_64. (#11889) 2017-09-12 19:34:08 -07:00
Michael Gottesman
6a6efd9f4e Merge pull request #11463 from KingOfBrian/documentation/DeveloperOnboarding
Update README.md
2017-08-21 14:00:33 -07:00
Ewa Matejska
1272cd3aac Making master call itself 4.1, updating the swift 3 compatiblity mode to be 3.3 (from 3.2), adding ability to pass swift-version 5. Importer work not done yet. 2017-08-17 20:57:01 -07:00
Brian King
28de02ea8b Remove day-to-day language. 2017-08-17 20:39:43 -04:00
Brian King
49bb53a0e3 Change from SWIFT_BUILD_ROOT to SWIFT_BUILD_DIR in examples. SWIFT_BUILD_ROOT means something else in the build script. 2017-08-15 20:31:32 -04:00
Brian King
797c0939f2 Emphasize lit testing 2017-08-14 19:37:45 -04:00
Brian King
22fc53c660 Move note on writing long tests into the section on writing tests 2017-08-14 19:26:46 -04:00
Michael Gottesman
2289e5c509 [gardening] Convert Testing.rst to markdown and rename to Testing.md 2017-07-04 23:33:04 -07:00