We build CMake on all platforms (except Darwin for an unknown reason) if CMake is not preinstalled. Since CMake 4.0 regresses certain build configurations, there's currently no way to build on Darwin without installing an older CMake version 3.x manually. This can be simplified if we build a pinned version of CMake consistently on all platforms.
Addresses rdar://159939866
(cherry picked from commit 7db6c5ddaa)
This new flag makes it easy to build Swift cross-compilation toolchains, by
disabling cross-compilation of all host tools, like the Swift compiler and
various macros, building on prior pulls #38441 and #82163.
Also, add two class methods to the Testing macros product so it works with #83260.
specifically the change that makes the following possible on linux:
"* under Linux, do not cross compile LLVM when building for the host
architecture -- that will ensure that the compiler-rt build will use
the just built compiler and not the system one (which may not be
new enough for this purpose);"
Clang driver only passes arch-specific library paths as search paths
to the linker for WebAssembly targets but we were installing
libxml2.a under `<sysroot>/lib` without the multiarch triple. It led
to the linker not finding libxml2.a when importing FoundationXML
Cherry-pick of #83128, #82399, and #82878, merged as ea6ca2b5db, 0c4e56174b, and e34eb3331f respectively.
**Explanation**: Currently `test/CMakeLists.txt` can only set `SWIFT_LIT_ARGS` for all tests uniformly. This means that we can't have tests for Embedded Swift with a different set of `lit.py` arguments.
Also, create new `check-swift-embedded-wasi` target from `test/CMakeLists.txt`, tweak `lit.cfg` to support WASI Clang resource dir, exclude unsupported tests based on `CPU=wasm32` instead of `OS=wasi`.
**Scope**: Limited to Embedded Swift test suite.
**Risk**: Low, due to limited scope.
**Testing**: #82878 was incubated on `main` for 2 weeks, #82399 for 3 weeks with no disruption, #83128 merged this week, but enables all these tests on CI, which are consistently passing.
**Issue**: rdar://156585717
**Reviewer**: @bnbarham
- **Explanation**: Install static XCTest library and its Swift module files into Swift SDK for Wasm
- **Scope**: Narrow, only affects Wasm Swift SDK.
- **Original PRs**: https://github.com/swiftlang/swift/pull/83191
- **Risk**: Low, just a change to Wasm Swift SDK
- **Testing**: CI
- **Reviewers**: @MaxDesiatov
- **Explanation**: Renames the WebAssembly target triple from `wasm32-unknown-wasi` to `wasm32-unknown-wasip1` to align with the broader WebAssembly ecosystem's naming conventions. It's important to include this in 6.2, which is going to be the first stable release of Swift SDK for Wasm, so that we won't introduce a breaking triple change in the stable version series later.
The WebAssembly ecosystem has been migrating from the generic wasi target name to more explicit version-specific names like `wasip1` and `wasip2`. This change has been adopted across multiple toolchains and language ecosystems:
- Rust: Renamed `wasm32-wasi` to `wasm32-wasip1` (https://github.com/rust-lang/compiler-team/issues/607)
- wasi-sdk: Renamed `wasm32-wasi` to `wasm32-wasip1` (https://github.com/WebAssembly/wasi-sdk/pull/386)
- Go: Already uses `wasip1` naming convention (https://github.com/golang/go/issues/58141)
The "p1" suffix explicitly indicates WASI Preview 1, which helps distinguish it from the newer WASI Preview 2 (wasip2) and provides clarity about which version of the WASI specification is being targeted.
Also we already use `wasm32-unknown-wasip1` for Embedded WASI target, and having different names for Embedded and non-Embedded is confusing users.
- **Scope**: Narrow, only affects Wasm Swift SDK.
- **Original PRs**: https://github.com/swiftlang/swift/pull/83167
- **Risk**: Low, just a change to Wasm Swift SDK
- **Testing**: CI.
- **Reviewers**: @MaxDesiatov
Cherry-pick of #82285, merged as 05204bdf63
**Explanation**: Include Swift Testing Foundation and libxml2 into the Swift SDK for WebAssembly
**Scope**: Limited to Swift SDK for Wasm
**Risk**: Low, due to limited scope. Changes were incubated on `main` for 2 weeks with no known issues. Swift Testing changes were incubated for more than a month.
**Testing**: Manual testing with Swift SDKs produced off `main`.
**Issue**: rdar://155865189
**Reviewer**: @MaxDesiatov
---------
Co-authored-by: Yuta Saito <kateinoigakukun@gmail.com>
We should eventually split these, but for now SWIFT_COMPILER_VERSION
matches SWIFT_TOOLCHAIN_VERSION when it's actually set. Just use it in
preference to SWIFT_TOOLCHAIN_VERSION.
(cherry picked from commit ec46a4f026)
I just noticed the install issue when cross-compiling Testing with a
freshly-built compiler, which I'd never done before. Also, fix the NDK
path shown in the CMake output.
- **Explanation**: Build wasmkit, a new toolchain executable, as universal binary on macOS using SwiftPM --arch flags. (Cherry picked from commit 6530236)
- **Scope**: WasmKit is new, and users targeting Wasm (and preferring WasmKit over other Wasm runtimes) might be few. WasmKit can be invoked directly by users and indirectly via `swift run` when using the official Swift SDKs for Wasm.
- **Issues**: Resolves#82390
- **Original PRs**: #82393
- **Risk**: The new configuration only affects macOS targets. The binary is output to a new location, but the code already introspects the location correctly. The bug and fix impacts users of WasmKit binary newly added to toolchain, directly and via `swift run`. Risk of no-fix: The Swift.org wasm "Getting Started" [1] refers explicitly to this (unreleased) 6.2 toolchain WasmKit binary. Without this fix, users on Apple Silicon following those instructions get an obscure "bad CPU" error because it is built only for the host x86_64 arch.
- **Testing**: @kateinoigakukun verified WasmKit is universal after change in `main`. To verify, run shell `file usr/bin/wasmkit` and verify outputs include x86_64 and arm64 arch. An outstanding question is whether to add a check that toolchain executables are universal binaries on macOS. Currently they are, except for `docc`.
- **Reviewers**: @kateinoigakukun
[1] https://www.swift.org/documentation/articles/wasm-getting-started.html
Passes the value of the environment variable `TOOLCHAIN_VERSION` through
to Swift's `SWIFT_TOOLCHAIN_VERSION` for use in eg.
`-print-target-info` (and some day, `--version`).
(cherry picked from commit 3982ebbbb3)
**Description**: This adds "task name" parameter to all task creating
functions.
This is done in a few ways, e.g. we can backdeploy this to 5.1 in APIs
which do not accept the `TaskExecutor` but it they do we provide a
version for 6.0+ etc. This was requested in the SE acceptable of this
proposal [Acceptance post
SE-0469](https://forums.swift.org/t/accepted-with-modifications-se-0469-task-naming/79438).
This moves all these declarations to gyb since going through them one by
one has become unmaintainable otherwise.
**Scope/Impact**: All task creation APIs now gain a new task name
parameter.
**Risk:** Medium, changes existing APIs rather than adding "even more
overloads" though this risk was discussed in the team and accepted. This
has a potential to be source breaking it someone used Task.init and
friends as function.
**Testing**: CI testing, source compatibility suite testing
**Reviewed by**:
**Original PR:**
- https://github.com/swiftlang/swift/pull/81107 build changes required
for this
- https://github.com/swiftlang/swift/pull/80984
**Radar:**
---------
Co-authored-by: Kuba Mracek <mracek@apple.com>
This tells build-script to build Swift-testing with WMO.
This results in a faster build products, but is also necessary for
configurations using the legacy swift driver, which would emit an
invalid swift interface in non-WMO builds.
Fixes: rdar://151357567
(cherry picked from commit 5f2b0022d1)
WASI with Embedded Swift provides WASI-libc and libc++ headers necessary to build the `_Concurrency` module for Wasm. We now add `wasm32-unknown-wasip1-wasm` triple to `EMBEDDED_STDLIB_TARGET_TRIPLES` when `SWIFT_WASI_SYSROOT_PATH` is set, which builds the necessary stdlib slice.
---------
Co-authored-by: Yuta Saito <kateinoigakukun@gmail.com>
If the directory where the build time log is supposed to go doesn't
exist, create it. The append file mode will create files, but won't
create directories. When we start building ninja, we haven't necessary
created the build directory yet, so this results in an error about the
missing directory when writing the build time log.
The ninja builder took a host argument that was unused by the function.
The ninja build failed to pass this argument, resulting in
an execution failure. Removing the argument.
Instead of using `--build-ninja` to decide to build ninja, build it
automatically if a sufficiently new enough version is not available.
Also record the build time taken to build the local Ninja so that we can
see how much time we would save by stashing a pre-built Ninja in CI.
Ninja builds its tests by default.
We don't run the Ninja test suite, we aren't doing development on Ninja,
and we are using a release tag that has been verified to work. There
isn't much point in building the tests if we're not going to use them.
Disabling building the Ninja tests. If it is desirable to build them,
one can set `BUILD_TESTING` to `YES` and re-run their build.
This patch switches the Ninja build from using the configure.py script
to building with the just-built CMake.
The configure.py in Ninja 1.11.1 still uses Python 2.7, importing the
`pipes` module. The pipes module was deprecated in Python 3.11 and
removed in 3.13, so folks using newer versions of Python are running
into issues with this.
The CMake build doesn't have this issue and is also perfectly valid, so
we can switch to that.
This patch updates the CMake-building mechanism to avoid
re-bootstrapping CMake if we already bootstrapped one that is new
enough.
I've made it so that all paths through the function return the path to a
CMake so we can use the result of the function as the cmake path without
having to check.
The function will choose one of the following ways of getting CMake in
order of preference:
- One we already built
- The system CMake
- Bootstrapping one from scratch
It prefers one we built over checking the system CMake because, if we
have already built a CMake previously, it's a good indication that
there either was no system CMake installed, or it wasn't new enough. We
shouldn't waste time checking it again if a previous run detected that
it wasn't good enough.
The system CMake is preferable to building one from scratch if we don't
need to though, so we determine if the system CMake is sufficient.
Finally, if one that we built either doesn't exist, or isn't new enough,
and the system either doesn't have a CMake, or a new enough CMake, build
one. It is built into the location that we are checking for caching, so
the next time we run build-script, it should hit the first case and
choose the already-built CMake instead of building it again.
Include the CMake bootstrap time in the build-script build times.
We're including everything else. Would be good to determine how much
time we can save by caching a new enough pre-built CMake in the builder
images.
Importing the log_time_in_scope exposes a cyclic dependency cycle
between the `swift_build_support` and `build_swift` python modules in
such a way that the tests fail due to re-importing parts of build_swift:
```
ImportError: Failed to import test module: tests.build_swift.test_migration
Traceback (most recent call last):
File "/usr/lib/python3.8/unittest/loader.py", line 436, in _find_test_path
module = self._get_module_from_name(name)
File "/usr/lib/python3.8/unittest/loader.py", line 377, in _get_module_from_name
__import__(name)
File "/home/build-user/swift/utils/build_swift/tests/build_swift/test_migration.py", line 14, in <module>
from build_swift import migration
File "/home/build-user/swift/utils/build_swift/build_swift/migration.py", line 18, in <module>
from swift_build_support.swift_build_support.targets import \
File "/home/build-user/swift/utils/swift_build_support/swift_build_support/targets.py", line 15, in <module>
from . import cmake
File "/home/build-user/swift/utils/swift_build_support/swift_build_support/cmake.py", line 26, in <module>
from swift_build_support.swift_build_support.utils import log_time_in_scope
File "/home/build-user/swift/utils/swift_build_support/swift_build_support/utils.py", line 20, in <module>
from build_swift.build_swift.constants import SWIFT_BUILD_ROOT
ModuleNotFoundError: No module named 'build_swift.build_swift'
```
I've put the import of log_time_in_scope into the function definition
to ensure that build_swift has been fully loaded by the time we need
log_time_in_scope, ensuring that there is order between the two pieces.
Python caches the imported module, so if we accidentally re-import the
log_time_in_scope, nothing actually changes.
This re-orders the instantiation of the BuildScriptInvocation object so
that it comes before the creation of the CMake path. This ensures that
BuildScriptInvocation() does not delete the build log after logging the
CMake bootstrap time. This is fine because the toolchain and arguments
are reference types, so updating the CMake path in both of those will be
reflected in the copy taken in the BuildScriptInvocation() object.
What is nice about this is that by not using extra-cmake-args, we can avoid
passing this into LLVM as well when attempting to reproduce failures on the bots
(thus avoiding having to rebuild LLVM as well).
I have been doing this using extra-cmake-args/etc... just feels better to have
an actual option to do this.
Just did this quickly while waiting for my Linux build to finish that uses
extra-cmake-args to set the linker.
