The 64-bit ARM architecture spelling on FreeBSD is aarch64, not arm64.
This results in a build failure about a missing
`freebsd/arm64/swiftrt.o` while building the runtimes.
This tells build-script to build Swift-testing with WMO.
This results in a faster build products, but is also necessary for
configurations using the legacy swift driver, which would emit an
invalid swift interface in non-WMO builds.
Fixes: rdar://151357567
...we disabled in #81354
This requires a couple of supporting changes
* under Linux, do not cross compile LLVM when building for the host
architecture -- that will ensure that the compiler-rt build will use
the just built compiler and not the system one (which may not be
new enough for this purpose);
* provide sanitizer flags depending on the linker the just built compiler
will use -- this detection is brittle, so print a message advising the
user how to override this.
Addresses rdar://150849329
This enables `swift run` and `swift test` to use WasmKit when cross-compiling to Wasm with Swift SDKs that have toolsets pointing to WasmKit.
rdar://150382758
Use it in the linux CI presets to set them to Debug mode and speed up
the linux CI, plus add a new preset which keeps building them in Release
mode.
I was looking at a passing linux CI run and saw the log timings at the
end: it takes [longer to build and test the swift-foundation repos than
to compile all 7k+ mostly C++ files in
LLVM](https://ci.swift.org/job/swift-PR-Linux/18996/console)!
```
--- Build Script Analyzer ---
Build Script Log: /home/build-user/build/.build_script_log
Build Percentage Build Duration (sec) Build Phase
================ ==================== ===========
9.2% 1132.94 Running tests for foundationtests
9.1% 1120.57 linux-x86_64-swift-build
9.0% 1104.2 Building llvm
7.2% 878.84 Running tests for swiftfoundationtests
6.5% 796.81 Running tests for swiftpm
5.6% 684.7 Building swiftpm
5.5% 667.92 linux-x86_64-swift-test
4.9% 597.64
```
Looking at the log, building swift-foundation in release mode takes a
long time, so let's see if changing it to debug mode helps. Some
background - the Foundation repos are built twice on the linux CI: once
by CMake, which is the version installed in the toolchain, then a second
time by SwiftPM purely for testing.
This pull only affects that second SwiftPM build for testing, which is
not shipped in the final toolchain but thrown away.
We build CMake on all platforms (except Darwin for an unknown reason) if CMake is not preinstalled. Since CMake 4.0 regresses certain build configurations, there's currently no way to build on Darwin without installing an older CMake version 3.x manually. This can be simplified if we build a pinned version of CMake consistently on all platforms.
The existing swift-cmake-options flag overwrites all flags computed by
build-script. Sometimes it is useful to be able to append additional
CMake flags without overwriting the existing flags.
This patch adds `--extra-swift-cmake-options` that adds the specified
flags to the Swift CMake configuration instead of overwriting them.
This also adds a similar `--extra-llvm-cmake-options`, which adds the
new flags to the end, allowing one to replace and overwrite CMake flags
that build-script computed.
Due to the parameter passing mechanisms in build-script-impl, while this
behavior would be useful for Swift, it is not immediately apparent how
one would best implement this at this time.
LLVM-21 plans to remove the legacy method for building compiler-rt
in the same invocation as LLVM using `LLVM_ENABLED_PROJECTS` and
`LLVM_BUILD_EXTERNAL_COMPILER_RT`.
Support the new way of building compiler-rt with a new build-script
opt-in flag `--llvm-build-compiler-rt-with-use-runtimes` --
this will allow a staged introduction, and will ensure we can revert
back to the old behaviour temporarily in case of unforeseen regression.
Since this flag is meant to be short lived, in an attempt to keep the
logic simple we are gating on it only the
CMake cache entries that strictly control the compilation mode, all the
other entries used for configuring are added in both modes.
Take this chance to remove some stale code from `build-script-impl`, and
move some code in the generic CMake product to the LLVM one.
Addresses rdar://147505298
WASI with Embedded Swift provides WASI-libc and libc++ headers necessary to build the `_Concurrency` module for Wasm. We now add `wasm32-unknown-wasip1-wasm` triple to `EMBEDDED_STDLIB_TARGET_TRIPLES` when `SWIFT_WASI_SYSROOT_PATH` is set, which builds the necessary stdlib slice.
---------
Co-authored-by: Yuta Saito <kateinoigakukun@gmail.com>
If the directory where the build time log is supposed to go doesn't
exist, create it. The append file mode will create files, but won't
create directories. When we start building ninja, we haven't necessary
created the build directory yet, so this results in an error about the
missing directory when writing the build time log.
The ninja builder took a host argument that was unused by the function.
The ninja build failed to pass this argument, resulting in
an execution failure. Removing the argument.
Instead of using `--build-ninja` to decide to build ninja, build it
automatically if a sufficiently new enough version is not available.
Also record the build time taken to build the local Ninja so that we can
see how much time we would save by stashing a pre-built Ninja in CI.
Ninja builds its tests by default.
We don't run the Ninja test suite, we aren't doing development on Ninja,
and we are using a release tag that has been verified to work. There
isn't much point in building the tests if we're not going to use them.
Disabling building the Ninja tests. If it is desirable to build them,
one can set `BUILD_TESTING` to `YES` and re-run their build.
This patch switches the Ninja build from using the configure.py script
to building with the just-built CMake.
The configure.py in Ninja 1.11.1 still uses Python 2.7, importing the
`pipes` module. The pipes module was deprecated in Python 3.11 and
removed in 3.13, so folks using newer versions of Python are running
into issues with this.
The CMake build doesn't have this issue and is also perfectly valid, so
we can switch to that.
This patch updates the CMake-building mechanism to avoid
re-bootstrapping CMake if we already bootstrapped one that is new
enough.
I've made it so that all paths through the function return the path to a
CMake so we can use the result of the function as the cmake path without
having to check.
The function will choose one of the following ways of getting CMake in
order of preference:
- One we already built
- The system CMake
- Bootstrapping one from scratch
It prefers one we built over checking the system CMake because, if we
have already built a CMake previously, it's a good indication that
there either was no system CMake installed, or it wasn't new enough. We
shouldn't waste time checking it again if a previous run detected that
it wasn't good enough.
The system CMake is preferable to building one from scratch if we don't
need to though, so we determine if the system CMake is sufficient.
Finally, if one that we built either doesn't exist, or isn't new enough,
and the system either doesn't have a CMake, or a new enough CMake, build
one. It is built into the location that we are checking for caching, so
the next time we run build-script, it should hit the first case and
choose the already-built CMake instead of building it again.
Include the CMake bootstrap time in the build-script build times.
We're including everything else. Would be good to determine how much
time we can save by caching a new enough pre-built CMake in the builder
images.
Importing the log_time_in_scope exposes a cyclic dependency cycle
between the `swift_build_support` and `build_swift` python modules in
such a way that the tests fail due to re-importing parts of build_swift:
```
ImportError: Failed to import test module: tests.build_swift.test_migration
Traceback (most recent call last):
File "/usr/lib/python3.8/unittest/loader.py", line 436, in _find_test_path
module = self._get_module_from_name(name)
File "/usr/lib/python3.8/unittest/loader.py", line 377, in _get_module_from_name
__import__(name)
File "/home/build-user/swift/utils/build_swift/tests/build_swift/test_migration.py", line 14, in <module>
from build_swift import migration
File "/home/build-user/swift/utils/build_swift/build_swift/migration.py", line 18, in <module>
from swift_build_support.swift_build_support.targets import \
File "/home/build-user/swift/utils/swift_build_support/swift_build_support/targets.py", line 15, in <module>
from . import cmake
File "/home/build-user/swift/utils/swift_build_support/swift_build_support/cmake.py", line 26, in <module>
from swift_build_support.swift_build_support.utils import log_time_in_scope
File "/home/build-user/swift/utils/swift_build_support/swift_build_support/utils.py", line 20, in <module>
from build_swift.build_swift.constants import SWIFT_BUILD_ROOT
ModuleNotFoundError: No module named 'build_swift.build_swift'
```
I've put the import of log_time_in_scope into the function definition
to ensure that build_swift has been fully loaded by the time we need
log_time_in_scope, ensuring that there is order between the two pieces.
Python caches the imported module, so if we accidentally re-import the
log_time_in_scope, nothing actually changes.
This re-orders the instantiation of the BuildScriptInvocation object so
that it comes before the creation of the CMake path. This ensures that
BuildScriptInvocation() does not delete the build log after logging the
CMake bootstrap time. This is fine because the toolchain and arguments
are reference types, so updating the CMake path in both of those will be
reflected in the copy taken in the BuildScriptInvocation() object.
What is nice about this is that by not using extra-cmake-args, we can avoid
passing this into LLVM as well when attempting to reproduce failures on the bots
(thus avoiding having to rebuild LLVM as well).
I have been doing this using extra-cmake-args/etc... just feels better to have
an actual option to do this.
Just did this quickly while waiting for my Linux build to finish that uses
extra-cmake-args to set the linker.
This was quite brittle and has now been superseded
by swift-xcodegen. Remove the CMake/build-script
logic for it, leaving the option behind to inform
users to switch to using xcodegen instead.
When adding a Swift Testing test to Swift PM repository, the `test`
portion of t he OSX package pipeline was building against x86_64 and
arm64.
Ensure Swift PM testing only runs against the host platform
architecture.
The main goal of this change is to ensure that the new build of the
stdlib matches the same level of verbosity of the compiler build that
spawn it.
For now I'm not matching this behaviour to the regular CMake build
products (which would be needed if want to target external projects
configured in LLVM).
Addresses rdar://144256800
