Commit Graph

945 Commits

Author SHA1 Message Date
Charles Zablit
f09449467b [update-checkout] refactor shell invocations 2025-10-22 10:56:19 -07:00
Charles Zablit
9d3d43c920 [update-checkout] switch to using a ThreadPool for scheduling git commands 2025-10-16 11:29:56 +01:00
Zhenchao Li
ed6ef79b51 Add brotli support to FoundationNetworking (#83441)
<!--
If this pull request is targeting a release branch, please fill out the
following form:

https://github.com/swiftlang/.github/blob/main/PULL_REQUEST_TEMPLATE/release.md?plain=1

Otherwise, replace this comment with a description of your changes and
rationale. Provide links to external references/discussions if
appropriate.
If this pull request resolves any GitHub issues, link them like so:

  Resolves <link to issue>, resolves <link to another issue>.

For more information about linking a pull request to an issue, see:

https://docs.github.com/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue
-->

<!--
Before merging this pull request, you must run the Swift continuous
integration tests.
For information about triggering CI builds via @swift-ci, see:

https://github.com/apple/swift/blob/main/docs/ContinuousIntegration.md#swift-ci

Thank you for your contribution to Swift!
-->

Add brotli as a dependency to enable brotli decoding for curl.

brotli is a general purpose compression algorithm used extensively on
web. (https://datatracker.ietf.org/doc/html/rfc7932)

brotli support for curl is enabled by integrating the brotli compression
library (https://github.com/google/brotli). I've also added some tests
for FoundationNetworking in
https://github.com/swiftlang/swift-corelibs-foundation/pull/5251 which
exercises this feature by
- looking at the accept-encoding http header 
- decoding a brotli encoded payload
2025-10-10 15:52:40 -07:00
Bassam Khouri
a96e339474 build-script: remove --verbose when building swiftpm
The SwiftPM stage added `--verbose` in #79330 to help troubleshoot build
error that were occuring on Linux.  These errors have since been
addresed, but the verbosity was not removed.

Remove the `--verbose` command line argument in the SwiftPM portion of
the `build-script`.
2025-10-03 21:17:22 -04:00
swift-ci
8028ad41cc Merge remote-tracking branch 'origin/main' into rebranch 2025-09-15 16:34:55 -07:00
Yuta Saito
5431bb115b [test][wasm] Add test to kill the missing func addr reservation 2025-09-15 14:30:10 +00:00
Anthony Latsis
44c480e83b Mnually merge main -> rebranch
Conflicts:
* `lib/ClangImporter/ClangModuleDependencyScanner.cpp`:
  https://github.com/swiftlang/swift/commit/5e09125
*	`utils/build.ps1`: https://github.com/swiftlang/swift/pull/83523
2025-08-19 21:51:34 +01:00
Max Desiatov
35823eca9f Make logging in wasmkit.py resilient to file renaming 2025-08-18 12:33:52 +01:00
Max Desiatov
e47f650880 Update utils/swift_build_support/swift_build_support/products/wasmkit.py 2025-08-16 16:20:52 +01:00
Max Desiatov
b055217430 Build WasmKit with host SwiftPM when SwiftPM is not built
Currently, passing `--wasmkit` to `build-script` also requires `--swiftpm --install-swiftpm` and all of their dependencies (at the very least Dispatch, Foundation, XCTest, Testing, llbuild, swift-build). We can conditionally switch to host SwiftPM in this case.
2025-08-16 14:52:05 +01:00
swift-ci
18cdf90d50 Merge remote-tracking branch 'origin/main' into rebranch 2025-08-12 22:56:03 -07:00
Kavon Farvardin
6cb38edad4 Merge pull request #83622 from kavon/kavon/test-opaque-values
test: provide an opaque-values testing mode
2025-08-12 19:57:57 -07:00
Kavon Farvardin
c3e0ce5bb6 test: provide an opaque-values testing mode
Introduces a new build-script flag `--test-optimize-none-with-opaque-values`
and build targets like `check-swift-optimize_none_with_opaque_values` that
runs the execution_test's while compiling them with -enable-sil-opaque-values.

This mode is quite useful while working to bring-up opaque values.
2025-08-12 15:07:40 -07:00
swift-ci
c08ba7749f Merge remote-tracking branch 'origin/main' into rebranch 2025-08-09 02:55:57 -07:00
finagolfin
4a3ef140d8 [build] Make the new --cross-compile-build-swift-tools flag public (#83422)
This new flag makes it easy to build Swift cross-compilation toolchains,
by disabling cross-compilation of all host tools, like the Swift
compiler and various macros, building on prior pulls #38441 and #82163.

Native compilation has more fine-grained flags like
`--build-swift-tools`, `--build-llvm`, and `--swift-testing-macros`, but
those will disable building them for _all_ platforms (with the exception
of `--build-swift-tools`, which I modified to only apply for the host,
since the initial introduction of this
`--cross-compile-build-swift-tools` flag a couple months ago), so
they're not suited for building a cross-compilation toolchain, where you
want all host tools built for the native host but not for any
cross-compilation SDKs.
2025-08-09 13:51:11 +05:30
Anthony Latsis
eaafb0ed7e Manually merge remote-tracking branch 'origin/main' into rebranch
Conflicts:
- lib/ClangImporter/ClangImporter.cpp
2025-08-05 14:24:07 +01:00
Eric Miotto
4de714fa51 Merge pull request #83464 from edymtt/edymtt/separate-deployment-version-for-tests
Test: allow different deployment target versions for tests on Darwin
2025-08-04 12:13:30 -07:00
Stefan Gränitz
d510e471d6 Reapply "Drop references to clang-pseudo-gen after removal"
pseudo was removed from clang-tools-extra in upstream PR https://github.com/llvm/llvm-project/pull/109154
Drop two explicit references in swift and unblock Windows CI
2025-08-04 18:13:36 +01:00
finagolfin
09a751f698 [build][android] Use a CMake toolchain file to cross-compile Testing (#83260)
Also, disable a recently failing test for Android armv7 and enable another one
that now passes.
2025-08-02 15:27:45 +05:30
Eric Miotto
4e854dea88 Do not require CMake to be in the path for build-script tests 2025-07-31 08:52:55 -07:00
Eric Miotto
0869919eca Test: allow different deployment target versions for tests on Darwin
Currently we are using the same versions we use to build compiler and
standard library. This is an unnecessary coupling especially when
testing the generation of backdeployed executables for macOS -- here
we have interest in being able to run the tests on previous OSes, not
the compiler itself.

To support this, add new `--darwin-test-deployment-version-<platform>`
flags to build-script, which by default take the same value as the
matching `--darwin-deployment-version-<platform>` ones.

Addresses rdar://156724078
2025-07-31 08:19:13 -07:00
Marc Prud'hommeaux
663ec9340d Support 16 KB page sizes on Android (#81596)
Android 15+ requires that native libraries be compiled with a linker
flag to support 16 KB page sizes. See:
https://developer.android.com/guide/practices/page-sizes#compile-r26-lower
2025-07-29 20:42:22 +05:30
Yuta Saito
dd6eb62d65 [wasm] Fix installation paths for libxml2.a
Clang driver only passes arch-specific library paths as search paths
to the linker for WebAssembly targets but we were installing
libxml2.a under `<sysroot>/lib` without the multiarch triple. It led
to the linker not finding libxml2.a when importing FoundationXML
2025-07-27 10:37:04 +00:00
Max Desiatov
c447bf5283 utils: add typing annotations to migration.py/utils.py (#83293)
These typing annotations are supported since Python 3.6, while the oldest Python we have to support is 3.7 on Amazon Linux 2.
2025-07-24 12:32:14 -07:00
Max Desiatov
15ceabc660 test/CMakeLists.txt: fix --lit-args unhandled in Wasm tests (#83285)
Due to the fact that Wasm testing targets are configured by Python `build-script` unlike `build-script-impl` for the rest of testing targets, `--lit-args` value was ignored and not recorded in these targets. These are configured as `LLVM_LIT_ARGS` in CMake.

Let's assign the `LLVM_LIT_ARGS` configuration value in Python `build-script` to handle `--lit-args` consistently for all testing targets.
2025-07-24 12:26:25 -07:00
Max Desiatov
9b23c08008 wasmstdlib.py: embedded stdlib not built for threads, exclude its tests 2025-07-23 14:32:54 +01:00
Max Desiatov
03f7ebc890 test/CMakeLists.txt: run Embedded Swift for Wasm tests
Run new `check-swift-embedded-wasi` target from `test/CMakeLists.txt`, tweak `lit.cfg` to support both `-wasi` and `-wasip1` triples, exclude unsupported tests based on `CPU=wasm32` instead of `OS=wasi`.
2025-07-23 10:10:47 +01:00
Max Desiatov
c535723b05 Merge pull request #83208 from MaxDesiatov/wasm-corelibs-host-compiler
wasmswiftsdk.py: consistently apply `build_runtime_with_host_compiler`

Following up on https://github.com/swiftlang/swift/pull/83134 https://github.com/swiftlang/swift/pull/82949 https://github.com/swiftlang/swift/pull/82946 https://github.com/swiftlang/swift/pull/82944.

With these changes I'm able to quickly build and test only Wasm Swift SDK without rebuilding the whole toolchain when latest development snapshot is installed, using this invocation:

```
./swift/utils/build-script --skip-build-benchmarks --release --skip-early-swift-driver \
  --skip-build-lldb --skip-build-cmark --skip-build-llvm --skip-build-swift \
  --build-runtime-with-host-compiler --build-wasm-stdlib 
```

Timing on M4 MacBook Pro:

```
--- Build Script Analyzer ---
Build Percentage         Build Duration (sec)    Build Phase
================         ====================    ===========
43.6%                    257.75                  Building wasmswiftsdk
20.6%                    121.91                  Building wasmthreadsstdlib
20.6%                    121.78                  Building wasmstdlib
5.4%                     32.03                   Building wasmllvmruntimelibs
3.6%                     21.39                   Running tests for wasmthreadsstdlib
3.5%                     20.91                   Running tests for wasmstdlib
2.7%                     15.74                   Building wasilibc
0.0%                     0.04                    macosx-arm64-extractsymbols
0.0%                     0.04                    macosx-arm64-package
0.0%                     0.03                    merged-hosts-lipo-core
0.0%                     0.03                    merged-hosts-lipo
Total Duration: 591.65 seconds (9m 51s)
```
2025-07-22 17:24:27 +01:00
Ben Barham
85f7797dd6 Merge pull request #82514 from bnbarham/use-compiler-version
Use SWIFT_COMPILER_VERSION before SWIFT_TOOLCHAIN_VERSION if it exists
2025-07-21 13:38:50 -07:00
Max Desiatov
44a68115bf Merge pull request #83135 from MaxDesiatov/test-wasm-stdlib
build-script: allow `--test-wasm-stdlib` without `--build-wasm-stdlib`

This allows testing Wasm stdlib without fully rebuilding WASI sysroot and stdlib, while changes to stdlib itself are still detected by ninja as dependencies and are built incrementally, which is great for local builds.
2025-07-21 17:22:25 +01:00
Max Desiatov
fa57cfc335 wasmswiftsdk.py: avoid CMAKE_AR/RANLIB with build_runtime_with_host_compiler 2025-07-21 16:56:14 +01:00
Max Desiatov
485333879f wasmswiftsdk.py: consistently apply build_runtime_with_host_compiler
Following up on https://github.com/swiftlang/swift/pull/83134 https://github.com/swiftlang/swift/pull/82949 https://github.com/swiftlang/swift/pull/82946 https://github.com/swiftlang/swift/pull/82944
2025-07-21 16:38:33 +01:00
Max Desiatov
cf0e96c84b build_script_invocation.py: add a note about wasi-sdk-26 adoption 2025-07-21 12:14:48 +01:00
Yuta Saito
e76997aad2 Merge pull request #83191 from kateinoigakukun/yt/xctest-build-wasi 2025-07-21 18:42:04 +09:00
Yuta Saito
960afc5754 Merge pull request #83167 from kateinoigakukun/yt/wasip1
[wasm] Rename `wasm32-unknown-wasi` to `wasm32-unknown-wasip1`
2025-07-21 10:35:33 +09:00
Yuta Saito
686d65513d [wasm] Build and install XCTest for Wasm Swift SDK
Install static XCTest library and its Swift module files in the following
directory structure:

```
usr/lib/swift_static/wasi/libXCTest.a
usr/lib/swift_static/wasi/XCTest.swiftmodule
```
2025-07-20 00:14:34 +00:00
Max Desiatov
0cd57218aa Merge pull request #83134 from MaxDesiatov/consistent-wasmkit-path
wasm-run.py: consistent `wasmkit` path with `build_runtime_with_host_compiler`

When passing `--build-runtime-with-host-compiler` there's no such executable `wasmkit-cli` in the host toolchain, where it's named `wasmkit` when installed. Let's make that consistent to make it work in both cases.
2025-07-18 15:52:18 +01:00
Max Desiatov
b9721408f7 wasmkit.py: copy to wasmkit not wasmkit-cli path 2025-07-18 09:54:46 +01:00
Yuta Saito
3a0d5fd0a1 [wasm] Rename wasm32-unknown-wasi to wasm32-unknown-wasip1 2025-07-18 02:03:16 +00:00
Max Desiatov
b6c791784d [NFC] wasmstdlib.py: fix Python typing error (#83125)
Fix type error in `should_test_executable`:
```
reportIncompatibleMethodOverride: Method "should_test_executable" overrides class "WasmStdlib" in an incompatible manner
Return type mismatch: base method returns type "Literal[True]", override returns type "Literal[False]"
"Literal[False]" is not assignable to type "Literal[True]"
```
2025-07-17 16:14:30 -07:00
Max Desiatov
f895331630 wasisysroot.py: fix build_runtime_with_host_compiler diagnostics (#83129)
Improve diagnostics when `llvm-ar` is not detected in the currently select toolchain.
2025-07-17 21:59:52 +01:00
Max Desiatov
95bbfd3406 build-script: allow --test-wasm-stdlib without --build-wasm-stdlib
This allows testing Wasm stdlib without rebuilding WASI sysroot and stdlib, while some changes are still detected by ninja as dependencies and are rebuilt, which is great for local incremental builds.
2025-07-17 17:46:01 +01:00
Ben Barham
ec46a4f026 Use SWIFT_COMPILER_VERSION before SWIFT_TOOLCHAIN_VERSION if it exists
We should eventually split these, but for now SWIFT_COMPILER_VERSION
matches SWIFT_TOOLCHAIN_VERSION when it's actually set. Just use it in
preference to SWIFT_TOOLCHAIN_VERSION.
2025-07-11 15:21:43 -07:00
Max Desiatov
113ddb45d5 wasisysroot.py: respect args.build_runtime_with_host_compiler (#82946)
This allows quickly building WASI sysroot when `--skip-build-llvm --skip-build-swift --build-runtime-with-host-compiler --build-wasm-stdlib` combination of options is passed to `build-script`.
2025-07-11 11:51:46 -07:00
finagolfin
d2ea4789d8 Merge pull request #82924 from finagolfin/cmake
[build] Fix libcurl and cross-compiling corelibs
2025-07-11 16:15:03 +05:30
Max Desiatov
f1a1bb09f7 WasmLLVMRuntimeLibs: respect args.build_runtime_with_host_compiler (#82949)
This allows quickly building `WasmLLVMRuntimeLibs` product with the host toolchain (usually Swift nightly development snapshot toolchain) when `--skip-build-llvm --skip-build-swift --build-runtime-with-host-compiler` combination of options is passed to `build-script`.
2025-07-10 17:58:22 -07:00
Max Desiatov
b0f1283c40 wasmstdlib.py: respect args.build_runtime_with_host_compiler (#82944)
This allows quickly building WASI-libc and Swift stdlib with WASI when `--skip-build-llvm --skip-build-swift --build-runtime-with-host-compiler` combination of options is passed to `build-script`.
2025-07-10 14:22:18 -07:00
Finagolfin
fd369624de [build] Fix libcurl for typos and correct name of build target 2025-07-10 12:41:09 +05:30
Max Desiatov
6f6ce0ad53 build-script: Add --install-wasmkit option (#82871)
This allows skipping the installation for WasmKit when building it, which is consistent to how other products are handled by `build-script`, where they have separate `--install` options. Existing behavior is preserved: passing `--wasmkit` alone will install it, but now `--wasmkit --install-wasmkit false` will skip its installation step.
2025-07-08 20:26:54 +01:00
Max Desiatov
9ae9696cd3 build-script: add missing ignore_extra_cmake_options for Wasm (#82873)
With the addition of libxml2 and Foundation builds, lack of `ignore_extra_cmake_options=True` regressed macOS builds, where `extra-cmake-options` from macOS presets are picked up, overriding required `CMAKE_CXX_FLAGS` that need to be used for these products instead.
2025-07-08 17:09:45 +01:00