Commit Graph

409 Commits

Author SHA1 Message Date
Saleem Abdulrasool
de7f292460 utils: adjust parameter for SDKs in build.ps1
This is a renaming of the `-SDKs` parameter to `-WindowsSDKs` to prepare
for building additional SDKs with `build.ps1`.
2024-02-12 12:34:50 -08:00
Saleem Abdulrasool
98b285b4d5 Merge pull request #71482 from tristanlabelle/wix-4.0.4
Update WiX to 4.0.4
2024-02-09 11:23:58 -08:00
Saleem Abdulrasool
72dc80e9ec Update build.ps1
Update hash for 4.0.4
2024-02-09 08:24:11 -08:00
Tristan Labelle
d251c04c5e Update WiX to 4.0.4 2024-02-08 16:58:04 -05:00
Saleem Abdulrasool
2fcab70914 utils: use enumerations to locate build locations
Rather than hardcode the build numbers, use named parameters to locate
them during the build process.
2024-02-07 14:31:47 -08:00
Saleem Abdulrasool
5953d87d85 utils: add support to use the swift-driver as the driver
This is motivated by an experiment to build DocC with CMake which
exposed a bug in the Swift driver. This allows us to workaround that bug
with the Swift Swift driver.
2024-02-06 09:21:30 -08:00
Saleem Abdulrasool
4e971a4064 Merge pull request #71378 from compnerd/summary
utils: gussy up timing information summary for Windows
2024-02-05 08:13:08 -08:00
Saleem Abdulrasool
485ff3e7bc utils: gussy up timing information summary for Windows
Emit this in a `finally` block ensuring that it is printed at the end
under all circumstances. Reduce the column width by making the checkout
`SourceCache` relative.
2024-02-04 11:20:23 -08:00
Saleem Abdulrasool
c6b0084c8f utils: add support for testing self-hosted SPM
`-Test swiftpm` will use the self-hosted build of SPM to run the test
suite. By using the `SWIFTCI_USE_LOCAL_DEPS=1` environment variable, we
do reduce some of the cost of this by not recloning the repositories,
but the entire dependency tree must be built a second time due to the
switch in the build system.
2024-02-03 10:53:39 -08:00
Saleem Abdulrasool
2da252e42a utils: introduce new -Summary option for build.ps1
Add a new `-Summary` option to understand the build times expenditure.
This prints a summary at the end of the build time, though should be
further refined to include tests, auxiliary operations, and the like.

~~~
Arch  Checkout                                                BuildID                 Elapsed Time
----  --------                                                -------                 ------------
x64   S:\SourceCache\swift-project\swift\tools\swift-inspect  swift-inspect           00:00:00.2372939
x64   S:\SourceCache\swift-project\sqlite-3.43.2              sqlite-3.43.2           00:00:00.5062697
x64   S:\SourceCache\swift-project\zlib                       zlib-1.3                00:00:00.5615253
arm64 S:\SourceCache\swift-project\zlib                       zlib-1.3                00:00:00.5623726
x86   S:\SourceCache\swift-project\zlib                       zlib-1.3                00:00:00.6185541
x64   S:\SourceCache\swift-project\swift-system               2                       00:00:00.6562858
x64   S:\SourceCache\swift-project\Yams                       5                       00:00:00.6647429
x64   S:\SourceCache\swift-project\swift-certificates         11                      00:00:00.6649416
arm64 S:\SourceCache\swift-project\swift-corelibs-xctest      304                     00:00:00.6688467
x64   S:\SourceCache\swift-project\swift-asn1                 10                      00:00:00.6795648
x64   S:\SourceCache\swift-project\swift-corelibs-xctest      104                     00:00:00.6938559
x64   S:\SourceCache\swift-project\swift-argument-parser      6                       00:00:00.7351350
x64   S:\SourceCache\swift-project\swift-tools-support-core   3                       00:00:00.7400418
x64   S:\SourceCache\swift-project\indexstore-db              15                      00:00:00.7402386
x86   S:\SourceCache\swift-project\swift-corelibs-xctest      204                     00:00:00.7414125
x64   S:\SourceCache\swift-project\cmark                      cmark-gfm-0.29.0.gfm.13 00:00:00.7435080
x64   S:\SourceCache\swift-project\swift-crypto               8                       00:00:00.7593080
arm64 S:\SourceCache\swift-project\libxml2                    libxml2-2.11.5          00:00:00.7659147
x64   S:\SourceCache\swift-project\libxml2                    libxml2-2.11.5          00:00:00.7885430
x64   S:\SourceCache\swift-project\sourcekit-lsp              16                      00:00:00.8459008
arm64 S:\SourceCache\swift-project\icu\icu4c                  icu-69.1                00:00:00.8544044
x86   S:\SourceCache\swift-project\libxml2                    libxml2-2.11.5          00:00:00.8552676
x64   S:\SourceCache\swift-project\icu\icu4c                  icu-69.1                00:00:00.8986529
x64   S:\SourceCache\swift-project\curl                       curl-8.4.0              00:00:00.9157392
arm64 S:\SourceCache\swift-project\curl                       curl-8.4.0              00:00:00.9292362
x64   S:\SourceCache\swift-project\swift-package-manager      12                      00:00:00.9352117
x86   S:\SourceCache\swift-project\icu\icu4c                  icu-69.1                00:00:00.9890039
x86   S:\SourceCache\swift-project\curl                       curl-8.4.0              00:00:01.0041779
x64   S:\SourceCache\swift-project\llbuild                    4                       00:00:01.0835003
arm64 S:\SourceCache\swift-project\swift-corelibs-libdispatch 302                     00:00:01.2923527
x64   S:\SourceCache\swift-project\swift-corelibs-libdispatch 102                     00:00:01.2940821
x86   S:\SourceCache\swift-project\swift-corelibs-libdispatch 202                     00:00:01.3725038
x64   S:\SourceCache\swift-project\swift-docc                 swift-docc              00:00:01.5638843
x64   S:\SourceCache\swift-project\swift-corelibs-foundation  103                     00:00:03.2156545
arm64 S:\SourceCache\swift-project\swift-corelibs-foundation  303                     00:00:03.2375292
x86   S:\SourceCache\swift-project\swift-corelibs-foundation  203                     00:00:03.3768394
x64   S:\SourceCache\swift-project\swift                      101                     00:00:03.4123377
x86   S:\SourceCache\swift-project\swift                      201                     00:00:03.4822752
arm64 S:\SourceCache\swift-project\swift                      301                     00:00:03.5000946
x64   S:\SourceCache\swift-project\swift-collections          9                       00:00:04.1620695
x64   S:\SourceCache\swift-project\swift-format               14                      00:00:07.9155136
arm64 S:\SourceCache\swift-project\llvm-project\llvm          300                     00:00:08.0239769
x64   S:\SourceCache\swift-project\llvm-project\llvm          100                     00:00:08.0504294
x86   S:\SourceCache\swift-project\llvm-project\llvm          200                     00:00:08.1190769
x64   S:\SourceCache\swift-project\swift-markdown             13                      00:00:12.4397452
x64   S:\SourceCache\swift-project\swift-driver               7                       00:00:24.9125831
x64   S:\SourceCache\swift-project\llvm-project\llvm          1                       00:01:04.5879144
x64   S:\SourceCache\swift-project\llvm-project\llvm          0                       00:01:10.9225001
~~~
2024-02-02 14:16:35 -08:00
Saleem Abdulrasool
fe90ed91a6 utils: adjust CMake flags for debug info handling
Use the new CMake options for controlling debug information generation.
This matches the recommendations from sccache and avoids us trying to
map flags manually.
2024-02-01 11:28:38 -08:00
Saleem Abdulrasool
d021c54533 Merge pull request #71284 from compnerd/optionality
utils: clean up unnecessary cmake options for packaging
2024-02-01 08:17:57 -08:00
Saleem Abdulrasool
37c593b076 utils: fix debug info generation for the standard library
When we build the standard library, we use the just built compilers.
However, we would not pass the necessary flags to the linker to generate
the debug information (PDBs) which would then make debugging more
challenging than it already is.
2024-01-31 15:29:32 -08:00
Saleem Abdulrasool
086134164d utils: clean up unnecessary cmake options for packaging
swift-format is no longer optional and is always built. The options
controlling the packaging have been removed. Update the invocation.
2024-01-31 13:16:13 -08:00
Saleem Abdulrasool
b58889ae55 utils: tweak cmark wiring
Use the installed image rather than the build tree for wiring up CMark.
This may be used across phases of the build, so it is better to use the
staged image.
2024-01-30 14:00:31 -08:00
Saleem Abdulrasool
81edeb1cab utils: use named enumerator for target libraries
We would use integral values to identify the target library rather than
using a named enumerator. Change to a named enumerator and inline some
cases where we would explain the code by creating variables.
2024-01-29 13:44:29 -08:00
Saleem Abdulrasool
1864caad5e build: migrate to CMake based build of swift-format
This allows us to build swift-format with dynamic linking against the
toolchain build of the swift-syntax and swift-argument-parser packages.
Wire up the swift-markdown build and hoist the swift-format build prior
to sourcekit-lsp. This sets us up for supporting swift-format based
formatting in the LSP.
2024-01-26 15:51:29 -08:00
Saleem Abdulrasool
0d3f6a9f06 Merge pull request #70791 from compnerd/cmark
build: update for cmark build revamp and split the build
2024-01-24 14:40:23 -08:00
Tristan Labelle
efab1ffe01 Build.ps1: more detailed exception printing 2024-01-24 11:39:51 -05:00
Saleem Abdulrasool
9e4153499f build: add a -Clean option to build.ps1
When the compiler changes, the SDK and tool builds do not notice the
compiler change due to the missing compiler tracking. This allows us to
quickly perform an incremental by not discarding the compiler build.
2024-01-19 10:34:46 -08:00
Saleem Abdulrasool
49a5098db4 build: update for cmark build revamp and split the build
Extricate cmark from the unified build system. This is preparatory work
to support building CMark GFM with dynamic linkage for sharing it across
swift-markdown and the compiler. It also simplifies the build logic for
the unified build.
2024-01-17 08:54:36 -08:00
Saleem Abdulrasool
b876662a46 Merge pull request #70879 from compnerd/install-to
build: prefer `-InstallTo` over custom CMake options
2024-01-15 11:02:33 -08:00
Saleem Abdulrasool
bab4fba196 build: prefer -InstallTo over custom CMake options
Re-use the existing `-InstallTo` option over the `CMAKE_INSTALL_PREFIX`
CMake parameter for building the targets. This cleans up the options
usage to use that mechanism. The intention is to minimise the custom
parameters to CMake in favour of a standard option handling.
2024-01-12 08:53:13 -08:00
Alejandro Alonso
677bcd6a97 Build Synchronization on Windows installer 2024-01-11 12:37:22 -08:00
Saleem Abdulrasool
a1146ff01b Merge pull request #70719 from compnerd/logs
build: put the toolchain extraction logs into the build tree
2024-01-05 07:41:47 -08:00
Saleem Abdulrasool
4609f0d8f0 Merge pull request #70665 from compnerd/md
build: tweak the Windows build for clean builds
2024-01-04 14:45:45 -08:00
Saleem Abdulrasool
321b8268c8 build: put the toolchain extraction logs into the build tree
Use an absolute path to ensure that we do not place the logs in the
current directory.
2024-01-04 10:53:36 -08:00
Hiroshi Yamauchi
30b6ab9e0c Fix the pinned toolchain extraction. 2024-01-03 16:13:16 -08:00
Saleem Abdulrasool
8b47616d6d build: simplify the unified build configuration
Conditionalise the inclusion of directory paths. If the include
directory is empty, CMake will raise an error. This allows us to disable
larger portions of the Swift build to speed up some of the build tool
configuration phase and prepares for consuming CMark GFM as a CMake
project.
2024-01-02 12:58:50 -08:00
Saleem Abdulrasool
5d1d2dd505 build: tweak the Windows build for clean builds
Add a directory creation path prior to downloading. In the case that the
build tree did not exist, we would fail to fetch the content. This
allows us to build on a fresh setup.
2024-01-02 08:09:16 -08:00
Saleem Abdulrasool
9a19aec0c1 build: remove -DefaultsLLD option for build.ps1
This option was rarely used.  Remove the option to simplify the build
types as we are growing in complexity.
2023-12-21 07:16:29 -08:00
Saleem Abdulrasool
5746d6ae3f build: restructure content fetching
Centralise the downloading of dependencies.  This merges the SDK,
toolchain, and WiX downloading.  This is a small clean up and
preparatory work to enable fetching sccache as well to ease the caching
setup.
2023-12-20 11:03:16 -08:00
Saleem Abdulrasool
e4d7cec837 Merge pull request #70543 from compnerd/debug
build: only apply linker flags for C/C++
2023-12-20 09:50:30 -08:00
Saleem Abdulrasool
2c5dce5898 build: update WiX to 4.0.3
Update to the latest WiX toolset for the extraction process in the hopes
that this will resolve some of the flakiness we see on the 5.10 branch.
2023-12-19 11:53:58 -08:00
Saleem Abdulrasool
a755a48202 build: only apply linker flags for C/C++
When building C/C++ code we need to add `/debug` to the flags.  However,
if we are building Swift code, this becomes a problem as the flag is
passed to the Swift compiler directly.  Only pass the flag when C/C++
code is being built.
2023-12-19 11:37:20 -08:00
Alex Lorenz
9db866f881 Merge pull request #70524 from hyp/eng/pdb-for-all-of-us
[windows][build] pass the /debug flag to the linker
2023-12-18 19:01:59 -08:00
Alex Lorenz
d2c897a1a1 [windows][build] pass the /debug flag to the linker
when building with debug information enabled

This ensures that every executable and dll get a corresponding
PDB file with the debug information.
2023-12-18 13:01:30 -08:00
Saleem Abdulrasool
ef1f9ebcb5 Merge pull request #70497 from compnerd/testing
build: add support for testing more of the compilers on Windows
2023-12-15 15:55:19 -08:00
Saleem Abdulrasool
96f1a449f5 build: add support for testing more of the compilers on Windows
This adds the ability to conditionally run the test suite for LLVM, LLD,
LLDB, Clang, and Swift.  This will allow us to increase the test
coverage in CI to avoid regressions.
2023-12-15 09:53:54 -08:00
Ian Anderson
94e860e795 [ClangImporter] Swift needs to pass -Xclang -fbuiltin-headers-in-system-modules for its module maps that group cstd headers
Swift has some module maps it overlays on Linux and Windows that groups all of the C standard library headers into a single module. This doesn’t allow clang and C++ headers to layer properly with the OS/SDK modules. clang will set -fbuiltin-headers-in-system-modules as necessary for Apple SDKs, but Swift will need to pass that flag itself when required by its module maps.
2023-12-08 22:33:12 -08:00
Saleem Abdulrasool
7978272185 utils: integrate support for caching when building
Integrate support for sccache into the build system.  This allows us to
get a reasonable performance increase for clean builds for the C/C++
side of the build.  This is currently limited to the MSVC compiler as
the extended compiler flags cause issues for sccache.
2023-12-04 08:00:36 -08:00
Saleem Abdulrasool
61bf9a4380 Update build.ps1
Update the variables for the pinned-toolchain usage to correctly specify the compilers.  Additionally use the new helpers to work with the new toolchain layout as well.
2023-12-02 19:33:28 -08:00
Saleem Abdulrasool
ecc302b7d7 Merge pull request #69743 from tristanlabelle/tristan/build.ps1-vcredist-msms
Update build.ps1 for installer changes for VCRuntime dlls
2023-11-30 08:35:44 -08:00
Hiroshi Yamauchi
5009b006a1 Merge pull request #70050 from hjyamauchi/nativearm64build
Support the new installer layout for the pinned toolchain in build.ps1.
2023-11-29 17:39:17 -08:00
Saleem Abdulrasool
d1bb5f68db Merge pull request #69960 from compnerd/tools
build: disable additional pieces for the build tools phase
2023-11-29 13:42:23 -08:00
Hiroshi Yamauchi
0150d0d3c5 Support the new installer layout for the pinned toolchain in build.ps1.
This fixes the native Windows ARM64 build.
2023-11-28 15:46:13 -08:00
Saleem Abdulrasool
e575578792 build: enable HTTPS for CURL again
Adjust the flags for the updated CURL version.  It seems that we accidentally disabled HTTPS support in the CURL builds with the last update.  This is required for `URLSession` and `URLRequest` to support HTTPS.
2023-11-28 14:14:13 -08:00
Tristan Labelle
9c7872788c build.ps1: Build installer wixproj as bundle 2023-11-17 17:00:41 -05:00
Saleem Abdulrasool
e3ecce14eb build: disable additional pieces for the build tools phase
This reduces the configure time and reduces the pieces that may check
for dependencies as we are growing the dependencies on Swift.
2023-11-17 09:10:17 -08:00
Saleem Abdulrasool
8f7d3fca24 build: add additional logging when extracting the MSIs
This adds additional logging for the extraction and explicitly marks the
extraction as per-machine.
2023-11-14 07:02:44 -08:00