Ninja requires CMake 3.15 or newer to build, while the CMake on Amazon
Linux 2 is only 2.8. We bootstrap a newer CMake anyway, so build it
before Ninja so that Ninja has a new enough CMake.
Instead of using `--build-ninja` to decide to build ninja, build it
automatically if a sufficiently new enough version is not available.
Also record the build time taken to build the local Ninja so that we can
see how much time we would save by stashing a pre-built Ninja in CI.
This patch updates the CMake-building mechanism to avoid
re-bootstrapping CMake if we already bootstrapped one that is new
enough.
I've made it so that all paths through the function return the path to a
CMake so we can use the result of the function as the cmake path without
having to check.
The function will choose one of the following ways of getting CMake in
order of preference:
- One we already built
- The system CMake
- Bootstrapping one from scratch
It prefers one we built over checking the system CMake because, if we
have already built a CMake previously, it's a good indication that
there either was no system CMake installed, or it wasn't new enough. We
shouldn't waste time checking it again if a previous run detected that
it wasn't good enough.
The system CMake is preferable to building one from scratch if we don't
need to though, so we determine if the system CMake is sufficient.
Finally, if one that we built either doesn't exist, or isn't new enough,
and the system either doesn't have a CMake, or a new enough CMake, build
one. It is built into the location that we are checking for caching, so
the next time we run build-script, it should hit the first case and
choose the already-built CMake instead of building it again.
Include the CMake bootstrap time in the build-script build times.
We're including everything else. Would be good to determine how much
time we can save by caching a new enough pre-built CMake in the builder
images.
Importing the log_time_in_scope exposes a cyclic dependency cycle
between the `swift_build_support` and `build_swift` python modules in
such a way that the tests fail due to re-importing parts of build_swift:
```
ImportError: Failed to import test module: tests.build_swift.test_migration
Traceback (most recent call last):
File "/usr/lib/python3.8/unittest/loader.py", line 436, in _find_test_path
module = self._get_module_from_name(name)
File "/usr/lib/python3.8/unittest/loader.py", line 377, in _get_module_from_name
__import__(name)
File "/home/build-user/swift/utils/build_swift/tests/build_swift/test_migration.py", line 14, in <module>
from build_swift import migration
File "/home/build-user/swift/utils/build_swift/build_swift/migration.py", line 18, in <module>
from swift_build_support.swift_build_support.targets import \
File "/home/build-user/swift/utils/swift_build_support/swift_build_support/targets.py", line 15, in <module>
from . import cmake
File "/home/build-user/swift/utils/swift_build_support/swift_build_support/cmake.py", line 26, in <module>
from swift_build_support.swift_build_support.utils import log_time_in_scope
File "/home/build-user/swift/utils/swift_build_support/swift_build_support/utils.py", line 20, in <module>
from build_swift.build_swift.constants import SWIFT_BUILD_ROOT
ModuleNotFoundError: No module named 'build_swift.build_swift'
```
I've put the import of log_time_in_scope into the function definition
to ensure that build_swift has been fully loaded by the time we need
log_time_in_scope, ensuring that there is order between the two pieces.
Python caches the imported module, so if we accidentally re-import the
log_time_in_scope, nothing actually changes.
This re-orders the instantiation of the BuildScriptInvocation object so
that it comes before the creation of the CMake path. This ensures that
BuildScriptInvocation() does not delete the build log after logging the
CMake bootstrap time. This is fine because the toolchain and arguments
are reference types, so updating the CMake path in both of those will be
reflected in the copy taken in the BuildScriptInvocation() object.
`os._exit` does not run any cleanup handlers or flush any buffers, which
means that we may end up missing helpful log output. It was being used
to avoid printing the log analysis when `--help` was passed, but we can
have the same behavior by just not using `finally`.
This was quite brittle and has now been superseded
by swift-xcodegen. Remove the CMake/build-script
logic for it, leaving the option behind to inform
users to switch to using xcodegen instead.
Currently, usage of `--preset-vars-file` (added in https://github.com/swiftlang/swift/pull/76058) introduces newlines in values of preset variables passed via this file. Calling `strip()` will remove trailing and preceding whitespace characters, including newlines.
Bump the deployment target from macOS 10.13-aligned versions to macOS
13.0-aligned versions. This allows us to stop linking CoreFoundation
in the swift runtime, which was previously required for availability
checking. It also lets us align the deployment target on x86_64 with
arm64, which was 11.0. Finally, it is a prerequisite to being able to
build swift using the macOS 15 beta SDKs.
It's not always convenient to pass preset substitution arguments on an interactive command-line. Sometimes one might also need to generate these vars automatically, outside of a `build-script` invocation and also store those vars in a file for later reuse.
This change adds a new optional `--preset-vars-file` customization point to `build-script`, which allows users to pass a a file with newline separated key-value pars for preset variables substitution in addition to passing those in a direct `build-script` invocation.
Declare a new `LINUX_STATIC` SDK and configure it.
Add options to set the build architectures for the `LINUX` and
`LINUX_STATIC` SDKs, similar to what we have for Darwin, because
we'll be cross-compiling.
Also add an option to point the build system at the sources for
the musl C library, which we're using for `LINUX_STATIC`.
rdar://123503470
This change introduces a new compilation target platform to the Swift compiler - visionOS.
- Changes to the compiler build infrastrucuture to support building compiler-adjacent artifacts and test suites for the new target.
- Addition of the new platform kind definition.
- Support for the new platform in language constructs such as compile-time availability annotations or runtime OS version queries.
- Utilities to read out Darwin platform SDK info containing platform mapping data.
- Utilities to support re-mapping availability annotations from iOS to visionOS (e.g. 'updateIntroducedPlatformForFallback', 'updateDeprecatedPlatformForFallback', 'updateObsoletedPlatformForFallback').
- Additional tests exercising platform-specific availability handling and availability re-mapping fallback code-path.
- Changes to existing test suite to accomodate the new platform.
This allows any versions above a specific version, instead of requiring
a specific list. Since it seems like many folks just disable this check
entirely, this list definitely isn't kept up well. If we need to exclude
a specific version, like if 1 Xcode beta didn't support it but the next
did, we can add a new list using the old method to validate build
numbers and disallow the bad version(s).
I don't recall why we didn't do this over a list originally, nor can I
find any context on why in the original discussion
https://github.com/apple/swift/pull/34227
When a user passes `--sccache` flag when `sccache` is not installed, we still attempt to make a shell invocation to `sccache`, which leads to an inscrutable error:
```
Traceback (most recent call last):
File "/root/swift-source/swift/utils/build-script", line 789, in <module>
exit_code = main()
File "/root/swift-source/swift/utils/build-script", line 784, in main
return main_normal()
File "/root/swift-source/swift/utils/build-script", line 687, in main_normal
shell.capture([toolchain.sccache, "--show-stats"])
File "/root/swift-source/swift/utils/swift_build_support/swift_build_support/shell.py", line 133, in capture
return subprocess.check_output(command, env=_env, stderr=stderr,
File "/usr/lib/python3.10/subprocess.py", line 420, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
File "/usr/lib/python3.10/subprocess.py", line 501, in run
with Popen(*popenargs, **kwargs) as process:
File "/usr/lib/python3.10/subprocess.py", line 969, in __init__
self._execute_child(args, executable, preexec_fn, close_fds,
File "/usr/lib/python3.10/subprocess.py", line 1720, in _execute_child
and os.path.dirname(executable)
File "/usr/lib/python3.10/posixpath.py", line 152, in dirname
p = os.fspath(p)
TypeError: expected str, bytes or os.PathLike object, not NoneType
```
`validate_arguments` should be called before `shell.capture([toolchain.sccache, "--show-stats"])` so that a more meaningful error message is shown to the user, asking them to install `sccache`.
The current implementation currently requires to have physical machine
for each architecture supported by macOS, which is not desirable.
To allow all cases to be tested on a random Mac machine, allow
to inject an arbitratry current architecture into the inference
of the cross compile hosts by means of an environment variable.
Addresses rdar://99096874
The CI currently uses Xcode 13.0 beta 4. It is very likely that if the build starts failing with Xcode 12.x, this will go unnoticed. Let's warn the users to upgrade to a newer version of Xcode.
The `__future__` we relied on is now, where the 3 specific things are
all included [since Python 3.0](https://docs.python.org/3/library/__future__.html):
* absolute_import
* print_function
* unicode_literals
* division
These import statements are no-ops and are no longer necessary.
The library `six` provides compatibility between Python 2, and 3. It's no
longer necessary once we migrate of Python 2 completely.
Also remove any custom logic for Python 2 (the ones referenced by
a commentanyways).
https://bugs.swift.org/browse/SR-16025
Currently, when trying to build with the latest stable Xcode, you get this error:
```
using unsupported Xcode version:
Xcode 13.2.1
Build version 13C100
```
Xcode 13.2.1 can successfully build the toolchain and should be marked as supported.
This changes build-swift script shebang to use Python 3 explicitly. The
code is already compatible and functional with python3, so this removes
any implied backward compatibility, and removes any ambiguity based on
the user's current environment, especially since some systems, like
macOS, still link 'python' to Python 2.
This also removes the now unnecessary 'from __future__' imports, and the
'six' module, which is only necessary for python2 compatibility.
This shrinks the number of lines in build-script by 50% and just makes it easier
to read. It will also enable me to refactor parts of it into pieces without
touching the main build-script file. This is really library code... build-script
itself doesn't care about how BuildScriptInvocation is actually implemented
under the hood.
Pass the Swift and CMake flags needed to cross-compile Foundation and so on, with
the first example of Android. Add a new flag, --cross-compile-deps-path, which is
used to search for cross-compiled libraries, like libcurl, that the corelibs
depend on. Also add a new flag, --common-swift-flags, to pass additional Swift
flags to the corelibs.
