Commit Graph

435 Commits

Author SHA1 Message Date
Andrew Trick
4a123f102e Fix the build-script --skip-build option.
This option configures the build directories without building any
targets. Splitting configuration from build allows for the decoupling
of build products. This decoupling is essential for the enlightened
way of developing Swift where the build-script is never actually used
to build anything, and build products can be independently
configured. When fully supported, this avoids many unnecessary
full/clean rebuilds and enables debugging by mixing-and-matching
configurations and rebuilding only select products after a change.

Sadly, the option has degraded, and a recent commit rendered it fully broken:

  commit 34848e6026
  Author: Alex Langford <apl@fb.com>
  Date:   Wed Jan 22 19:27:44 2020

    [build] Unify logic to skip building projects in build-script-impl

The breaking commit was itself a reasonable cleanup. The underlying
problem was the original --skip-build was implemented using hacks that
conflated configuration with build.

This fix reinstates a reasonable situation:

--skip-build has no effect on configuration, as documented. It merely
  skips building the targets. This is how it must behave to work as
  intended.

--skip-build-{product} and its inverse --build-{product} controls
  which products will be configured. These options are in heavy use
  throughout the scripts, so changing the name (e.g. to --skip-config)
  would be disruptive and of questionable benefit.

None of this changes the fact that any required build logic that
people have dumped into build-script-impl still effectively breaks the
enlightened way of building Swift, particularly when building
toolchain components.
2020-01-28 13:09:08 -08:00
Saleem Abdulrasool
e26b21675d Merge pull request #29366 from compnerd/optional-pythonkit-support
build: add support for optionally building PythonKit
2020-01-25 09:15:44 -08:00
Saleem Abdulrasool
cad0c7ae21 build: add support for optionally building PythonKit
This adds support for building PythonKit as a package as part of the
toolchain build.
2020-01-24 11:17:41 -08:00
Ross Bayer
603a47266e [Build System: build-script] Allow users to enable distcc via the USE_DISTCC environment variable. 2020-01-24 01:59:54 -08:00
Ross Bayer
cc9ff6aed8 [Build System: build-script] Support Swift versions with up to four version components, matching Clang. 2020-01-23 16:39:58 -08:00
Devin Coughlin
9b4a890262 [Utils] Attempt to appease python_lint 2020-01-21 20:27:14 -08:00
Devin Coughlin
89e381ca66 [CMake/Tests] CMake and lit.cfg support for running macCatalyst tests
Add support for testing with macCatalyst to lit.cfg and the test CMake.

This adds lit test features for whether the standard library and runtime was
built with macCatalyst support:

  REQUIRES: maccatalyst_support

The test suite can also be run in two modes: one where the macOS tests
are run as usual (against a zippered standard library, runtime, and overlays)
and another where iOS tests are compiled with the macCatalyst target
triple and executed as macCatalyst processes.

The iOS tests for macCatalyst can be run by passing `--maccatalyst-ios-tests`
to build-script. There are new lit test features to enable a test to specify
whether it supports that environment:

  REQUIRES: OS=maccatalyst
  UNSUPPORTED: OS=macCatalyst
2020-01-21 18:28:17 -08:00
Devin Coughlin
63ce243437 [CMake] Add initial build system support for macCatalyst
This commit adds initial build system support for macCatalyst,
an Apple technology that enables code targeting iOS
to be recompiled so that it can be executed on macOS while still using
iOS APIs. This is the first in a series of commits building out support for
macCatalyst in the compiler, runtime, standard library, and overlays. Swift
for macCatalyst represents the work of multiple people, including
Devin Coughlin, Ross Bayer, and Brent Royal-Gordon.

Under macCatalyst, compiler-provided shared libraries (including overlays)
are built as one of four kinds (or "flavors") of libraries,
each with different install names and Mach-O load commands. This commit
adds the build system infrastructure to produce these different
library flavors.

**macOS-like Libraries**

A "macOS-like" library (such as the GLKit overlay) is a plain-old macOS library
that can only be loaded into regular macOS processes. It has a macOS slice with
a single load command allowing it to be loaded into normal macOS processes.

**iOS-like Libraries**

An "iOS-like" library, such as the UIKit overlay, is a library with a
macOS slice but with a load command that only allows it be loaded into
macCatalyst processes. iOS-like libraries are produced by passing a new
target tuple to the compiler:

  swiftc ... -target x86_64-apple-ios13.0-macabi ...

Here 'ios' (and an iOS version number) is used for OS portion
of the triple, but the 'macabi' environment tells the compiler
that the library is intended for macCatalyst.

**Zippered Libraries**

A "zippered" library can be loaded into either a macCatalyst process or
a standard macOS process. Since macCatalyst does not introduce a new Mach-O
slice, the same code is shared between both processes. Zippered libraries
are usually relatively low level and with an API surface that is similar
between macOS and iOS (for example, both the Foundation overlay and the Swift
Standard Library/Runtime itself are zippered).

Zippered libraries are created by passing both the usual `-target`
flag to the compiler and an additional `-target-variant` flag:

   swiftc ... -target x86_64-apple-macos10.15 \
              -target-variant x86_64-apple-ios13.0-macabi

Just like the -target flag, -target-variant takes a target tuple.
This tells the compiler to compile the library for the -target tuple but
to add an extra load command, allowing the library to be loaded into processes
of the -target-variant flavor as well.

While a single zippered library and slice is shared between macOS and
macCatalyst, zippered libraries require two separate .swiftinterface/.swiftmodule
files, one for macOS and one for macCatalyst. When a macOS or macCatalyst client
imports the library, it will use module file for its flavor to determine what
symbols are present. This enables a zippered library to expose a subset of its
target APIs to its target-variant.

**Unzippered-Twin Libraries**

"Unzippered Twins" are pairs of libraries with the same name but different
contents and install locations, one for use from macOS processes and one for
use from macCatalyst processes. Unzippered twins are usually libraries that
depend on AppKit on macOS and UIKit on iOS (for example, the MapKit overlay)
and so do not share a common implementation between macOS and macCatalyst.

The macCatalyst version of an unzippered twin is installed in a parallel
directory hierarchy rooted at /System/iOSSupport/. So, for example, while macOS
and zippered Swift overlays are installed in /usr/lib/swift/, iOS-like and
the macCatalyst side of unzippered twins are installed in
/System/iOSSupport/usr/lib/swift. When building for macCatalyst, the build system
passes additional search paths so that the macCatalyst version of libraries is
found before macOS versions.

The add_swift_target_library() funciton now take an
optional  MACCATALYST_BUILD_FLAVOR, which enables swift libraries to indicate
which flavor of library they are.
2020-01-21 18:26:13 -08:00
Ross Bayer
5852aea3e8 [Build System: build-script] Updated the presets module to simplify the API and improve testability. 2020-01-20 02:34:45 -08:00
Ross Bayer
a6dab52f31 [Build System: build-script] Adds a new xcrun module to build_swift which replaces the existing module from swift_build_support.
This new module uses the build_swift.shell.ExecutableWrapper API to create a wrapper class around 'xcrun'. The wrapper class is instantiated and exposed under the name build_swift.wrappers.xcrun.
2020-01-19 17:19:38 -08:00
Ross Bayer
7587c7c0ac [Build System: build-script] Re-structured the build_swift module tests to contain a build_swift directory.
Having the test directory match the module we are testing means we can have scripts in the top level of utils/build_swift which can also have tests. As part of this re-structure the test utilties have been simplified somewhat and all tests no longer use a custom TestCase, rather the standard one exposed by the unittest module.
2020-01-19 01:31:21 -08:00
Ross Bayer
d21a603829 Merge pull request #29268 from Rostepher/shell-module
[Build System: build-script] Shell module.
2020-01-18 15:16:29 -08:00
Ross Bayer
885fd01d49 [Build System: build-script] Adds a new shell module to build_swift which wraps the standard subprocess module and provides convenience functions for common shell tasks. 2020-01-18 12:24:47 -08:00
Ross Bayer
67d6b10dd0 [Build System: build-script] Add a new versions module to build_swift which provides the Version class.
Version acts very similarly to distutils.version.LooseVersion, but with some more flexibility around character group boundries.
2020-01-18 12:04:53 -08:00
Ross Bayer
516ad28c50 Merge pull request #29244 from Rostepher/adopt-six-in-build-swift
[Build System: build-script] Adopt the six compatibility library in the build_swift module.
2020-01-18 11:56:21 -08:00
Andrew Trick
5d3e8b5b7b Merge pull request #29283 from atrick/show-build-targets
Add the target list to build-script --help.
2020-01-17 19:54:49 -08:00
Andrew Trick
90c478f6fc Add the target list to build-script --help.
Teach build-script to print the list of valid targets for the
--stdlib-deployment-targets option. Unfortunately, passing all
supported targets to this option is the only way to force
configuration of those targets. Simply using --ios is no longer
sufficient--none of the iOS targets are actually configured unless you
ask them to be built.

(The reasonable way to use a build config script is to first configure
for all supported platforms, but only build the platforms/targets one
by one when you actually need them).

This currently prints:

  --stdlib-deployment-targets STDLIB_DEPLOYMENT_TARGETS
                        The targets to compile or cross-compile the Swift
                        standard library for. None by default. Comma separated
                        list: android-aarch64 android-armv7 appletvos-arm64
                        appletvsimulator-x86_64 cygwin-x86_64 freebsd-x86_64
                        haiku-x86_64 iphoneos-arm64 iphoneos-armv7 iphoneos-
                        armv7s iphonesimulator-i386 iphonesimulator-x86_64
                        linux-aarch64 linux-armv6 linux-armv7 linux-i686
                        linux-powerpc64 linux-powerpc64le linux-s390x linux-
                        x86_64 macosx-x86_64 watchos-armv7k
                        watchsimulator-i386 windows-x86_64
2020-01-17 18:06:00 -08:00
Ross Bayer
0fdef59633 [Build System: build-script] Adopt the six compatibility library in the build_swift module. 2020-01-17 15:43:54 -08:00
Ross Bayer
bfeb13451e [Build System: build-script] Update the auto-generated preset parsing tests to always supply and exhaustive set of substitutions. 2020-01-17 14:37:15 -08:00
Ross Bayer
7b8401c3f7 [Build System: build-script] Remove the old implementation of the preset parser hidden away in swift_build_support. 2020-01-17 00:25:35 -08:00
Ross Bayer
525708e485 Merge pull request #29265 from Rostepher/remove-assert-not-raises
[NFC][Build System: build-script] Remove TestCase.assertNotRaises method.
2020-01-16 23:30:59 -08:00
Ross Bayer
f5c56f201c [Build System: build-script] Remove the dumb assertNotRaises test method which does less than nothing as it produces right-ward drift without any meaningful semantic benefit. 2020-01-16 21:43:36 -08:00
Ross Bayer
bb22690af0 [Build System: build-script] Moves the rest of the build-script-impl migration code from swift_build_support into build_swift. 2020-01-16 21:00:11 -08:00
Ross Bayer
ff60592ac3 [Build System: build-script] Re-organized the build_swift module to maintain a similar nested structure to swift_build_support and other Python scripting that already exists in the project. 2020-01-15 18:05:59 -08:00
Saleem Abdulrasool
d8cc616602 Merge pull request #28663 from ahoppen/install-swiftsyntax-usr-lib
[build-script] Install SwiftSyntax to usr/lib instead of usr/lib/swift
2020-01-13 14:43:32 -08:00
Mishal Shah
e50b8f9858 [Build System] Add support to set C and CXX launcher with env C_COMPILER_LAUNCHER and CXX_COMPILER_LAUNCHER 2019-12-12 23:47:11 -08:00
Alex Hoppen
91ca8ca093 [build-script] Install SwiftSyntax to usr/lib instead of usr/lib/swift
SwiftSyntax is not part of the standard library and thus should not be
installed in usr/lib/swift.

This also removes the code to install SwiftSyntax's .swiftmodule file
since that code path was never exercised.
2019-12-09 18:27:55 -08:00
najacque
8b378626e4 bumping swift version number to 5.2 rdar://problem/56622958 2019-12-05 17:22:02 -08:00
Eric Miotto
e5e274333f Add support for flag in driver_arguments 2019-12-03 12:09:05 -08:00
Mishal Shah
e74c34c6a1 Bumping Swift version number to 5.1.2 2019-11-30 18:39:05 -08:00
Daniel Rodríguez Troitiño
9498287043 [android] Disable SwiftPM in CI build.
The Android CI build only builds the stdlib because the rest of the
components are build for the host, which is not very useful, since they
are already tested in other CI configurations.
2019-11-04 15:00:45 -08:00
Ankit Aggarwal
7241030876 Move swiftpm to swift_build_support infra
This will allow cleaning up most of the hacks in SwiftPM's build script.

<rdar://problem/56220087>
2019-11-03 03:23:35 +00:00
Ben Langmuir
56e9b2a099 python-lint fixes 2019-10-31 14:44:26 -07:00
Ben Langmuir
11aeb8b7fa Add build-script option --skip-test-toolchain-benchmarks 2019-10-31 09:47:20 -07:00
Alex Hoppen
fcd3457560 [build-script] Migrate SwiftSyntax to swift_build_support 2019-10-29 10:40:09 -07:00
Alex Hoppen
776e2c0030 Revert "Migrate building SwiftSyntax to swift_build_support" 2019-10-29 09:55:32 -07:00
Alex Hoppen
7ed085cb55 [build-script] Migrate SwiftSyntax to swift_build_support 2019-10-25 15:58:07 -07:00
swift-ci
781b1b177f Merge pull request #27809 from ahoppen/typo-build-to-test 2019-10-21 14:56:11 -07:00
Alex Hoppen
af2d9a780a [build-script] Fix typo skip-build-swift -> skip-test-swift 2019-10-21 13:25:23 -07:00
Rintaro Ishizaki
4d31e7e755 [build-script] Fix python-lint issue
rdar://problem/56469555
2019-10-21 11:43:20 -07:00
Alex Hoppen
1926da3fe7 [build-script] Move building swiftevolve to swift_build_support 2019-10-19 19:10:41 -07:00
Alex Hoppen
adbd96bcf1 [build-script] Move building skstresstester to swift_build_support 2019-10-19 19:10:41 -07:00
Dan Zheng
2bd55f6755 [Autodiff upstream] Add build-script flag for differentiable programming. (#27595)
Add `--enable-experimental-differentiable-programming` build-script flag.

The build-script flag enables/disables standard library additions
related to differentiable programming. This will allow official Swift
releases to disable these additions.

The build-script flag is on by default to ensure testing of
differentiable programming standard library additions. An additional
driver flag must be enabled to use differentiable programming features:
https://github.com/apple/swift/pull/27446
2019-10-14 14:34:48 -07:00
swift-ci
f692420e31 Merge pull request #27567 from dcci/skip-local-build 2019-10-08 13:21:24 -07:00
Davide Italiano
b2a1e1c1ff [build-script] Introduce an option to skip local build.
This is useful for cross-compiling.

<rdar://problem/55916729>
2019-10-08 11:16:08 -07:00
Mishal Shah
fff1a39364 [Version] Bump the Swift version to 5.1.1 2019-10-03 14:35:57 -07:00
Ben Langmuir
718a5bf368 [build-script] Installation support for sourcekit-lsp
Add --install-sourcekit-lsp option to build-script and update presets
for package bots to install it.
2019-08-12 13:24:49 -07:00
Ross Bayer
094696702c Merge pull request #25534 from Rostepher/append-no-more-stdlib-deployment-targets
[Build System: build-script] Update the --stdlib-deployment-targets flag to no longer append from multiple uses, instead use the standard last-wins strategy.
2019-06-18 13:24:16 -07:00
Ross Bayer
9dc44b70b5 Merge pull request #25520 from Rostepher/empty-swift-sdks
[Build System: build-script] Update the migration logic for the preset-only option '--swift-sdks' to execute in-place and support empty SDK lists.
2019-06-17 16:15:16 -07:00
Ross Bayer
563bef609f [Build System: build-script] Update the --stdlib-deployment-targets flag to no longer append from multiple uses, instead use the standard last-wins strategy. This aligns better with the existing documentation for the argument and the similar --build-stdlib-deployment-targets flag. 2019-06-17 16:13:58 -07:00