This commit adds initial build system support for macCatalyst,
an Apple technology that enables code targeting iOS
to be recompiled so that it can be executed on macOS while still using
iOS APIs. This is the first in a series of commits building out support for
macCatalyst in the compiler, runtime, standard library, and overlays. Swift
for macCatalyst represents the work of multiple people, including
Devin Coughlin, Ross Bayer, and Brent Royal-Gordon.
Under macCatalyst, compiler-provided shared libraries (including overlays)
are built as one of four kinds (or "flavors") of libraries,
each with different install names and Mach-O load commands. This commit
adds the build system infrastructure to produce these different
library flavors.
**macOS-like Libraries**
A "macOS-like" library (such as the GLKit overlay) is a plain-old macOS library
that can only be loaded into regular macOS processes. It has a macOS slice with
a single load command allowing it to be loaded into normal macOS processes.
**iOS-like Libraries**
An "iOS-like" library, such as the UIKit overlay, is a library with a
macOS slice but with a load command that only allows it be loaded into
macCatalyst processes. iOS-like libraries are produced by passing a new
target tuple to the compiler:
swiftc ... -target x86_64-apple-ios13.0-macabi ...
Here 'ios' (and an iOS version number) is used for OS portion
of the triple, but the 'macabi' environment tells the compiler
that the library is intended for macCatalyst.
**Zippered Libraries**
A "zippered" library can be loaded into either a macCatalyst process or
a standard macOS process. Since macCatalyst does not introduce a new Mach-O
slice, the same code is shared between both processes. Zippered libraries
are usually relatively low level and with an API surface that is similar
between macOS and iOS (for example, both the Foundation overlay and the Swift
Standard Library/Runtime itself are zippered).
Zippered libraries are created by passing both the usual `-target`
flag to the compiler and an additional `-target-variant` flag:
swiftc ... -target x86_64-apple-macos10.15 \
-target-variant x86_64-apple-ios13.0-macabi
Just like the -target flag, -target-variant takes a target tuple.
This tells the compiler to compile the library for the -target tuple but
to add an extra load command, allowing the library to be loaded into processes
of the -target-variant flavor as well.
While a single zippered library and slice is shared between macOS and
macCatalyst, zippered libraries require two separate .swiftinterface/.swiftmodule
files, one for macOS and one for macCatalyst. When a macOS or macCatalyst client
imports the library, it will use module file for its flavor to determine what
symbols are present. This enables a zippered library to expose a subset of its
target APIs to its target-variant.
**Unzippered-Twin Libraries**
"Unzippered Twins" are pairs of libraries with the same name but different
contents and install locations, one for use from macOS processes and one for
use from macCatalyst processes. Unzippered twins are usually libraries that
depend on AppKit on macOS and UIKit on iOS (for example, the MapKit overlay)
and so do not share a common implementation between macOS and macCatalyst.
The macCatalyst version of an unzippered twin is installed in a parallel
directory hierarchy rooted at /System/iOSSupport/. So, for example, while macOS
and zippered Swift overlays are installed in /usr/lib/swift/, iOS-like and
the macCatalyst side of unzippered twins are installed in
/System/iOSSupport/usr/lib/swift. When building for macCatalyst, the build system
passes additional search paths so that the macCatalyst version of libraries is
found before macOS versions.
The add_swift_target_library() funciton now take an
optional MACCATALYST_BUILD_FLAVOR, which enables swift libraries to indicate
which flavor of library they are.
This new module uses the build_swift.shell.ExecutableWrapper API to create a wrapper class around 'xcrun'. The wrapper class is instantiated and exposed under the name build_swift.wrappers.xcrun.
Add `--enable-experimental-differentiable-programming` build-script flag.
The build-script flag enables/disables standard library additions
related to differentiable programming. This will allow official Swift
releases to disable these additions.
The build-script flag is on by default to ensure testing of
differentiable programming standard library additions. An additional
driver flag must be enabled to use differentiable programming features:
https://github.com/apple/swift/pull/27446
- Forward several environment variables to the test environment because
Windows uses them to inform the processes about things like the number
of processors and the architecture.
- Normalize some literal Unix paths to be the same as the results in
Windows, that will have forward slashes and the drive letter.
- Skip some test that use build-script-impl and tests that check for
files being executable (everything is executable in Windows).
- Don't use the owner and group arguments for tar on Windows.
- Hide the stderr output of which. In Windows it prints the full PATH in
case of failures, which is disrupting.
- Quote many paths in Windows in the output of build-script results.
- Provide a version of mock-distcc that can be executed in Windows. The
raw Python script cannot.
- Change the expected results for clang/clang++ to the right values in
Windows (clang-cl in both cases).
This will allow using `HostSpecificConfiguration` from other parts that
are not the main script in the future. This is interesting because the
information is mostly useful when building Swift. The rest of products
are not really interested in the results of these calculations.
Includes a suite of tests that check the implementation correctly
calculates the right targets to build under diverse circumstances.
CMakeOptions was briefly used in the cmake module, but mostly unused in
the rest of the modules, however, several products were dealing with
what eventually will end as CMake options, but not using the already
existing code, and duplicating it.
The changes tries to unify all usage around CMakeOptions which avoids
code duplication. Additionally, it provides some more API surface in
CMakeOptions to make it more useful.
- The initializer can be passed a list of tuples or another CMakeOptions
to copy into the newly created options.
- Boolean Python values are treated automatically as boolean CMake
options and transformed into `TRUE` and `FALSE` automatically.
- Provides `extend`, which will add all the tuples from a list or all
the options from another `CMakeOptions` into the receiving
`CMakeOptions`.
- Provides `__contains__`, which makes checking for existing options a
little easier (however, being `CMakeOptions` basically a list, the
checking has to include also the value, which is not that helpful).
- Modify LLVM and Swift products to use `CMakeOptions`. It makes the
boolean values clearer and avoid a lot of repetitive string
formatting.
- Modify the tests to make them pass and provide new test for newly
introduced features.
ProductBuilder allows us to tackle the different way than the different
products need to be build. The builders follow a very simple interface,
but inside them the details are hidden.
Previously the Ninja product was both a Product and ProductBuilder. The
methods that did the build have moved into ProductBuilder to match the
future ProductBuilders.
As proposed in #23822, the do_* methods should only be using the * parts. Because all building should have the same names, this commit removes the prefix from all the method instances.
This reverts commit b9e17a7656.
Unfortunately, this caused conflicts on the Apple internal CI.
Reverting this for now. This can be split up into smaller patches to
make it easier to merge.
This will allow using HostSpecificConfiguration from other parts that
are not the main script in the future.
Additionally, a lot of code that was part of BuildScriptInvocation has
been moved into HostSpeficiaConfiguration because nobody else needed it,
and HostSpecificConfiguration was highly coupled with the Invocation
object (through one of its arguments). This should make the
Configuration object a better citizen in other scenarios.
I have been meaning to do this change for a minute, but kept on putting it off.
This describes what is actually happening and is a better name for the option.
This is a cleaner, more principled way of adding "compiler launcher" support and
ensures that cmake understands that distcc is not the "actual" compiler.
This ensures that when we compile SwiftRemoteMirrors for the host, we do not try
to compile using distcc without needing to reset CMAKE_{C,CXX}_COMPILER_ARG1
(which is unset when compiling things in the stdlib).
This only enables the swift compiler (not its output) to be used with leaks
sanitizer on linux.
Some important notes: On Linux, right now we are not completely leak clean. I
was only able to get a -Onone build of the stdlib without triggering lsan (I was
unable to run the optimizer and the tests successfully). Additionally even at
-Onone, I had to suppress some leaks in the driver. The point of this though is
to prevent any further -Onone leaks from being committed to the tree since
-Onone leaks that are not bounded (unlike the driver bugs) could cause SourceKit
to leak memory. Since SourceKit is a long running process... such a type of leak
would be bad.
rdar://32876901
Why are the python reasons for having our code nested in a directory
called ``swift_build_support.swift_build_support/`` instead of simply
``swift_build_support/``? Is that what we really want here?
In a normal swiftless clang build, if it doesnt know CLANG_VERSION_*,
clang's cmake falls back on LLVM_VERSION_*. We pass a CLANG_COMPILER_VERSION
to swift, but it is not necessarily what clang wants to know.
The intention of this CLANG_VERSION_* flag in Clang is to know
what Swift calls clang_user_visible_version. This is the non-specific
version of Clang. For example, clang-900 - this is 9.0.0, while Swift
would like clang_compiler_version to be the specific version that the
driver will print out.
Flag naming!
Fix tests
Fix the mocked output files of swiftc. Change the lookup of
clang from build-script to build-script-impl like all the other
commands. Pass more args to LLVM. Fix swiftenv creation. Fix unit tests.
The reason this patch works is that build-script-impl gets
all the arguments that are not handled by build-script.
Today, when you enable ASAN, build-script just reconfigures your normal build
directory to use ASAN. This forces you to recompile LLVM and Swift with ASAN
enabled and then (once you have finished using ASAN) to recompile LLVm/Swift
without ASAN.
By using a different build-directory, one still has to (potentially) recompile
LLVM/Swift, but one's original directory has not become invalidated. Thus when
you switch back to a normal build, one does not have to recompile LLVM and
(potentially) Swift!
This should help speed up people trying to compile the standard library and do
SILGen work. *NOTE* This will not necessarily result in a type checker that is
as fast as a release build since most likely the type checker will use some
link_once odr functions that are debug. But it should still be significantly
faster otherwise.
This makes getting to SILGen take 16 seconds on my machine instead of forever
when compiling with everything else in the compiler in debug mode.
