This commit adds initial build system support for macCatalyst,
an Apple technology that enables code targeting iOS
to be recompiled so that it can be executed on macOS while still using
iOS APIs. This is the first in a series of commits building out support for
macCatalyst in the compiler, runtime, standard library, and overlays. Swift
for macCatalyst represents the work of multiple people, including
Devin Coughlin, Ross Bayer, and Brent Royal-Gordon.
Under macCatalyst, compiler-provided shared libraries (including overlays)
are built as one of four kinds (or "flavors") of libraries,
each with different install names and Mach-O load commands. This commit
adds the build system infrastructure to produce these different
library flavors.
**macOS-like Libraries**
A "macOS-like" library (such as the GLKit overlay) is a plain-old macOS library
that can only be loaded into regular macOS processes. It has a macOS slice with
a single load command allowing it to be loaded into normal macOS processes.
**iOS-like Libraries**
An "iOS-like" library, such as the UIKit overlay, is a library with a
macOS slice but with a load command that only allows it be loaded into
macCatalyst processes. iOS-like libraries are produced by passing a new
target tuple to the compiler:
swiftc ... -target x86_64-apple-ios13.0-macabi ...
Here 'ios' (and an iOS version number) is used for OS portion
of the triple, but the 'macabi' environment tells the compiler
that the library is intended for macCatalyst.
**Zippered Libraries**
A "zippered" library can be loaded into either a macCatalyst process or
a standard macOS process. Since macCatalyst does not introduce a new Mach-O
slice, the same code is shared between both processes. Zippered libraries
are usually relatively low level and with an API surface that is similar
between macOS and iOS (for example, both the Foundation overlay and the Swift
Standard Library/Runtime itself are zippered).
Zippered libraries are created by passing both the usual `-target`
flag to the compiler and an additional `-target-variant` flag:
swiftc ... -target x86_64-apple-macos10.15 \
-target-variant x86_64-apple-ios13.0-macabi
Just like the -target flag, -target-variant takes a target tuple.
This tells the compiler to compile the library for the -target tuple but
to add an extra load command, allowing the library to be loaded into processes
of the -target-variant flavor as well.
While a single zippered library and slice is shared between macOS and
macCatalyst, zippered libraries require two separate .swiftinterface/.swiftmodule
files, one for macOS and one for macCatalyst. When a macOS or macCatalyst client
imports the library, it will use module file for its flavor to determine what
symbols are present. This enables a zippered library to expose a subset of its
target APIs to its target-variant.
**Unzippered-Twin Libraries**
"Unzippered Twins" are pairs of libraries with the same name but different
contents and install locations, one for use from macOS processes and one for
use from macCatalyst processes. Unzippered twins are usually libraries that
depend on AppKit on macOS and UIKit on iOS (for example, the MapKit overlay)
and so do not share a common implementation between macOS and macCatalyst.
The macCatalyst version of an unzippered twin is installed in a parallel
directory hierarchy rooted at /System/iOSSupport/. So, for example, while macOS
and zippered Swift overlays are installed in /usr/lib/swift/, iOS-like and
the macCatalyst side of unzippered twins are installed in
/System/iOSSupport/usr/lib/swift. When building for macCatalyst, the build system
passes additional search paths so that the macCatalyst version of libraries is
found before macOS versions.
The add_swift_target_library() funciton now take an
optional MACCATALYST_BUILD_FLAVOR, which enables swift libraries to indicate
which flavor of library they are.
When force linking auto-linked libraries, an overlay will fail to link if the dependence
libraries are missing from the source. This change provides linker flags
to search overlay libraries from the SDK.
Now that CMAKE_HOST_SYSTEM_NAME and CMAKE_SYSTEM_NAME are set by default to
Android in the Termux app, make the needed tweaks. Some tests were adapted
to work natively on Android too, adds sys/cdefs.h to the Bionic modulemap,
and includes the start of native Android platform support in the build-script.
Add `--enable-experimental-differentiable-programming` build-script flag.
The build-script flag enables/disables standard library additions
related to differentiable programming. This will allow official Swift
releases to disable these additions.
The build-script flag is on by default to ensure testing of
differentiable programming standard library additions. An additional
driver flag must be enabled to use differentiable programming features:
https://github.com/apple/swift/pull/27446
Certain versions of CMake behave differently with variable expansion. This may
be evaluated incorrectly in some versions. Dereference the value explicitly.
This commit introduces a CMake target for each component, adds install targets
for them, and switches build-script-impl to use the target `install-components`
for installation. Each of the targets for each component depends on each
of the individual targets and outputs that are associated with the
corresponding swift-component.
This is equivalent to what already exists, because right now install rules are
only generated for components that we want to install. Therefore, this commit
should be an NFC.
This is a resubmission (with modifications) of an earlier change. I originally
committed this but there were problems with some installation rules.
This commit introduces a CMake target for each component, adds install targets
for them, and switches build-script-impl to use the target `install-components`
for installation. Each of the targets for each component depends on each
of the individual targets and outputs that are associated with the
corresponding swift-component.
This is equivalent to what already exists, because right now install rules are
only generated for components that we want to install. Therefore, this commit
should be an NFC.
This makes it more explicit what the install component of a target
library is if you don't see one (and its marked as IS_SDK_OVERLAY).
Explicit in this case makes more sense, as you don't have to rely on
knowledge of how `add_swift_target_library` is implemented to understand
what component is used to install the target.
This option should be `SWIFTLIB_SINGLE_TARGET_LIBRARY`. If you rely on
`SWIFTLIB_TARGET_LIBRARY`, you get whatever was set in add_swift_target_library.
While these should be the same, I'm trying to remove `SWIFTLIB_TARGET_LIBRARY`
entirely.
Check if building on Android through the ANDROID_DATA environment variable, then set
SWIFT_ANDROID_NATIVE_SYSROOT to the default layout for the Termux app, and key all the
include, lib, and other SDK paths off of that. The system libc and a few other libraries
are linked against from /system/lib[64]. Finally, check if lit is running natively on
Android and don't use adb if so.
add_swift_target_library should always have `TARGET_LIBRARY` set. A few
lines down, there is a precondition that makes sure that it is set. This
check and subsequent assignment of `SWIFTLIB_INSTALL_IN_TARGET` should
never get triggered.
When installing a target, you could have multiple target files with
different rules for installation. For example, you can specify rules for
`RUNTIME` files, `LIBRARY` files, `ARCHIVE` files, etc. Each of these
can take several options, e.g. `DESTINATION`. When you want to specify
the install component for a target, each of the different target file
rules need to have their own `COMPONENT` field. If not, they end up as
"Unspecified" which means they will not get installed with the
associated swift component.
The system headers should be given lower priority than the user
specified headers. This requires that we include the system C++ headers
as `-isystem`. Without this, it is not possible to use a custom
toolchain to replace the clang compiler for building the android
runtime.
On Android, we explicitly setup the include paths for the C++ headers.
As a result, we do not want to pick up any system headers that the
toolchain may have visible to it. If this flag is not added then clang
will look in the wrong places to get the headers for things like stddef
etc.
Some versions of CMake dereference the variable here and others do not.
Explicitly dereference to ensure that the library name is properly set.
Failure to do so will fail to load the library as the output path
becomes the name.
The clang in the android NDK (as well as the current stable Clang in
Swift) have a bug with Android x86_64 targets where the CPU's 16-byte
(double-word) CAS instruction feature is disabled resulting in the
double word CAS being lowered to a libcall
(`__sync_val_compare_and_swap_16`) rather than the CPU instruction
(`cmpxchg16b`). This results in the inability to link the content.
Work around this by enabling the feature manually in the build for now.
A previous change did the same change, but only for libraries. The
executables also need to be build to be linked as C to avoid clang++
automatic linking of C++.
Use the variable that holds the path to the ICU libraries to link
against ICU rather than hardcoding the names. This restores control to
the ICU linking to the way that traditional CMake `find_package` based
linking works.
Rather than using `-nolibstd++` which is not supported by the android toolchain,
use CMake's `LINKER_LANGAUGE` property on the targets. Since we explicitly link
against the C++ runtime anyways, this avoids the problem of having to remove the
C++ runtime from the link.
We need to pass `-fuse-ld=` `gold.exe` on Windows when targeting
Android. Given that we do not have a host toolchain for Linux on
Windows currently, do this unconditionally. This is required for
cross-compiling from Windows to android.
Android doesn't support the GNU backtrace call, but unwind.h is there to
help. The code is a little bit more complicated than a simple backtrace
call, but it is just a callback and the context struct.
Sadly, it seems that -funwind-tables is needed in every piece of code
that the backtrace needs to traverse, which means that at least the
runtime and the stubs need -funwind-tables. However, the flag has been
enabled for all targets (only on Android), just in case some other code
ends up being in the stack. This might only be necessary for ARM
targets, but currently we only build ARM targets in Android. The Swift
parts of the stack seems to have unwind information and can be unwind.
Another problem at the moment is the symbols from the executable are
normally missing. To get symbol names two things need to happen: the
symbol has to be public (so it is an exported symbol, and not just a
local symbol that dladdr seems to not provide the name), and the linker
has to be invoked with -export-dynamic. Without those two pieces, the
addresses are returned (and match the binary just fine), but no names
are printed.
Although it's a static archive, its use is only relevant to dynamically linked builds of the
standard library. (If you're statically linking a Swift runtime into your process, you don't
need to worry about compatibility with older runtimes.)
