After talking with compnerd, because we're passing this directly to the
swift driver, we shouldn't have to do anything weird with the flags and
it should "just work". I'm frustrated enough with Windows that I'm just
going to go with that and go to bed. If I broke something, I'm sorry. I
tried as far as my sanity would allow.
This patch fixes the part of the build system where we aren't passing
the sanitizer flags to swift, so the linker isn't linking against the
sanitizer libraries.
* add the (still empty) libswift package
* add build support for libswift in CMake
* add libswift to swift-frontend and sil-opt
The build can be controlled with the LIBSWIFT_BUILD_MODE cmake variable: by default it’s “DISABLE”, which means that libswift is not built. If it’s “HOSTTOOLS”, libswift is built with a pre-installed toolchain on the host system.
We have two directories for Swift libraries,
* `SWIFT_SDK_<platform>_LIB_SUBDIR`, a.k.a., the SDK subdir,
a.k.a., `swift-<platform>-<arch>/lib/swift/<platform>`, and
* `SWIFTLIB_SINGLE_SUBDIR`,
a.k.a., `swift-<platform>-<arch>/lib/swift/<platform>/<arch>`.
Through the Swift build, libraries are emitted to both
`.../lib/swift/<platform>` and `.../lib/swift/<platform>/<arch>`.
However, when building toolchains, only `.../lib/swift/<platform>/` is
populated with libraries.
None of this normally isn't a problem; the Swift libraries do not have
inherent interdependencies. This however changes with Concurrency:
Concurrency has an implicit dependency on libdispatch.
When Swift is built, we have two copies of `libswift_Concurrency.so`:
one in `.../lib/swift/<platform>` and one in
`.../lib/swift/<platform>/<arch>`. Prior to this commit, we
unconditionally copy `libdispatch.so` and `libBlocksRuntime.so` to only
_one_ place -- that is, `.../lib/swift/<platform>/<arch>`.
swiftc emits binaries on ELF systems with an rpath of
`.../lib/swift/<platform>`. These binaries implicitly import
Concurrency, so they link against `libswift_Concurrency.so` (whether
they use Concurrency features or not). The library's `$ORIGIN` is
searched to find `libdispatch.so`.
Now, nothing breaks on Linux because there the loader, when given an
rpath, searches both `.../lib/swift/<platform>` and
`.../lib/swift/<platform>/<arch>` even though the rpath only specifies
one directory..
However, on other platforms, only the given rpath is searched.
`libdispatch.so` does not reside next to `libswift_Concurrency.so`
because it has been copied to `.../lib/swift/<platform>/<arch>`; not in
the rpath.
There are a few ways to solve this: change the way rpaths are
configured, only emit libraries into one place, copy `libdispatch.so`
only to the path matching the rpath, or copy `libdispatch.so` wherever
`libswift_Concurrency.so` is copied,
Because the toolchain file layout is different to the file layout when
only Swift is built, hacking the rpath is brittle. Presumably, the
reason why we have a `libswift_Concurrency.so` residing in two places is
to support builds where multiple architectures are supported in the one
build directory, so we cannot just copy `libdispatch.so` _only_ to
`.../lib/swift/<platform>`.
Ultimately, We need to ensure that every instance where
`libswift_Concurrency.so` can be used has `libdispatch.so` residing next
to it, which we do here.
Note that this implicit dependency resolution would not happen unless we
added a `-ldispatch` flag to make this all work, but other platforms are
instaed using `$ORIGIN` to get the search to work properly, so we also
do this for OpenBSD in this commit.
This is just cruft that remained from the time when swift's host/target used the
same cmake code. Host tools do not statically link against just built artifacts.
There are three things going on here (note all on Darwin):
1. If one compiles a swift static library and links the static library into a
cxx executable, the cxx executable will need the -L flags to find the
appropriate libraries in the SDK/toolchain.
2. I fixed an rpath issue where due to old code I didn't understand/propagated
forward, we were setting the rpath to be the -L directory in the appropriate
SDK. After reasoning about this a little bit I realized that this was code
that was actually intended to be used for target libraries (for which the
given rpath would be correct). On the host side though on Darwin, we want to
use the rpath for the standard stabilized stdlib on the system.
3. I added Build System Unittests to ensure that this keeps on working. I also
added test cases that I should have added before. I just had never thought
about how to test this and I realized this method would work and would
prevent regressions while I am waiting for a new swiftc with driver fixes to
land.
Before a8ae9525aa, we could not do this since we
would fail to link since we didn't pass to clang the path to the toolchain dir
when the compatibility libraries live.
Clang and Swift both support this so we can do this until the actual fix lands
in a host toolchain to unblock ErikE.
The specific problem is the swift driver should just push tbd files through to
the linker, but it treats it as an input file. I am going to be putting a fix
into 5.5. This patch in the mean time filters out the tbd files in cmake. This
is a temporary fix.
I also had to add direct dependencies from swift-refactor and
libSwiftSyntaxParser on LLVMSupport to ensure that the right linker flags get to
them.
I already did this for executables. The reason why we must do this is that right
now there are bugs in cmake's swift support that makes it so that one can not
use swiftc to link targets with mixed c/c++/swift content even if the swift
content is indirectly added via linking. To work around this we must also ensure
that all libraries with mixed c/c++/swift objects link using clang. As an
additional side-effect of this, we must pass to the clang linker -L flags that
swiftc would normally provide for the linker. The two cases where I needed to do
this were:
1. -L path for the compatibility libraries. This path points into the toolchain
where these live. This is important since otherwise we will fail to link given
the minimum deployment target that swift (both host/stdlib) target today (10.9).
2. -L path to the SDK. This path points to the SDKs lib/swift/${HOST_PLATFORM}
for swiftCore and friends.
That being said, we still want people to be able to link pure swift libraries
using swiftc, so we introduce an option called PURE_SWIFT to add_host_library
that preserves this and is used to also set
IGNORE_LLVM_UPDATE_COMPILE_FLAGS (which arguably should have been called
PURE_SWIFT).
As a postprocessing step for unittest executables, `utils/swift-rpathize.py` unconditionally rewrites the install name of any library linked from /usr/lib/swift to be `@rpath`-relative, in an effort to load the just-built libraries instead of the OS-supplied ones.
This works great for dylibs like libswiftCore.dylib that are built as part of the toolchain, but `/usr/lib/swift` also includes a huge number of overlay dylibs that are no longer part of this repository. These dylibs must ~always be loaded from the OS location.
In order to prevent load-time issues, we need to add /usr/lib/swift to the rpath, so any libraries that we haven’t built will be picked up from there.
We could also do this by extending swift-rpathize.py with an allow (or deny) list, but keeping the list up to date would generate a bunch of maintenance work we could do without.
The newest VS2019 release updates CMake to 3.20, which picks up
`llvm-mt` as a preferred manifest tool. However, we do not build
`llvm-mt` with libxml2 support currently, which prevents the use of the
just built manifest tool for building libdispatch. Explicitly opt into
using the MSVC manifest-tool.
This is specifically important for host libraries that include swift
content. llvm_update_compile_flags currently doesn't conditionalize the flags it
adds since it needs to be updated like I updated Swift's cmake in
558c9d4086.
In the short amount of time that I am dealing with upstream to patch those
flags, this patch will unblock eeckstein and let him continue with his libswift
work.
In certain cases our executables are going to have a mix of cxx,swift. There are
current issues when linking cxx,swift code with swiftc itself, so this lets us
avoid that problem.
I am in the process now of preparing the tree for the addition of Swift code in
the optimizer as a normal source of source file. The goal is to make it so that
one can just include a random swift library anywhere in the swift project host
build and the cmake will use the swift compiler from the host
toolchain/compile-link the code just as if it was a normal host side thing (1).
In order to do this though, we need to deal with the legacy of our cmake
creating compile flags without constraining the flags to only being used if
cmake is compiling c/c++ code. To fix this, I just inserted generator
expressions into the host side swift build's cmake that uses generator
expressions to perform such conditionalization. The result is that the parts of
a target that are c/c++ will get these flags, but these flags will not propagate
to any Swift files that we add.
(1) With time this implies that we will need to be able to bootstrap the swift
compiler. We are not crossing that bridge now since the only places that we are
going to use this today is in the SILOptimizer optimier passes. These can always
be disabled while cross compiling, meaning that we can make progress here
without having the bootstrapping completely ironed out.
Otherwise we set it on all targets/languages in a subdirectory (I forgot if it
propagates up). Regardless, this type of viral stuff is something we want to
move away from since it creates a code that is a "forall" piece of code rather
than a piece of code that only effects a single target.
I also conditionalized the actual definitions being added on the compiled file's
language being C,CXX,OBJC,OBJCXX since as we add Swift sources to the host side
of the compiler, we will not want these flags to propagate to Swift sources.
gtest, gtest_main LINK_LIBRARIES dependencies changed by that removed scripts to absolute library file path. as a result losing necessary include path dirs.
This will make sure that compiler developers are using the new driver when they build the compiler locally and use it.
- Adds a new build-script product category: before_build_script_impl for products we wish to build before the impl products.
- Adds a new EarlySwiftDriver product to that category, which gets built with the host toolchain.
- Adds an escape hatch: --skip-early-swift-driver
- Adjusts the swift CMake configuration with an additional step: swift_create_early_driver_symlinks which (if one was built) creates symlinks in the swift build bin directory to the EarlySwiftDriver swift-driver and swift-help executables.
- Adds a new test subset : only_early_swiftdriver, which will get built into a corresponding CMake test target: check-swift-only_early_swiftdriver-* which runs a small subset of driver-related tests against the Early SwiftDriver.
- This subset is run always when the compiler itself is tested (--test is specified)
- With an escape disable-switch: --skip-test-early-swift-driver
- All tests outside of only_early_swiftdriver are forced to run using the legacy C++ driver (to ensure it gets tested, still).
NOTE: SwiftDriver product (no 'Early') is still the main product used to build the driver for toolchain installation into and for executing the product's own tests. This change does not affect that.
Commit the platform definition and build script work necessary to
cross-compile for arm64_32.
arm64_32 is a variant of AARCH64 that supports an ILP32 architecture.
add target include directories on AddSwiftUnittests.cmake module.
currently generated Xcode project doesn't include googletest headers after https://reviews.llvm.org/D86616
For cross-compiling Android one can simply provide
CMAKE_ANDROID_ARCH_ABI and the right CMAKE_SYSTEM_PROCESSOR will be
chosen by CMake. Since the SDK arch names are not the expected ones by the Android NDK,
it is better not to use them. The change should keep sending the
CMAKE_SYSTEM_PROCESSOR for SDKs that are not ANDROID, while skipping it
for the ANDROID SDKs.
* [Concurrency] Fix Android C libdispatch build
We need to pass CMAKE_ANDROID_NDK and CMAKE_ANDROID_ARCH_ABI to the
build.
* Set proper ANDROID_ARCH_ABI
* Add -DCMAKE_ANDROID_API to C libdispatch build
* Fix compiler config for Android
Since the NDK removes the platforms/ and sysroot/ directories in the latest NDK
22, switch to the unified sysroot in toolchains/llvm/ and take advantage of a
bunch of simplification that's now possible.
