[Driver] implement -static-stdlib for Linux
Implement the -static-stdlib driver flag for Linux, allowing the static
linking of the standard library.
This implementation largely follows the Darwin implementation in #1817,
although some pecularities warrant extended discussion.
The original "link with stdlib" implementation had some redundancies
with getRuntimeLibraryPath; these redundancies are resolved, and the
implementation alternates between getRuntimeLibraryPath and
getStaticRuntimeLibraryPath cleanly as appropriate.
A variety of libraries are required to link statically on Linux. The
implementation currently dynamically links with them. We should
probably support static linking of those as well, but I think that is
beyond the scope of a -static-stdlib flag.
The test coverage uses ldd here, as otool is not available on Linux. As
a result, we currently have separate tests for Linux vs the other
platforms; that isn't ideal, but it seems necessary.
Perhaps the oddest part, and the one worth the most discussion, is the
use of --dynamic-list. Inside
stdlib/public/runtime/ProtocolConformances.cpp appears the following
code:
#elif defined(__ELF__)
static int _addImageProtocolConformances(struct dl_phdr_info *info,
size_t size, void *data) {
// inspectArgs contains addImage*Block function and the section name
InspectArgs *inspectArgs = reinterpret_cast<InspectArgs *>(data);
void *handle;
if (!info->dlpi_name || info->dlpi_name[0] == '\0') {
handle = dlopen(nullptr, RTLD_LAZY);
} else
handle = dlopen(info->dlpi_name, RTLD_LAZY | RTLD_NOLOAD);
auto conformances = reinterpret_cast<const uint8_t*>(
dlsym(handle, inspectArgs->sectionName));
The effect of this is to search for protocol_conformances_start inside
the images. However, dlsym only finds symbols that exist in the dynamic
table. Failure to find the protocol conformances can be diagnosed by a
"hello world" program printing
String(_core: Swift._StringCore(_baseAddress: Swift.OpaquePointer(_rawValue: (Opaque Value)), _countAndFlags: Swift.UInt(_value: (Opaque Value)), _owner: Swift.Optional<Swift.AnyObject>.none))
instead of "hello world". (And also by the test coverage in this commit.)
Surprisingly, this behavior can still occur on ELF platforms even if
`objdump -t` reports a valid `.protocol_conformances_start`. This is
because `objdump -t` searches the global table, not the dynamic table,
while dlsym only searches the dynamic table. To configure objdump to
search only the dynamic table, use `-T`.
Inquiring minds may wonder whether dynamically-linked programs (e.g. all
Linux binaries up until now) also have a broken protocol conformance
table on ELF. The answer is, surprisingly, no; I checked, and ordinary
ELF programs are fine. The distinction is probably this, from the ld
manpage:
> the dynamic symbol table will normally contain only those
symbols which are referenced by some dynamic object mentioned in the
link.
I think the linker sees `.protocol_conformances_start` inside
libswiftCore.so and erroneously concludes the one in *the executable* is
"referenced by some dynamic object" (e.g. the standard library). This
behavior seems to save the dyanmically-linked executable from a broken
protocol conformance table. I wonder if it would be wise to apply a
similar fix to dynamically-linked programs to avoid relying on the
linker "helping" us here, but that's out of scope of this commit.
The linker manpage reflects that many people have been bitten by dlsym
"surprise", and encourages the use of `--export-dynamic`:
> If you use "dlopen" to load a dynamic object which needs to refer back
> to the symbols defined by the program, rather than some other dynamic
> object, then you will probably need to use [--export-dynamic] when
> linking the program itself.
However in this situation, the use of `--export-dynamic` causes the
entire stdlib to be exported, which is not ideal. However, by combining
with the `--exclude-libs ALL` argument, we avoid exporting the entire stdlib.
This commit adds the flags -static-stdlib and -no-static-stdlib to
create programs statically linked (or not) with the standard library.
Not is the default, which is also the current behavior.
These flags are currently placebos on non-Darwin platforms.
On the Raspberry Pi 2 when trying to import Glibc, without this patch, it will attempt to
find the module map at "/usr/lib/swift/linux/armv7l/glibc.modulemap" and
fail to do so.
With this patch it will attempt to find the module map at
"/usr/lib/swift/linux/armv7/glibc.modulemap" where it will succeed in
finding the module map.
Similar behavior currently happens in the Driver and Frontend. To DRY up
this behavior it has been extracted to the Swift platform.
This adds an Android target for the stdlib. It is also the first
example of cross-compiling outside of Darwin.
Mailing list discussions:
1. https://lists.swift.org/pipermail/swift-dev/Week-of-Mon-20151207/000171.html
2. https://lists.swift.org/pipermail/swift-dev/Week-of-Mon-20151214/000492.html
The Android variant of Swift may be built using the following `build-script`
invocation:
```
$ utils/build-script \
-R \ # Build in ReleaseAssert mode.
--android \ # Build for Android.
--android-ndk ~/android-ndk-r10e \ # Path to an Android NDK.
--android-ndk-version 21 \
--android-icu-uc ~/libicu-android/armeabi-v7a/libicuuc.so \
--android-icu-uc-include ~/libicu-android/armeabi-v7a/icu/source/common \
--android-icu-i18n ~/libicu-android/armeabi-v7a/libicui18n.so \
--android-icu-i18n-include ~/libicu-android/armeabi-v7a/icu/source/i18n/
```
Android builds have the following dependencies, as can be seen in
the build script invocation:
1. An Android NDK of version 21 or greater, available to download
here: http://developer.android.com/ndk/downloads/index.html.
2. A libicu compatible with android-armv7.
There is currently a great deal of duplication across the
`GenericUnix` and `Windows` toolchains. The Android port will
add even more duplication.
To mitigate this, have `Windows` inherit from `GenericUnix`, and
have them share most of their implementation.
In addition, rename `Windows` to `Cygwin` (it would be pretty strange
to have a `Windows` toolchain inherit from something named `*Unix`).
ASan allows to catch and diagnose memory corruption errors, which are possible
when using unsafe pointers.
This patch introduces a new driver/frontend option -sanitize=address to enable
ASan. When option is passed in, the ASan llvm passes will be turned on and
all functions will gain SanitizeAddress llvm attribute.
(1) We no longer put the Clang version string in our copy of or symlink to
Clang's resource directory.
(2) Newer Clang builds now generate a separate library for the Apple OS
simulators, instead of a fat binary.
We still need a proper end-to-end test for this, but that depends on
building compiler-rt with Swift, which isn't a standard config yet.
Exposes the global warning suppression and treatment as errors
functionality to the Swift driver. Introduces the flags
"-suppress-warnings" and "-warnings-as-errors". Test case include.
This is only the driver side of the work; the frontend doesn't understand
this new -output-filelist option yet. Next commit.
More https://bugs.swift.org/browse/SR-280.
Start sketching out a way for individual jobs to request filelists for
their inputs or their outputs. This should cover all the cases mentioned
in ad945426.
More https://bugs.swift.org/browse/SR-280.
Generate frontend commands with -filelist in them. This isn't actually
implemented yet, but we can start testing at this point.
Part 1 of https://bugs.swift.org/browse/SR-280.
Previously jobs had to grovel this information out of the raw argument
list, which dropped the types we had inferred on input files. This
makes things more consistent across the compiler, though arguably we
should be able to designate "primary" and "non-primary" inputs on a
per-action basis rather than resorting to "global" state.
Use this new information to stop passing object file inputs to the
Swift frontend, fixing rdar://problem/23213785.
The list wouldn't have to live on the Compilation, but I'm going to use
it to fix SR-280 / rdar://problem/23878192 as well.
...when interpreting. Otherwise, the script may depend on library X, which
depends on library Y, where library Y is a standard Swift library, located
in lib/swift/$PLATFORM/.
Finishes rdar://problem/23588774
That's DYLD_LIBRARY_PATH on OS X and LD_LIBRARY_PATH on Linux for -L,
and DYLD_FRAMEWORK_PATH on OS X for -F.
Note that this commit doesn't actually include the setenv calls yet, so an
end-to-end test is coming in the next commit.
Part of rdar://problem/23588774
This makes it easier to make interpreter modes behave differently from
compilation modes. Obviously that's a trade-off, since the two modes also
share plenty, but given how few of the existing CompileJobAction checks had
to be modified for the new InterpretJobAction, I think this is the right
way to go.
Groundwork for setting [DY]LD_LIBRARY_PATH ahead of time when invoking the
interpreter, which is rdar://problem/23588774.
This reverts commit 58cfa27eb5.
We require at least clang-3.6 for C++14 build support but
Ubuntu 14.04's clang-3.6 package does not lay down the symlink
/usr/bin/clang++ -> /usr/bin/clang++-3.6. We will require a
clang++ alternatives entry or symlink when building for these
systems (see README.md).
In most Linux distributions, installing a clang package other than the
default unversioned one will not install a symlink from /usr/bin/clang++
-> /usr/bin/clang++-N-M, which can break builds with a not so great
diagnostic (a separate problem).
"ld" and "clang++" are hard-coded in the link job actions, so provide a
frontend flag, -linker-path, as a customization point for these.
rdar://problem/23537079
Deploying to older OSs requires linking in a compatibility library called
"arclite", but this library isn't open source and won't be distributed with
our open source downloads. Fall back to the version in Xcode.
The next step is to remove the local symlink used in builds, but I wanted to
handle that separately.
rdar://problem/23421436
inside a swift ast section in an object file so it can be passed to the
linker. The driver automatically wraps merged swiftmodules iff the target
is ELF.
rdar://problem/22407666
Swift SVN r31641
The "Tool" abstraction wasn't buying us enough to deserve the added
complexity. Now a ToolChain turns Actions into Jobs, and every helper
tool is searched for relative to Swift first. Much simpler.
Swift SVN r31563
