[Driver] implement -static-stdlib for Linux
Implement the -static-stdlib driver flag for Linux, allowing the static
linking of the standard library.
This implementation largely follows the Darwin implementation in #1817,
although some pecularities warrant extended discussion.
The original "link with stdlib" implementation had some redundancies
with getRuntimeLibraryPath; these redundancies are resolved, and the
implementation alternates between getRuntimeLibraryPath and
getStaticRuntimeLibraryPath cleanly as appropriate.
A variety of libraries are required to link statically on Linux. The
implementation currently dynamically links with them. We should
probably support static linking of those as well, but I think that is
beyond the scope of a -static-stdlib flag.
The test coverage uses ldd here, as otool is not available on Linux. As
a result, we currently have separate tests for Linux vs the other
platforms; that isn't ideal, but it seems necessary.
Perhaps the oddest part, and the one worth the most discussion, is the
use of --dynamic-list. Inside
stdlib/public/runtime/ProtocolConformances.cpp appears the following
code:
#elif defined(__ELF__)
static int _addImageProtocolConformances(struct dl_phdr_info *info,
size_t size, void *data) {
// inspectArgs contains addImage*Block function and the section name
InspectArgs *inspectArgs = reinterpret_cast<InspectArgs *>(data);
void *handle;
if (!info->dlpi_name || info->dlpi_name[0] == '\0') {
handle = dlopen(nullptr, RTLD_LAZY);
} else
handle = dlopen(info->dlpi_name, RTLD_LAZY | RTLD_NOLOAD);
auto conformances = reinterpret_cast<const uint8_t*>(
dlsym(handle, inspectArgs->sectionName));
The effect of this is to search for protocol_conformances_start inside
the images. However, dlsym only finds symbols that exist in the dynamic
table. Failure to find the protocol conformances can be diagnosed by a
"hello world" program printing
String(_core: Swift._StringCore(_baseAddress: Swift.OpaquePointer(_rawValue: (Opaque Value)), _countAndFlags: Swift.UInt(_value: (Opaque Value)), _owner: Swift.Optional<Swift.AnyObject>.none))
instead of "hello world". (And also by the test coverage in this commit.)
Surprisingly, this behavior can still occur on ELF platforms even if
`objdump -t` reports a valid `.protocol_conformances_start`. This is
because `objdump -t` searches the global table, not the dynamic table,
while dlsym only searches the dynamic table. To configure objdump to
search only the dynamic table, use `-T`.
Inquiring minds may wonder whether dynamically-linked programs (e.g. all
Linux binaries up until now) also have a broken protocol conformance
table on ELF. The answer is, surprisingly, no; I checked, and ordinary
ELF programs are fine. The distinction is probably this, from the ld
manpage:
> the dynamic symbol table will normally contain only those
symbols which are referenced by some dynamic object mentioned in the
link.
I think the linker sees `.protocol_conformances_start` inside
libswiftCore.so and erroneously concludes the one in *the executable* is
"referenced by some dynamic object" (e.g. the standard library). This
behavior seems to save the dyanmically-linked executable from a broken
protocol conformance table. I wonder if it would be wise to apply a
similar fix to dynamically-linked programs to avoid relying on the
linker "helping" us here, but that's out of scope of this commit.
The linker manpage reflects that many people have been bitten by dlsym
"surprise", and encourages the use of `--export-dynamic`:
> If you use "dlopen" to load a dynamic object which needs to refer back
> to the symbols defined by the program, rather than some other dynamic
> object, then you will probably need to use [--export-dynamic] when
> linking the program itself.
However in this situation, the use of `--export-dynamic` causes the
entire stdlib to be exported, which is not ideal. However, by combining
with the `--exclude-libs ALL` argument, we avoid exporting the entire stdlib.
This commit adds the flags -static-stdlib and -no-static-stdlib to
create programs statically linked (or not) with the standard library.
Not is the default, which is also the current behavior.
These flags are currently placebos on non-Darwin platforms.
On the Raspberry Pi 2 when trying to import Glibc, without this patch, it will attempt to
find the module map at "/usr/lib/swift/linux/armv7l/glibc.modulemap" and
fail to do so.
With this patch it will attempt to find the module map at
"/usr/lib/swift/linux/armv7/glibc.modulemap" where it will succeed in
finding the module map.
Similar behavior currently happens in the Driver and Frontend. To DRY up
this behavior it has been extracted to the Swift platform.
This adds an Android target for the stdlib. It is also the first
example of cross-compiling outside of Darwin.
Mailing list discussions:
1. https://lists.swift.org/pipermail/swift-dev/Week-of-Mon-20151207/000171.html
2. https://lists.swift.org/pipermail/swift-dev/Week-of-Mon-20151214/000492.html
The Android variant of Swift may be built using the following `build-script`
invocation:
```
$ utils/build-script \
-R \ # Build in ReleaseAssert mode.
--android \ # Build for Android.
--android-ndk ~/android-ndk-r10e \ # Path to an Android NDK.
--android-ndk-version 21 \
--android-icu-uc ~/libicu-android/armeabi-v7a/libicuuc.so \
--android-icu-uc-include ~/libicu-android/armeabi-v7a/icu/source/common \
--android-icu-i18n ~/libicu-android/armeabi-v7a/libicui18n.so \
--android-icu-i18n-include ~/libicu-android/armeabi-v7a/icu/source/i18n/
```
Android builds have the following dependencies, as can be seen in
the build script invocation:
1. An Android NDK of version 21 or greater, available to download
here: http://developer.android.com/ndk/downloads/index.html.
2. A libicu compatible with android-armv7.
There is currently a great deal of duplication across the
`GenericUnix` and `Windows` toolchains. The Android port will
add even more duplication.
To mitigate this, have `Windows` inherit from `GenericUnix`, and
have them share most of their implementation.
In addition, rename `Windows` to `Cygwin` (it would be pretty strange
to have a `Windows` toolchain inherit from something named `*Unix`).
ASan allows to catch and diagnose memory corruption errors, which are possible
when using unsafe pointers.
This patch introduces a new driver/frontend option -sanitize=address to enable
ASan. When option is passed in, the ASan llvm passes will be turned on and
all functions will gain SanitizeAddress llvm attribute.
(1) We no longer put the Clang version string in our copy of or symlink to
Clang's resource directory.
(2) Newer Clang builds now generate a separate library for the Apple OS
simulators, instead of a fat binary.
We still need a proper end-to-end test for this, but that depends on
building compiler-rt with Swift, which isn't a standard config yet.
Exposes the global warning suppression and treatment as errors
functionality to the Swift driver. Introduces the flags
"-suppress-warnings" and "-warnings-as-errors". Test case include.
This is only the driver side of the work; the frontend doesn't understand
this new -output-filelist option yet. Next commit.
More https://bugs.swift.org/browse/SR-280.
Start sketching out a way for individual jobs to request filelists for
their inputs or their outputs. This should cover all the cases mentioned
in ad945426.
More https://bugs.swift.org/browse/SR-280.
This is simply a newline-separated list of files, matching the format
from Darwin ld's -filelist option.
Part 2 of https://bugs.swift.org/browse/SR-280.
Generate frontend commands with -filelist in them. This isn't actually
implemented yet, but we can start testing at this point.
Part 1 of https://bugs.swift.org/browse/SR-280.
Previously jobs had to grovel this information out of the raw argument
list, which dropped the types we had inferred on input files. This
makes things more consistent across the compiler, though arguably we
should be able to designate "primary" and "non-primary" inputs on a
per-action basis rather than resorting to "global" state.
Use this new information to stop passing object file inputs to the
Swift frontend, fixing rdar://problem/23213785.
The list wouldn't have to live on the Compilation, but I'm going to use
it to fix SR-280 / rdar://problem/23878192 as well.
Also, cluster the flags on Action into a single word. This probably doesn't
make any real difference, but it's the convention.
No functionality change.
The existing code was only recording a change in the /last/ dependency as needing
to rebuild the file. This could have caused things not to get rebuilt when they
should have.
In practice, a number of things have to coincide for this to cause problems:
- A dependency (let's say a type) has to be considered "modified" in this build.
- Some other file has to get rebuilt that didn't previously depend on the type,
or only depended on it in a non-cascading way (e.g. it's only used in function
bodies).
- The file had to not be rebuilt because of any cascading dependencies (or
because it itself changed).
- The file has to now depend on the type in a cascading way, so that downstream
files need to be rebuilt when they didn't before.
Noticed by inspection while fixing the previous issue.
This only shows up when there's a present-but-empty sequence node in
a swiftdeps file. The compiler never generates this, but it's in some of
the tests. I'm not sure why ASan didn't catch this.
This should fix the failures some people have been seeing with
test/Driver/Dependencies/private-after.swift, rdar://problem/23382722.
