I had set up the driver to invoke a separate frontend invocation with
the "update code" mode. We sort of did this last release, except we
forked to the swift-update binary instead. This is causing problems with
testing in Xcode.
Instead, let's perform a single compile and add the remap file as an
additional output during normal compiles. The driver, seeing
-update-code, will add -emit-remap-file-path $PATH to the -c frontend
invocation.
rdar://problem/31857580
The Swift 4 Migrator is invoked through either the driver and frontend
with the -update-code flag.
The basic pipeline in the frontend is:
- Perform some list of syntactic fixes (there are currently none).
- Perform N rounds of sema fix-its on the primary input file, currently
set to 7 based on prior migrator seasons. Right now, this is just set
to take any fix-it suggested by the compiler.
- Emit a replacement map file, a JSON file describing replacements to a
file that Xcode knows how to understand.
Currently, the Migrator maintains a history of migration states along
the way for debugging purposes.
- Add -emit-remap frontend option
This will indicate the EmitRemap frontend action.
- Don't fork to a separte swift-update binary.
This is going to be a mode of the compiler, invoked by the same flags.
- Add -disable-migrator-fixits option
Useful for debugging, this skips the phase in the Migrator that
automatically applies fix-its suggested by the compiler.
- Add -emit-migrated-file-path option
This is used for testing/debugging scenarios. This takes the final
migration state's output text and writes it to the file specified
by this option.
- Add -dump-migration-states-dir
This dumps all of the migration states encountered during a migration
run for a file to the given directory. For example, the compiler
fix-it migration pass dumps the input file, the output file, and the
remap file between the two.
State output has the following naming convention:
${Index}-${MigrationPassName}-${What}.${extension}, such as:
1-FixitMigrationState-Input.swift
rdar://problem/30926261
Track the types we've seen instead of the type declarations we've
passed through, which eliminates some holes relating to generic types.
Detect infinite expansions by imposing an arbitrary limit.
Fixes rdar://30355804
The `-warn-swift3-objc-inference` option turns out to be extremely
useful in vetting code for unintended `@objc` entry points, so make it
available directly on `swiftc`.
But, bury the enable/disable flags under `-frontend` (they were
effectively there anyway because the driver wasn't propagating them).
Add an -enforce-exclusivity=... flag to control enforcement of the law of
exclusivity. The flag takes one of four options:
"checked": Perform both static (compile-time) and dynamic (run-time) checks.
"unchecked": Perform only static enforcement. This is analogous to -Ounchecked.
"dynamic-only": Perform only dynamic checks. This is for staging purposes.
"none": Perform no checks at all. This is also for staging purposes.
The default, for now, is "none".
The intent is that in the fullness of time, "checked" and "unchecked" will
be the only legal options with "checked" the default. That is, static
enforcement will always be enabled and dynamic enforcement will be enabled
by default.
Add a -verify-debug-info option that invokes dwarfdump --verify as the last step after running dsymutil. dwarfdump is invoked with same options clang 802.0.35 uses to invoke it:
dwarfdump --verify --debug-info --eh-frame --quiet
A warning is produced if -verify-debug-info is set and no debug option is also set.
dwarfdump is failing to validate the debug info in the test verify-debug-info.swift. The failure is:
error: .debug_line[0x0000007d].row[0].file = 1 is not a valid index
https://bugs.swift.org/browse/SR-2396
This is purely designed to cheaply compute dependency graphs between
modules, and thus only lists the top-level names (i.e. not submodules)
and doesn't do any form of semantic analysis.
Some projects may be setup in a way where the order of '-F' flags is significant and changing the order, by turning some
of them to '-Fsystem', can break them.
This has the effect of propagating the search path to the clang importer as '-iframework'.
It doesn't affect whether a swift module is treated as system or not, this can be done as follow-up enhancement.
- Create separate swift_begin.o/swift_end.o for lib/swift and
lib/swift_static. The static swift_begin.o does not call
swift_addNewDSOImage() at startup.
- Update ToolChains.cpp to use the correct swift_begin.o/swift_end.o
files for the `-static-stdlib` and `-static-executable` options.
LLDB will automatically pick the host OS if no target is passed; a
later commit will teach immediate mode to do the same thing. For now,
they default to the same triple the Driver did in the past, which is
x86_64-apple-macosx10.9 on macOS and an arbitrary unversioned triple
compatible with the host elsewhere.
Part of rdar://problem/29433205.
Changes:
* Terminate all namespaces with the correct closing comment.
* Make sure argument names in comments match the corresponding parameter name.
* Remove redundant get() calls on smart pointers.
* Prefer using "override" or "final" instead of "virtual". Remove "virtual" where appropriate.
- Add ImageInspectionStatic.cpp to lookup protocol conformance
and metadata sections in static binaries
- For Linux, build libswiftImageInspectionShared.a and
libswiftImageInspectionStatic.a for linking with libswiftCore.a.
This allows static binaries to be built without linking to
libdl. libswiftImageInspectionShared (ImageInspectionELF.cpp) is
automatically compiled into libswiftCore.so
- Adds -static-executable option to swiftc to use along with
-emit-executable that uses linker arguments in
static-executable-args.lnk. This also requires a libicu
to be compiled using the --libicu which has configure options
that dont require libdl for accessing ICU datafiles
- Static binaries only work on Linux at this time
-modulewrap invocations create an object file.
The target should be passed along so that the object file is created for the same target as any other outputs.
This flag switches the "effective language version" of the compiler,
at least to any version supported (as of this change: "3" or "3.0").
At the moment nothing uses it except the language version build
configuration statements (#if swift(...)) and various other places
that report, encode, or otherwise check version numbers.
In the future, it's intended as scaffolding for backwards compatibility.
Fixes SR-2582
Background
----------
Now that Swift AST type support in LLDB has matured, we can stop emitting DWARF
type information by default to reduce compile time and ibject file size.
A future commit will change -g to emit only AST type references.
The full set of debug options will be
-gnone
-gline-tables-only
-g // AST types (= everything that LLDB needs)
-gdwarf-types // AST types + DWARF types (for legacy debuggers)
(macOS? OS X? How does this work for past OSs?)
Noticed by inspection. Xcode doesn't use swiftc to link, and the few
things that went into arclite between iOS 7 and iOS 8 weren't critical,
but we should still get this right.
"Sanitizer Coverage" with a new flag ``-sanitize-coverage=``. This
flag is analogous to Clang's ``-fsanitize-coverage=``.
This instrumentation currently requires ASan or TSan to be enabled
because the module pass created by ``createSanitizerCoverageModulePass()``
inserts calls into functions found in compiler-rt's "sanitizer_common".
"sanitizer_common" is not shipped as an individual library but instead
exists in several of the sanitizer runtime libraries so we have to
link with one of them to avoid linking errors.
The rationale between adding this feature is to allow experimentation
with libFuzzer which currently relies on "Sanitizer Coverage"
instrumentation.
[Driver] implement -static-stdlib for Linux
Implement the -static-stdlib driver flag for Linux, allowing the static
linking of the standard library.
This implementation largely follows the Darwin implementation in #1817,
although some pecularities warrant extended discussion.
The original "link with stdlib" implementation had some redundancies
with getRuntimeLibraryPath; these redundancies are resolved, and the
implementation alternates between getRuntimeLibraryPath and
getStaticRuntimeLibraryPath cleanly as appropriate.
A variety of libraries are required to link statically on Linux. The
implementation currently dynamically links with them. We should
probably support static linking of those as well, but I think that is
beyond the scope of a -static-stdlib flag.
The test coverage uses ldd here, as otool is not available on Linux. As
a result, we currently have separate tests for Linux vs the other
platforms; that isn't ideal, but it seems necessary.
Perhaps the oddest part, and the one worth the most discussion, is the
use of --dynamic-list. Inside
stdlib/public/runtime/ProtocolConformances.cpp appears the following
code:
#elif defined(__ELF__)
static int _addImageProtocolConformances(struct dl_phdr_info *info,
size_t size, void *data) {
// inspectArgs contains addImage*Block function and the section name
InspectArgs *inspectArgs = reinterpret_cast<InspectArgs *>(data);
void *handle;
if (!info->dlpi_name || info->dlpi_name[0] == '\0') {
handle = dlopen(nullptr, RTLD_LAZY);
} else
handle = dlopen(info->dlpi_name, RTLD_LAZY | RTLD_NOLOAD);
auto conformances = reinterpret_cast<const uint8_t*>(
dlsym(handle, inspectArgs->sectionName));
The effect of this is to search for protocol_conformances_start inside
the images. However, dlsym only finds symbols that exist in the dynamic
table. Failure to find the protocol conformances can be diagnosed by a
"hello world" program printing
String(_core: Swift._StringCore(_baseAddress: Swift.OpaquePointer(_rawValue: (Opaque Value)), _countAndFlags: Swift.UInt(_value: (Opaque Value)), _owner: Swift.Optional<Swift.AnyObject>.none))
instead of "hello world". (And also by the test coverage in this commit.)
Surprisingly, this behavior can still occur on ELF platforms even if
`objdump -t` reports a valid `.protocol_conformances_start`. This is
because `objdump -t` searches the global table, not the dynamic table,
while dlsym only searches the dynamic table. To configure objdump to
search only the dynamic table, use `-T`.
Inquiring minds may wonder whether dynamically-linked programs (e.g. all
Linux binaries up until now) also have a broken protocol conformance
table on ELF. The answer is, surprisingly, no; I checked, and ordinary
ELF programs are fine. The distinction is probably this, from the ld
manpage:
> the dynamic symbol table will normally contain only those
symbols which are referenced by some dynamic object mentioned in the
link.
I think the linker sees `.protocol_conformances_start` inside
libswiftCore.so and erroneously concludes the one in *the executable* is
"referenced by some dynamic object" (e.g. the standard library). This
behavior seems to save the dyanmically-linked executable from a broken
protocol conformance table. I wonder if it would be wise to apply a
similar fix to dynamically-linked programs to avoid relying on the
linker "helping" us here, but that's out of scope of this commit.
The linker manpage reflects that many people have been bitten by dlsym
"surprise", and encourages the use of `--export-dynamic`:
> If you use "dlopen" to load a dynamic object which needs to refer back
> to the symbols defined by the program, rather than some other dynamic
> object, then you will probably need to use [--export-dynamic] when
> linking the program itself.
However in this situation, the use of `--export-dynamic` causes the
entire stdlib to be exported, which is not ideal. However, by combining
with the `--exclude-libs ALL` argument, we avoid exporting the entire stdlib.
This commit adds the flags -static-stdlib and -no-static-stdlib to
create programs statically linked (or not) with the standard library.
Not is the default, which is also the current behavior.
These flags are currently placebos on non-Darwin platforms.
On the Raspberry Pi 2 when trying to import Glibc, without this patch, it will attempt to
find the module map at "/usr/lib/swift/linux/armv7l/glibc.modulemap" and
fail to do so.
With this patch it will attempt to find the module map at
"/usr/lib/swift/linux/armv7/glibc.modulemap" where it will succeed in
finding the module map.
Similar behavior currently happens in the Driver and Frontend. To DRY up
this behavior it has been extracted to the Swift platform.
This adds an Android target for the stdlib. It is also the first
example of cross-compiling outside of Darwin.
Mailing list discussions:
1. https://lists.swift.org/pipermail/swift-dev/Week-of-Mon-20151207/000171.html
2. https://lists.swift.org/pipermail/swift-dev/Week-of-Mon-20151214/000492.html
The Android variant of Swift may be built using the following `build-script`
invocation:
```
$ utils/build-script \
-R \ # Build in ReleaseAssert mode.
--android \ # Build for Android.
--android-ndk ~/android-ndk-r10e \ # Path to an Android NDK.
--android-ndk-version 21 \
--android-icu-uc ~/libicu-android/armeabi-v7a/libicuuc.so \
--android-icu-uc-include ~/libicu-android/armeabi-v7a/icu/source/common \
--android-icu-i18n ~/libicu-android/armeabi-v7a/libicui18n.so \
--android-icu-i18n-include ~/libicu-android/armeabi-v7a/icu/source/i18n/
```
Android builds have the following dependencies, as can be seen in
the build script invocation:
1. An Android NDK of version 21 or greater, available to download
here: http://developer.android.com/ndk/downloads/index.html.
2. A libicu compatible with android-armv7.
There is currently a great deal of duplication across the
`GenericUnix` and `Windows` toolchains. The Android port will
add even more duplication.
To mitigate this, have `Windows` inherit from `GenericUnix`, and
have them share most of their implementation.
In addition, rename `Windows` to `Cygwin` (it would be pretty strange
to have a `Windows` toolchain inherit from something named `*Unix`).
