This PR introduces three new instrumentation flags and plumbs them
through to IRGen:
1. `-ir-profile-generate` - enable IR-level instrumentation.
2. `-cs-profile-generate` - enable context-sensitive IR-level
instrumentation.
3. `-ir-profile-use` - IR-level PGO input profdata file to enable
profile-guided optimization (both IRPGO and CSIRPGO)
**Context:**
https://forums.swift.org/t/ir-level-pgo-instrumentation-in-swift/82123
**Swift-driver PR:** https://github.com/swiftlang/swift-driver/pull/1992
**Tests and validation:**
This PR includes ir level verification tests, also checks few edge-cases
when `-ir-profile-use` supplied profile is either missing or is an
invalid IR profile.
However, for argument validation, linking, and generating IR profiles
that can later be consumed by -cs-profile-generate, I’ll need
corresponding swift-driver changes. Those changes are being tracked in
https://github.com/swiftlang/swift-driver/pull/1992
This is the C++ driver counterpart to a change that landed in the Swift
driver a while ago to use the clang-linker to determine what the default
linker is. This is to avoid hard-coding gold, which is deprecated and
not available on some newer Linux distributions. The challenge is that
these newer Linux distributions don't already have Swift so we have to
use the old C++ driver implementation.
To work-around #80059, we need to stop return address signing and
opt-out of BTCFI enforcement via enabling a platform linker option.
We don't want to completely undo the BTCFI work in the rare case that
we later figure out how to properly address the above issue, or allow
users who might want to benefit from BTCFI enforcement and won't use
Concurrency. To do this, condition the existing BTCFI flag enforcement
into a configuration option that defaults to off for now.
Because the new swift-driver needs to "know" whether the frontend is
configured to opt-out or not, and since the new driver communicates with
the frontend via the target info JSON to begin with, we add a field
that emits the build flavor to signal the right behavior.
This is required to bootstrap the `-static-stdlib` support for Windows.
With this, we are able to properly build the Swift SDK both dynamically
and statically, which is needed to enable us to make further progress
towards an early swift-driver.
This flag was added back in 2020, but it didn't function properly, since a lot of other code in the compiler assumed the platform-default C++ stdlib until recently (https://github.com/swiftlang/swift/pull/75589).
The recommended way to use a non-default C++ stdlib in Swift now is to pass `-Xcc -stdlib=xyz` argument to the compiler.
This change removes the `-experimental-cxx-stdlib` flag.
PR #73725 introduced the in-process plugin server library, but the
selection of the library depends on the selected toolchain, which
depends on the compiler target, not the host. When cross-compiling (for
example from macOS to a embedded Unix target), the compiler will
incorrectly chose the `.so` file, not find it, and fail to compile
things like the `@debugDescription` macro.
Move the in-process plugin server library code from the platform
toolchains into the parent type, and code it so it uses the right name
depending on the compiler host at compilation time. This discards the
target and only relies on the compiler host for selecting the right
library.
Separate swift-syntax libs for the compiler and for the library plugins.
Compiler communicates with library plugins using serialized messages
just like executable plugins.
* `lib/swift/host/compiler/lib_Compiler*.dylib`(`lib/CompilerSwiftSyntax`):
swift-syntax libraries for compiler. Library evolution is disabled.
* Compiler (`ASTGen` and `swiftIDEUtilsBridging`) only depends on
`lib/swift/host/compiler` libraries.
* `SwiftInProcPluginServer`: In-process plugin server shared library.
This has one `swift_inproc_plugins_handle_message` entry point that
receives a message and return the response.
* In the compiler
* Add `-in-process-plugin-server-path` front-end option, which specifies
the `SwiftInProcPluginServer` shared library path.
* Remove `LoadedLibraryPlugin`, because all library plugins are managed
by `SwiftInProcPluginServer`
* Introduce abstract `CompilerPlugin` class that has 2 subclasses:
* `LoadedExecutablePlugin` existing class that represents an
executable plugin
* `InProcessPlugins` wraps `dlopen`ed `SwiftInProcPluginServer`
* Unified the code path in `TypeCheckMacros.cpp` and `ASTGen`, the
difference between executable plugins and library plugins are now
abstracted by `CompilerPlugin`
Although I don't plan to bring over new assertions wholesale
into the current qualification branch, it's entirely possible
that various minor changes in main will use the new assertions;
having this basic support in the release branch will simplify that.
(This is why I'm adding the includes as a separate pass from
rewriting the individual assertions)
We changed to `llvm.compiler.used` because of the behaviour of `gold`,
which refuses to coalesce sections that have different `SHF_GNU_RETAIN`
flags, which causes problems with metadata.
Originally I thought we were going to have to generate two sections
with distinct names and have the runtime look for both of them, but
it turns out that the runtime only wants to see sections that have
`SHF_GNU_RETAIN` in any case. It's really the reflection code that
is interested in being able to see non-retained sections. The upshot
is that we don't need to use `llvm.compiler.used`; it's just fine if
we have duplicate sections, as long as the reflection code looks for
them when it's inspecting an ELF image.
This also means we no longer need to pass `-z nostart-stop-gc` to the
linker if we're using `lld`.
rdar://123504095
The plugin layouts are different across platforms. Move this into a
virtual method and allow replacement. On Windows, the plugins are
placed into the `bin` directory as the DLLs should always be co-located
to ensure that the proper DLLs are found (there is no concept of RPATH).
Fixes#60406
Ran the linux x86_64 compiler validation suite on this pull with `-use-ld=lld -Xlinker --gc-sections`, then without stripping as in the linked issue, and only saw the 20 unrelated failures just like @drodriguez had.
The latest Long Term Support NDK finally removed binutils, including the bfd/gold
linkers and libgcc. This simplifies our Android support, including making lld the
default linker for Android. Disable three reflection tests that now fail, likely
related to issues with swift-reflection-dump and switching to lld.
llvm-project `ErrorHandling.h` was updated to remove std::string. This
added a new `report_fatal_error` overload taking a `const Twine &`,
removed the overload that took `const std::string &`, and updated
`fatal_error_handler_t` to use `const char *` rather than `const
std::string &`.
Fix uses of these functions to take into account these updates. Note
that without the `const std::string &` overload, passing a `std::string`
into `report_fatal_error` now results in an ambiguous match between the
`StringRef` and `Twine` overloads so we need to be explicit about one or
the other.
The latest Long Term Support NDK finally removed binutils, including the bfd/gold
linkers and libgcc. This simplifies our Android support, including making lld the
default linker for Android. Disable three reflection tests that now fail, likely
related to issues with swift-reflection-dump and switching to lld.
Also, add the libatomic dependency for Android armv7, just as on linux.
Commit the platform definition and build script work necessary to
cross-compile for arm64_32.
arm64_32 is a variant of AARCH64 that supports an ILP32 architecture.
The target was added for Unix toolchains in #901, but a later pull #1891 added
it again. Since clang only uses the last target flag that's passed in, all
customization done for the first one was unused these last 4+ years, so remove
it and change tests that look for custom strings passed by the first one.
This commit adds LTO support for handling linker options and LLVM BC
emission. Even for ELF, swift-autolink-extract is unnecessary because
linker options are embeded in LLVM BC content when LTO.
This doesn't yet allow including C++ headers on platforms where libc++
isn't the default; see comments in UnixToolChains.cpp for details.
However, it does, for example, allow throwing and catching exceptions in C++
code used through interop, unblocking
https://github.com/apple/swift/pull/30674/files.
The flags (-enable-experimental-cxx-interop and -experimental-cxx-stdlib) carry
"experimental" in the name to emphasize that C++ interop is still an
experimental feature.
Co-authored-by: Michael Forster <forster@google.com>
This commit adds -lto flag for driver to enable LTO at LLVM level.
When -lto=llvm given, compiler emits LLVM bitcode file instead of object
file and perform thin LTO using libLTO.dylib plugin.
When -lto=llvm-full given, perform full LTO instead of thin LTO.
Handle `-no-toolchain-stdlib-rpath` on Linux/android as well as Darwin. This
enables the flags on other Unix platforms as it can be useful to control the
embedded rpath for the library.
Add the platform conditional and set up other basics for the toolchain.
The ConditionalCompilation tests are updated to match, since otherwise
they seem to trip when building on non-OpenBSD platforms. The
Driver/linker test is updated to ensure lld is passed on this platform.
Note that OpenBSD calls "x86_64" as "amd64", so we use that name for the
architecture instead of trying to alias one to the other, as this makes
things simpler.
Now that CMAKE_HOST_SYSTEM_NAME and CMAKE_SYSTEM_NAME are set by default to
Android in the Termux app, make the needed tweaks. Some tests were adapted
to work natively on Android too, adds sys/cdefs.h to the Bionic modulemap,
and includes the start of native Android platform support in the build-script.
Currently we only support building for android armv7, arm64, x86,
x86_64. In the future, if support for MIPS and MIPS64 is added, we
should normalise those as well. This is needed to support compilation
against modern NDKs.
Use `clang` rather than `clang++` as the linker driver. This ensures
that we do not force a C++ runtime on the general code. This is fine
for now as C++ interop is not yet available for Swift. This prevents
the accidental mix-and-match of various C++ runtimes. This can cause
problems on platforms like android where `libstdc++` is an unsupported
runtime but is generally the default for Linux platforms.
Previously, the TSan runtime only required libdispatch on Darwin, which
required no explicit linker flags, because libdispatch is always
provided by the system (libSystem).
Now, the TSan runtime also has a link-time dependency on libdispatch on
Linux, where libdispatch (and the blocks runtime) is just another
library. We therefore need to specify them as additonal linker options.
