Rearrange the slow paths a bit to make them tail calls, which allows the compiler to emit these functions without frames.
Clang is happy to emit frameless functions on ARM64 if no stack space is needed on all execution paths. However, when there's a fast path which doesn't need stack space, and a slow path which does, clang emits code that pushes a stack frame and then decides which path to take. This is fine, but it means we're paying more than we'd like to on the fast path.
We can work around that by manually outlining the slow path, and ensuring that it's invoked with a tail call. Then the original function doesn't need a stack frame on any path and clang omits the stack frame.
We tweak RefCounts::increment to return the object it's being called on, which allows `swift_retain` to tail-call it. We manually outline the objc_retain call in swift_bridgeObjectRetain, which allows the swift_retain path to be frameless.
rdar://101764509
In the general case, the compiler should add ptrauth instructions even
for simple pointer equality checks. We therefore need to manually avoid
these checks; making this workaround permanent.
This replaces a number of `#include`-s like this:
```
#include "../../../stdlib/public/SwiftShims/Visibility.h"
```
with this:
```
#include "swift/shims/Visibility.h"
```
This is needed to allow SwiftCompilerSources to use C++ headers which include SwiftShims headers. Currently trying to do that results in errors:
```
swift/swift/include/swift/Demangling/../../../stdlib/public/SwiftShims/module.modulemap:1:8: error: redefinition of module 'SwiftShims'
module SwiftShims {
^
Builds.noindex/swift/swift/bootstrapping0/lib/swift/shims/module.modulemap:1:8: note: previously defined here
module SwiftShims {
^
```
This happens because the headers in both the source dir and the build dir refer to SwiftShims headers by relative path, and both the source root and the build root contain SwiftShims headers (which are equivalent, but since they are located in different dirs, Clang treats them as different modules).
Moved all the threading code to one place. Added explicit support for
Darwin, Linux, Pthreads, C11 threads and Win32 threads, including new
implementations of Once for Linux, Pthreads, C11 and Win32.
rdar://90776105
SWIFT_STDLIB_SINGLE_THREADED_RUNTIME is too much of a blunt instrument here.
It covers both the Concurrency runtime and the rest of the runtime, but we'd
like to be able to have e.g. a single-threaded Concurrency runtime while
the rest of the runtime is still thread safe (for instance).
So: rename it to SWIFT_STDLIB_SINGLE_THREADED_CONCURRENCY and make it just
control the Concurrency runtime, then add a SWIFT_STDLIB_THREADING_PACKAGE
setting at the CMake/build-script level, which defines
SWIFT_STDLIB_THREADING_xxx where xxx depends on the chosen threading package.
This is especially useful on systems where there may be a choice of threading
package that you could use.
rdar://90776105
Moved all the threading code to one place. Added explicit support for
Darwin, Linux, Pthreads, C11 threads and Win32 threads, including new
implementations of Once for Linux, Pthreads, C11 and Win32.
rdar://90776105
SWIFT_STDLIB_SINGLE_THREADED_RUNTIME is too much of a blunt instrument here.
It covers both the Concurrency runtime and the rest of the runtime, but we'd
like to be able to have e.g. a single-threaded Concurrency runtime while
the rest of the runtime is still thread safe (for instance).
So: rename it to SWIFT_STDLIB_SINGLE_THREADED_CONCURRENCY and make it just
control the Concurrency runtime, then add a SWIFT_STDLIB_THREADING_PACKAGE
setting at the CMake/build-script level, which defines
SWIFT_STDLIB_THREADING_xxx where xxx depends on the chosen threading package.
This is especially useful on systems where there may be a choice of threading
package that you could use.
rdar://90776105
Apply a blanket pass of including `new` for the placement new allocation
and namespacing the call to the global placement new allocator. This
should repair the Android ARMv7 builds.
Starting with Android 11, AArch64 placed a tag in the top byte of pointers to
allocations, which has been slowly rolling out to more devices and collides
with Swift's tags. Moving these tags to the second byte works around this
problem.
state of the actor similar to the ActiveTaskStatus. Refactor the default
actor runtime implementation to set us up for priority escalation
support
Radar-Id: rdar://problem/86100521
The conditional should have been "Intel simulators" but it was actually "x86-64 simulators." The i386 simulator doesn't have the weak formation callout, which causes it to miss cleaning up associated objects when the Swift runtime thinks it does.
rdar://79672466
Added SWIFT_RUNTIME_WEAK_IMPORT/CHECK/USE macros.
Everything supports fast dealloc except x86 iOS simulators, so we no longer need
to look up objc_has_weak_formation_callout.
Added direct references for
objc_setHook_lazyClassNamer
_objc_realizeClassFromSwift
objc_setHook_getClass
os_system_version_get_current_version
_dyld_is_objc_constant
Commit the platform definition and build script work necessary to
cross-compile for arm64_32.
arm64_32 is a variant of AARCH64 that supports an ILP32 architecture.
This gives us faster lookups and a small advantage in memory usage. Most of these maps need stable addresses for their entries, so we add a level of indirection to ConcurrentReadableHashMap for these cases to accommodate that. This costs some extra memory, but it's still a net win.
A new StableAddressConcurrentReadableHashMap type handles this indirection and adds a convenience getOrInsert to take advantage of it.
ConcurrentReadableHashMap is tweaked to avoid any global constructors or destructors when using it as a global variable.
ForeignWitnessTables does not need stable addresses and it now uses ConcurrentReadableHashMap directly.
rdar://problem/70056398
In an assert built of the library, store an extra boolean flag (isImmutable) in the object side-buffer table.
This flag can be set and get by the Array implementation to sanity check the immutability status of the buffer object.
This reduces the dependency on `LLVMSupport`. This is the first step
towards helping move towards a local fork of the LLVM ADT to ensure that
static linking of the Swift runtime and core library does not result in
ODR violations.
This replaces `LLVM_LIBRARY_VISIBILITY` with `SWIFT_LIBRARY_VISIBILTIY`
througout the runtime. The purpose of this attribution is unclear -
building with `-fvisibility=hidden` would accomplish this. This is an
entirely mechanical change replacing the macro with the Swift namespaced
variant instead.
Instruments relies on the old values being there so it can call the original implementation. This has very slightly worse codegen but the impact should be minimal.
Functions like swift_retain call through a function pointer so that Instruments can interpose. This slows down the common case where there is no interposition. Instead, initialize the function pointers to NULL and call through directly to the real implementation when it's NULL. The compiler is smart enough to inline this call and the result is a single conditional branch rather than a function pointer call.
rdar://problem/18307425
The assertions here are based around the idea that `std::atomic` is
trivially constructible which is not a guarantee that the standard fully
provides. The default initialization of the `std::atomic` type may
leave it in an undetermined state. These were caught using the Visual
C++ preview runtime.
Ideally, the object constructor would use a placement new operator.
However, prior to C++17, the C++ standard mandated that there be a
NULL pointer check in the placement new operator. This is something
which is no longer the case with C++17. Switch to the placement new
operator for C++17 and newer and enable that codepath for Windows as
well (which seemingly elides the null-pointer check with clang-cl).
We've been running doxygen with the autobrief option for a couple of
years now. This makes the \brief markers into our comments
redundant. Since they are a visual distraction and we don't want to
encourage more \brief markers in new code either, this patch removes
them all.
Patch produced by
for i in $(git grep -l '\\brief'); do perl -pi -e 's/\\brief //g' $i & done
The Allocations Instrument overrides swift_retain with a function that records the retain count by calling swift_retainCount. Its assert for bits.getIsDeiniting() is incorrect in that case, so remove it.
The recent change to ObjC tagged pointer bits on x86-64 also caused the various bridgeObjectRetain/Release functions to call through to swift_retain for BridgeObject tagged values on Mac. swift_retain ignored those values so there was no functional change, except when Instruments overrode it and passed them to swift_retainCount, which tried to dereference them and crashed. Modify bridgeObjectRetain/Release to bail out early again. Also modify swift_retainCount to ignore those values in case anything else expects retainCount to work on any pointer swift_retain accepts.
rdar://problem/45102538
The functions in LibcShims are used externally, some directly and some through @inlineable functions. These are changed to SWIFT_RUNTIME_STDLIB_SPI to better match their actual usage. Their names are also changed to add "_swift" to the front to match our naming conventions.
Three functions from SwiftObject.mm are changed to SPI and get a _swift prefix.
A few other support functions are also changed to SPI. They already had a prefix and look like they were meant to be SPI anyway. It was just hard to notice any mixup when they were #defined to the same thing.
rdar://problem/35863717
Update the instances of checks for architectures to be more broad for different
spellings of the architecture macro. Certain targets use `_M_ARM` and others
use `__arm__`. Similarly, arm64/aarch64 has `_M_ARM64`, `__arm64__` and
`__aarch64__` as spellings. This just mechanically goes through and encodes the
various spellings.
Take the opportunity to replace some raw checks with `defined` checks which
avoids a pedantic warning due to the undefined macro when performing the check
as the preprocessor may warn about an undefined condition evaluating to `0`.
lldb will use it to reimplement `language swift refcount <obj>`
which is currently not working. Asking the compiler allows us
to avoid maintinaing a bunch of information in the debugger which
are likely to change and break.
<rdar://problem/30538363>
Minimize the generic class metadata template by removing the
class header and base-class members. Add back the set of
information that's really required for instantiation.
Teach swift_allocateGenericClass how to allocate classes without
superclass metadata. Reorder generic initialization to establish
a stronger phase-ordering between allocation (the part that doesn't
really care about the generic arguments) and initialization (the
part that really does care about the generic arguments and therefore
might need to be delayed to handle metadata cycles).
A similar thing needs to happen for resilient class relocation.
