* When ptrauth-copying vtable/wtables, allow NULL entries (due to VFE)
* Mark virtual-function-elimination-generics-exec.swift UNSUPPORTED: arm64e until the rebranch
* Fix test expectations
The current system is based on MetadataCompletionQueueEntry
objects which are allocated and then enqueued on dependencies.
Blocking is achieved using a condition variable associated
with the lock on the appropriate metadata cache. Condition
variables are inherently susceptible to priority inversions
because the waiting threads have no dynamic knowledge of
which thread will notify the condition. In the current system,
threads that unblock dependencies synchronously advance their
dependent metadata completions, which means the signaling
thread is unreliable even if we could represent it in condition
variables. As a result, the current system is wholly unsuited
for eliminating these priority inversions.
An AtomicWaitQueue is an object containing a lock. The queue
is eagerly allocated, and the lock is held, whenever a thread
is doing work that other threads might wish to block on. In
the metadata completion system, this means whenever we construct
a metadata cache entry and the metadata isn't already allocated
and transitively complete after said construction. Blocking
is done by safely acquiring a shared reference to the queue
object (which, in the current implementation, requires briefly
taking a lock that's global to the surrounding metadata cache)
and then acquiring the contained lock. For typical lock
implementations, this avoids priority inversions by temporarily
propagating the priority of waiting threads to the locking
threads.
Dependencies are unblocked by simply releasing the lock held
in the queue. The unblocking thread doesn't know exactly what
metadata are blocked on it and doesn't make any effort to
directly advance their completion; instead, the blocking
thread will wake up and then attempt to advance the dependent
metadata completion itself, eliminating a source of priority
overhang that affected the old system. Successive rounds of
unblocking (e.g. when a metadata makes partial progress but
isn't yet complete) can be achieved by creating a new queue
and unlocking the old one. We can still record dependencies
and use them to dynamically diagnose metadata cycles.
The new system allocates more eagerly than the old one.
Formerly, metadata completions which were never blocked never
needed to allocate a MetadataCompletionQueueEntry; we were
then unable to actually deallocate those entries once they
were allocated. The new system will allocate a queue for
most metadata completions, although, on the positive side,
we can reliably deallocate these queues. Cache entries are
also now slightly smaller because some of the excess storage
for status has been folded into the queue.
The fast path of an actual read of the metadata remains a
simple load-acquire. Slow paths may require a bit more
locking. On Darwin, the metadata cache lock can now use
os_unfair_lock instead of pthread_mutex_t (which is a massive
improvement) because it does not need to support associated
condition variables.
The excess locking could be eliminated with some sort of
generational scheme. Sadly, those are not portable, and I
didn't want to take it on up-front.
rdar://76127798
We remove the existing `swift_reflection_iterateAsyncTaskAllocations` API that attempts to provide all necessary information about a tasks's allocations starting from the task. Instead, we split it into two pieces: `swift_reflection_asyncTaskSlabPointer` to get the first slab for a task, and `+swift_reflection_asyncTaskSlabAllocations` to get the allocations in a slab, and a pointer to the next slab.
We also add a dummy metadata pointer to the beginning of each slab. This allows tools to identify slab allocations on the heap without needing to locate every single async task object. They can then use `swift_reflection_asyncTaskSlabAllocations` on such allocations to find out about the contents.
rdar://82549631
On Windows, std::max_align_t is only 8-byte aligned, but Swift assumes 16-byte alignment. MaximumAlignment is our notion of the maximum alignment of a type, so use that instead.
The latest Long Term Support NDK finally removed binutils, including the bfd/gold
linkers and libgcc. This simplifies our Android support, including making lld the
default linker for Android. Disable three reflection tests that now fail, likely
related to issues with swift-reflection-dump and switching to lld.
We remove the existing `swift_reflection_iterateAsyncTaskAllocations` API that attempts to provide all necessary information about a tasks's allocations starting from the task. Instead, we split it into two pieces: `swift_reflection_asyncTaskSlabPointer` to get the first slab for a task, and `+swift_reflection_asyncTaskSlabAllocations` to get the allocations in a slab, and a pointer to the next slab.
We also add a dummy metadata pointer to the beginning of each slab. This allows tools to identify slab allocations on the heap without needing to locate every single async task object. They can then use `swift_reflection_asyncTaskSlabAllocations` on such allocations to find out about the contents.
rdar://82549631
MetadataAllocator should never return NULL, but bugs or corruption could potentially make that happen. On the large path, switch from malloc to swift_slowAlloc, which aborts on failure. On the pool path, check for a NULL allocation pointer, and log a bunch of information about the allocation request and the allocator's current state.
rdar://84503396
The latest Long Term Support NDK finally removed binutils, including the bfd/gold
linkers and libgcc. This simplifies our Android support, including making lld the
default linker for Android. Disable three reflection tests that now fail, likely
related to issues with swift-reflection-dump and switching to lld.
Also, add the libatomic dependency for Android armv7, just as on linux.
When back-deploying concurrency support, do not use the standard
substitutions for _Concurrency-defined types (such as `Task`) in type
metadata because older Swift runtimes will not be able to demangle
them. Instead, use the full mangled names so the runtime can still
demangle them appropriately.
Addresses rdar://82931890.
Mangling can fail, usually because the Node structure has been built
incorrectly or because something isn't supported with the old remangler.
We shouldn't just terminate the program when that happens, particularly
if it happens because someone has passed bad data to the demangler.
rdar://79725187
If anyone else is building Windows ARM64 they should be using a new
enough Visual Studio. This workaround is more difficult to keep working
properly and the CI hosts should have a new enough Visual Studio
installation hopefully in order to enable the ARM64 builds of the
runtime. If they do not, we can re-evaluate whether to re-instate the
workaround. This allows building part of the runtime with Visual Studio
2022 and reduces the maintenance overheads for the runtime.
