This is a follow up to https://github.com/swiftlang/swift/pull/80862, where `storeEnumTagSinglePayload` in a special implementation of `ResultTypeInfo` for Embedded Swift had a mismatching number of arguments. The actual declaration of it in `ABI/ValueWitness.def` clearly includes one more argument.
```
/// void (*storeEnumTagSinglePayload)(T* enum, UINT_TYPE whichCase,
/// UINT_TYPE emptyCases, M *self);
/// Given uninitialized memory for an instance of a single payload enum with a
/// payload of this witness table's type (e.g Optional<ThisType>), store the
/// tag.
FUNCTION_VALUE_WITNESS(storeEnumTagSinglePayload,
StoreEnumTagSinglePayload,
VOID_TYPE,
(MUTABLE_VALUE_TYPE, UINT_TYPE, UINT_TYPE, TYPE_TYPE))
```
This function type mismatch is illegal when targeting Wasm and traps at run time.
Similarly to #80862, we're passing `nullptr` as the newly added argument, which is equivalent to the existing behavior on other platforms.
rdar://157219474
Because `TaskAllocator` is not a round multiple of the machine word
size on 64-bit platforms, I think we end up with padding before the
`TaskLocal::Storage` following it, which makes the `PrivateStorage`
structure larger than the calculation in `ABI/Task.h`.
rdar://149067144
ActiveTaskStatusSize is 2 words, not 4, on most targets.
PrivateStorageSize is measured in bytes, not words, so the Storage field needs to be a char[].
Move to a recursive lock inline in the Task. This avoids the need to allocate a lock record and simplifies the code somewhat.
Change Task's OpaquePrivateStorage to compute its size at build time based on the sizes of its components, rather than having it be a fixed size. It appears that the fixed size was intended to be part of the ABI, but that didn't happen and we're free to change this size. We need to expand it slightly when using pthread_mutex as the recursive lock, as pthread_mutex is pretty big. Other recursive locks allow it to shrink slightly.
We don't have a recursive mutex in our Threading support code, so add a RecursiveMutex type.
rdar://113898653
There cannot be undefined behavior at compile time, so these warnings can be
ignored when `offsetof` is used in a static assert:
```
warning: offset of on non-standard-layout type 'AsyncTask' [-Winvalid-offsetof]
```
When using a future adapter, the resume context may not be valid after the task starts running. Only peer through the adapter when we're starting to run.
rdar://126298035
other threads/tasks which have a reference to the task might need to
access parts of it even if the task itself is completed.
Radar-Id: rdar://problem/88093007
This is done using a condition variable upon which the awaiting thread
will block if the continuation has not be resumed by the point of await.
The resuming thread will signal this condition variable, thereby
unblocking the awaiting thread.
Rdar://99977665
Generated code has never actually initialized this field, so we
might as well remove it. Doing so mostly doesn't impact the ABI
since we don't store anything for arguments or results in the
context as part of the normal call sequence. We do need to adjust
some of the hard-coded contexts, however, such as continuation
contexts and the statically-sized context for special runtime
async functions.
A task can be in one of 4 states over its lifetime:
(a) suspended
(b) enqueued
(c) running
(d) completed
This change provides priority inversion avoidance support if a task gets
escalated when it is in state (a), (c), (d).
Radar-Id: rdar://problem/76127624
thread has the task status record lock.
Today, if a thread is holding the StatusRecordLock, then no other
modification of the task status is possible - including a thread
starting to execute the task or stopping execution of the task.
However, the TaskStatusRecordLock is really about protecting the linked
list that is maintained in the ActiveTaskStatus. As such, other
operations which don't need to look at that linked list of us records
really shouldn't have to block on the StatusRecordLock.
This change allows for concurrent modification of the veTaskStatus while
the TaskStatusRecordLock is held. In particular, a task can cancelled,
escalated, start and stop running, all while another ad is holding onto
the task's StatusRecordLock. In the event of cancellation and
escalation, the task's StatusRecordLock must be n in order to propagate
cancellation and escalation to its child tasks is not needed to cancel
or escalate the task itself.
Radar-Id: rdar://problem/76127624
Each trace point is declared as a function in the new `Tracing.h` header. These functions are called from the appropriate places in the concurrency runtime.
On Darwin, an implementation of these functions is provided which uses the `os/signpost.h` API to emit signpost events/intervals.
When the signpost API is not available, no-op stub implementations are provided. Implementations for other OSes can be provided by providing implementations of the trace functions for that OS.
rdar://81858487
The 32-bit identifier in Job is locked down at this point, so we expand the ID by storing the top 32 bits separately inside AsyncTask::PrivateStorage.
rdar://85167409
Darwin OSes support vouchers, which are key/value sets that can be adopted on a thread to influence its execution, or sent to another process. APIs like Dispatch propagate vouchers to worker threads when running async code. This change makes Swift Concurrency do the same.
The change consists of a few different parts:
1. A set of shims (in VoucherShims.h) which provides declarations for the necessary calls when they're not available from the SDK, and stub implementations for non-Darwin platforms.
2. One of Job's reserved fields is now used to store the voucher associated with a job.
3. Jobs grab the current thread's voucher when they're created.
4. A VoucherManager class manages adoption of vouchers when running a Job, and replacing vouchers in suspended tasks.
5. A VoucherManager instance is maintained in ExecutionTrackingInfo, and is updated as necessary throughout a Job/Task's lifecycle.
rdar://76080222
Tracking this as a single bit is actually largely uninteresting
to the runtime. To handle priority escalation properly, we really
need to track this at a finer grain of detail: recording that the
task is running on a specific thread, enqueued on a specific actor,
or so on. But starting by tracking a single bit is important for
two reasons:
- First, it's more realistic about the performance overheads of
tasks: we're going to be doing this tracking eventually, and
the cost of that tracking will be dominated by the atomic
access, so doing that access now sets the baseline about right.
- Second, it ensures that we've actually got runtime involvement
in all the right places to do this tracking.
A propos of the latter: there was no runtime involvement with
awaiting a continuation, which is a point at which the task
potentially transitions from running to suspended. We must do
the tracking as part of this transition, rather than recognizing
in the run-loops that a task is still active and treating it as
having suspended, because the latter point potentially races with
the resumption of the task. To do this, I've had to introduce
a runtime function, swift_continuation_await, to do this awaiting
rather than inlining the atomic operation on the continuation.
As part of doing this work, I've also fixed a bug where we failed
to load-acquire in swift_task_escalate before walking the task
status records to invoke escalation actions.
I've also fixed several places where the handling of task statuses
may have accidentally allowed the task to revert to uncancelled.
introduce new options parameter to all task spawning
[Concurrency] ABI for asynclet start to accept options
[Concurrency] fix unittest usages of changed task creation ABI
[Concurrency] introduce constants for parameter indexes in ownership
[Concurrency] fix test/SILOptimizer/closure_lifetime_fixup_concurrency.swift
This commit changes JobFlags storage to be 32bits, but leaves the runtime
API expressed in terms of size_t. This allows us to pack an Id in the
32bits we freed up.
The offset of this Id in the AsyncTask is an ABI constant. This way
introspection tools can extract the currently running task identifier
without any need for special APIs.
