The 32-bit identifier in Job is locked down at this point, so we expand the ID by storing the top 32 bits separately inside AsyncTask::PrivateStorage.
rdar://85167409
If we didn't do this (and we didn't), the tasks get released as we
perform the next() impl, and move the value from the ready task to the
waiting task. Then, the ready task gets destroyed.
But as the task group exists, it performs a cancelAll() and that
iterates over all records. Those records were not removed previously
(!!!) which meant we were pointing at now deallocated tasks.
Previously this worked because we didn't deallocate the tasks, so they
leaked, but we didn't crash. With the memory leak fixed, this began to
crash since we'd attempt to cancel already destroyed tasks.
Solution:
- Remove task records whenever they complete a waiting task.
- This can ONLY be done by the "group owning task" itself, becuause
the contract of ONLY this task being allowed to modify records. o
It MUST NOT be done by the completing tasks as they complete, as it
would race with the owning task modifying this linked list of child
tasks in the group record.
Darwin OSes support vouchers, which are key/value sets that can be adopted on a thread to influence its execution, or sent to another process. APIs like Dispatch propagate vouchers to worker threads when running async code. This change makes Swift Concurrency do the same.
The change consists of a few different parts:
1. A set of shims (in VoucherShims.h) which provides declarations for the necessary calls when they're not available from the SDK, and stub implementations for non-Darwin platforms.
2. One of Job's reserved fields is now used to store the voucher associated with a job.
3. Jobs grab the current thread's voucher when they're created.
4. A VoucherManager class manages adoption of vouchers when running a Job, and replacing vouchers in suspended tasks.
5. A VoucherManager instance is maintained in ExecutionTrackingInfo, and is updated as necessary throughout a Job/Task's lifecycle.
rdar://76080222
Change the code generation patterns for `async let` bindings to use an ABI based on the following
functions:
- `swift_asyncLet_begin`, which starts an `async let` child task, but which additionally
now associates the `async let` with a caller-owned buffer to receive the result of the task.
This is intended to allow the task to emplace its result in caller-owned memory, allowing the
child task to be deallocated after completion without invalidating the result buffer.
- `swift_asyncLet_get[_throwing]`, which replaces `swift_asyncLet_wait[_throwing]`. Instead of
returning a copy of the value, this entry point concerns itself with populating the local buffer.
If the buffer hasn't been populated, then it awaits completion of the task and emplaces the
result in the buffer; otherwise, it simply returns. The caller can then read the result out of
its owned memory. These entry points are intended to be used before every read from the
`async let` binding, after which point the local buffer is guaranteed to contain an initialized
value.
- `swift_asyncLet_finish`, which replaces `swift_asyncLet_end`. Unlike `_end`, this variant
is async and will suspend the parent task after cancelling the child to ensure it finishes
before cleaning up. The local buffer will also be deinitialized if necessary. This is intended
to be used on exit from an `async let` scope, to handle cleaning up the local buffer if necessary
as well as cancelling, awaiting, and deallocating the child task.
- `swift_asyncLet_consume[_throwing]`, which combines `get` and `finish`. This will await completion
of the task, leaving the result value in the result buffer (or propagating the error, if it
throws), while destroying and deallocating the child task. This is intended as an optimization
for reading `async let` variables that are read exactly once by their parent task.
To avoid an epoch break with existing swiftinterfaces and ABI clients, the old builtins and entry
points are kept intact for now, but SILGen now only generates code using the new interface.
This new interface fixes several issues with the old async let codegen, including use-after-free
crashes if the `async let` was never awaited, and the inability to read from an `async let` variable
more than once.
rdar://77855176
The original async ABI made callees deallocate the context,
which allows tail calls (at the async-function level) but
interferes with callers' ability to optimize callee frame
allocation. The purpose of this bit was to allow callers
to do that optimization, but we've since just made callers
responsible for deallocating the context, which is overall
just a lot simpler. So this has been dead for quite some
time.
Tracking this as a single bit is actually largely uninteresting
to the runtime. To handle priority escalation properly, we really
need to track this at a finer grain of detail: recording that the
task is running on a specific thread, enqueued on a specific actor,
or so on. But starting by tracking a single bit is important for
two reasons:
- First, it's more realistic about the performance overheads of
tasks: we're going to be doing this tracking eventually, and
the cost of that tracking will be dominated by the atomic
access, so doing that access now sets the baseline about right.
- Second, it ensures that we've actually got runtime involvement
in all the right places to do this tracking.
A propos of the latter: there was no runtime involvement with
awaiting a continuation, which is a point at which the task
potentially transitions from running to suspended. We must do
the tracking as part of this transition, rather than recognizing
in the run-loops that a task is still active and treating it as
having suspended, because the latter point potentially races with
the resumption of the task. To do this, I've had to introduce
a runtime function, swift_continuation_await, to do this awaiting
rather than inlining the atomic operation on the continuation.
As part of doing this work, I've also fixed a bug where we failed
to load-acquire in swift_task_escalate before walking the task
status records to invoke escalation actions.
I've also fixed several places where the handling of task statuses
may have accidentally allowed the task to revert to uncancelled.
Implement a version of addImage that takes in a closure responsible
for finding the sections of interest. This allows for the registration
of metadata even in situations where the object file is not complete
(for example, when generated by the JIT).
The flags that are useful for task creation are a bit different from
the flags that go on a job. Create a separate flag set for task
creation and use that in the API for `swift_task_create`. For now,
have the callers do the remapping.
Collapse the `group` parameter of this API into the task options, and
have existing callers set up the options appropriately. The goal for
this function is to become the centralized entry point for all task
creation, with an extensible interface.
ExecutorRefs are a (pointer, witness table) pair that are meant to be
passed around. We don't need to form a reference to one because
they are ABI already.
introduce new options parameter to all task spawning
[Concurrency] ABI for asynclet start to accept options
[Concurrency] fix unittest usages of changed task creation ABI
[Concurrency] introduce constants for parameter indexes in ownership
[Concurrency] fix test/SILOptimizer/closure_lifetime_fixup_concurrency.swift
I added Builtin.buildMainActorExecutor before, but because I never
implemented it correctly in IRGen, it's not okay to use it on old
versions, so I had to introduce a new feature only for it.
The shim dispatch queue class in the Concurrency runtime is rather
awful, but I couldn't think of a reasonable alternative without
just entirely hard-coding the witness table in the runtime.
It's not ABI, at least.
Changes the task, taskGroup, asyncLet wait funtion call ABIs.
To reduce code size pass the context parameters and resumption function
as arguments to the wait function.
This means that the suspend point does not need to store parent context
and resumption to the suspend point's context.
```
void swift_task_future_wait_throwing(
OpaqueValue * result,
SWIFT_ASYNC_CONTEXT AsyncContext *callerContext,
AsyncTask *task,
ThrowingTaskFutureWaitContinuationFunction *resume,
AsyncContext *callContext);
```
The runtime passes the caller context to the resume entry point saving
the load of the parent context in the resumption function.
This patch adds a `Metadata *` field to `GroupImpl`. The await entry
pointer no longer pass the metadata pointer and there is a path through
the runtime where the task future is no longer available.
Isolated parameters are part of function types. Encode them in function
type manglings and metadata, and ensure that they round-trip through
the various mangling and metadata facilities. This nails down the ABI
for isolated parameters.
Implement name mangling, type metadata, runtime demangling, etc. for
global-actor qualified function types. Ensure that the manglings
round-trip through the various subsystems.
Implements rdar://78269642.
This commit changes JobFlags storage to be 32bits, but leaves the runtime
API expressed in terms of size_t. This allows us to pack an Id in the
32bits we freed up.
The offset of this Id in the AsyncTask is an ABI constant. This way
introspection tools can extract the currently running task identifier
without any need for special APIs.
* [Concurrency] Reduce overhead of Task.yield and Task.sleep
Instead of creating a new task, we create a simple job that wraps a Builtin.RawUnsafeContinuation and resumes the continuation when it is executed. The job instance is allocated on the task local allocator, meaning we don't malloc anything.
* Update stdlib/public/Concurrency/Task.swift
Co-authored-by: Konrad `ktoso` Malawski <konrad.malawski@project13.pl>
Co-authored-by: Konrad `ktoso` Malawski <konrad.malawski@project13.pl>
It seems that MSVC 2017 trips parsing an using of a using of a variadic
template. Removing one level of using seems to work fine. A preprocessor
macro allows to keep using the same syntax in both MSVC 2017 and other
compilers without making a lot of a mess.
I think this might have been uncovered by landing apple/llvm-project#2898
when it was picked up by the auto-merger for the swift/main branch.
I think this was not a problem until now, because Metadata.h was
compiled using the just compiled Clang until now. LLDB is compiled using
MSVC in Windows.
Previously, they were storing a low-bit flag that indicated that they
were a default actor. Using an extra inhabitant frees up the low bit
for future use without being conspicuously more expensive to check.
