Due to malloc quanta rounding, 1024-byte async task allocation slabs
actually end up allocating 1536 bytes on Darwin. Instead, use 1000-byte
slabs.
Fixes rdar://81181856.
The actor runtime has some known issues with deadlock when an actor has
to give up its thread because it's running lower-priority work. To
avoid deadlocks here, disable all of the logic that tries to give up
higher-priority threads when only lower-priority work is available, or
to escalate work, effectively making the actor runtime ignore
priorities internally.
Fixes rdar://79378762.
Due to a missing `~` when trying to mask in a new priority + the
`IsEscalated` flag, we were instead getting an incorrect priority as
well as dropping other useful bits. This led to assertions about the
running state of a task not being set.
Without this fix, we saw a crash involving URLSession where we'd hit
assertions or corruption in the runtime in various forms:
- Assertion on trying to enqueue work onto a zombie actor.
- job->SchedulerPrivate being overwritten (crash in getNextJobInQueue).
- *(job->SchedulerPrivate) being overwritten (crash in getAsPreprocessedJob
right after call to getNextJobInQueue).
I haven't been able to create a minimal crashing example yet, but manual
testing shows this does fix the issue.
Fixes rdar://79859254.
The implemented semantics are that:
1. Tasks have separate exclusivity access sets.
2. Any synchronous context that creates tasks will have its exclusive access set
merged into the Tasks while the Task is running.
rdar://80492364
I am going to use this to test that we propagate synchronous accesses into
asynchronous tasks access sets.
To ensure this is not ABI, I underscored/marked this as alwaysEmitIntoClient.
Change the code generation patterns for `async let` bindings to use an ABI based on the following
functions:
- `swift_asyncLet_begin`, which starts an `async let` child task, but which additionally
now associates the `async let` with a caller-owned buffer to receive the result of the task.
This is intended to allow the task to emplace its result in caller-owned memory, allowing the
child task to be deallocated after completion without invalidating the result buffer.
- `swift_asyncLet_get[_throwing]`, which replaces `swift_asyncLet_wait[_throwing]`. Instead of
returning a copy of the value, this entry point concerns itself with populating the local buffer.
If the buffer hasn't been populated, then it awaits completion of the task and emplaces the
result in the buffer; otherwise, it simply returns. The caller can then read the result out of
its owned memory. These entry points are intended to be used before every read from the
`async let` binding, after which point the local buffer is guaranteed to contain an initialized
value.
- `swift_asyncLet_finish`, which replaces `swift_asyncLet_end`. Unlike `_end`, this variant
is async and will suspend the parent task after cancelling the child to ensure it finishes
before cleaning up. The local buffer will also be deinitialized if necessary. This is intended
to be used on exit from an `async let` scope, to handle cleaning up the local buffer if necessary
as well as cancelling, awaiting, and deallocating the child task.
- `swift_asyncLet_consume[_throwing]`, which combines `get` and `finish`. This will await completion
of the task, leaving the result value in the result buffer (or propagating the error, if it
throws), while destroying and deallocating the child task. This is intended as an optimization
for reading `async let` variables that are read exactly once by their parent task.
To avoid an epoch break with existing swiftinterfaces and ABI clients, the old builtins and entry
points are kept intact for now, but SILGen now only generates code using the new interface.
This new interface fixes several issues with the old async let codegen, including use-after-free
crashes if the `async let` was never awaited, and the inability to read from an `async let` variable
more than once.
rdar://77855176
Introduce a fake (but non-ABI) declaration to the swiftinterface
which marks that an SDK support swift_continuation_await, and then
only call it if that declaration exists, otherwise falling back
on the old atomic sequence. Using that sequence will badly mess
up the runtime's tracking of task state, but it might still work,
and more importantly things will still build, which solves the
short-term problem. Hopefully we can remove this hack soon.
Fixes rdar://problem/80787731.
We mostly get away without this because we're fairly disciplined
about using constant memory orderings, and apparently that's
usually good enough to get inline accesses and avoid needing to
link atomic. However, we have a few places with the task status
atomic that use a non-constant load ordering with load and
compare_exchange_weak, and my recent change to make that atomic
a double-word was apparently sufficient on some (but not all)
Linux distributions to get the compiler to call the runtime
function. Regardless, we shouldn't be playing around in the
margins here: Linux requires us to link libatomic, so we should.
See rdar://79378762, SR-14802, SR-14841, SR-14875.
This doesn't resolve all hangs, such as those occurring
due to explicit usage of priorities and certain other
situations where priorities seem to be causing issues
(rdar://79823345), but it does resolve some cases.
The original async ABI made callees deallocate the context,
which allows tail calls (at the async-function level) but
interferes with callers' ability to optimize callee frame
allocation. The purpose of this bit was to allow callers
to do that optimization, but we've since just made callers
responsible for deallocating the context, which is overall
just a lot simpler. So this has been dead for quite some
time.
Tracking this as a single bit is actually largely uninteresting
to the runtime. To handle priority escalation properly, we really
need to track this at a finer grain of detail: recording that the
task is running on a specific thread, enqueued on a specific actor,
or so on. But starting by tracking a single bit is important for
two reasons:
- First, it's more realistic about the performance overheads of
tasks: we're going to be doing this tracking eventually, and
the cost of that tracking will be dominated by the atomic
access, so doing that access now sets the baseline about right.
- Second, it ensures that we've actually got runtime involvement
in all the right places to do this tracking.
A propos of the latter: there was no runtime involvement with
awaiting a continuation, which is a point at which the task
potentially transitions from running to suspended. We must do
the tracking as part of this transition, rather than recognizing
in the run-loops that a task is still active and treating it as
having suspended, because the latter point potentially races with
the resumption of the task. To do this, I've had to introduce
a runtime function, swift_continuation_await, to do this awaiting
rather than inlining the atomic operation on the continuation.
As part of doing this work, I've also fixed a bug where we failed
to load-acquire in swift_task_escalate before walking the task
status records to invoke escalation actions.
I've also fixed several places where the handling of task statuses
may have accidentally allowed the task to revert to uncancelled.
The self object isn't actually a Swift object, so we can neither
do class dispatch on it nor retain it with swift_retain.
Some of the credit goes to Mike Ash on this one. All the
blame is mine, of course.
This builtin never occurs in @inlinable code. But apparently we still
need to add a language feature for every builtin. This must allow
older compilers to reparse the library source (though I don't know why
that would ever happen!)
Fixes rdar://80525569 error: module 'Builtin' has no member named 'hopToActor')
The prior implementation of `Task.sleep()` effectively had two
different atomic words to capture the state, which could lead to cases
where cancelling before a sleep operation started would fail to
throw `CancellationError`. Reimplement the logic for the cancellable
sleep with a more traditional lock-free approach by
packing all of the state information into a single word, where we
always load, figure out what to do, then compare-and-swap.
