Do this as a staged change to the ABI, introducing an underscored
`@usableFromInline` implementation to the ABI that we can rely on
later, and an `@_alwaysEmitIntoClient` version we can inline now.
The symbol is swift_async_extendedFramePointerFlags. Since the value doesn't need to be dynamically computed, we save a level of indirection by emitting a fake global variable whose address is the value we want, similar to objc_absolute_packed_isa_class_mask.
This bit is mixed in to the frame pointer address stored on the stack to signal that a frame is an async frame. The compiler can emit code that ORs in the address of this symbol to apply the appropriate flag when it doesn't know the flag statically.
rdar://80277146
* Synchronize both versions of actor_counters.swift test
* Synchronize on Job address
Make sure to synchronize on Job address (AsyncTasks are Jobs, but not
all Jobs are AsyncTasks).
* Add fprintf debug output for TSan acquire/release
* Add tsan_release edge on task creation
without this, we are getting false data races between when a task
is created and immediately scheduled on a different thread.
False positive for `Sanitizers/tsan/actor_counters.swift` test:
```
WARNING: ThreadSanitizer: data race (pid=81452)
Read of size 8 at 0x7b2000000560 by thread T5:
#0 Counter.next() <null>:2 (a.out:x86_64+0x1000047f8)
#1 (1) suspend resume partial function for worker(identity:counters:numIterations:) <null>:2 (a.out:x86_64+0x100005961)
#2 swift::runJobInEstablishedExecutorContext(swift::Job*) <null>:2 (libswift_Concurrency.dylib:x86_64+0x280ef)
Previous write of size 8 at 0x7b2000000560 by main thread:
#0 Counter.init(maxCount:) <null>:2 (a.out:x86_64+0x1000046af)
#1 Counter.__allocating_init(maxCount:) <null>:2 (a.out:x86_64+0x100004619)
#2 runTest(numCounters:numWorkers:numIterations:) <null>:2 (a.out:x86_64+0x100006d2e)
#3 swift::runJobInEstablishedExecutorContext(swift::Job*) <null>:2 (libswift_Concurrency.dylib:x86_64+0x280ef)
#4 main <null>:2 (a.out:x86_64+0x10000a175)
```
New edge with this change:
```
[4357150208] allocate task 0x7b3800000000, parent = 0x0
[4357150208] creating task 0x7b3800000000 with parent 0x0
[4357150208] tsan_release on 0x7b3800000000 <<< new release edge
[139088221442048] tsan_acquire on 0x7b3800000000
[139088221442048] trying to switch from executor 0x0 to 0x7ff85e2d9a00
[139088221442048] switch failed, task 0x7b3800000000 enqueued on executor 0x7ff85e2d9a00
[139088221442048] enqueue job 0x7b3800000000 on executor 0x7ff85e2d9a00
[139088221442048] tsan_release on 0x7b3800000000
[139088221442048] tsan_release on 0x7b3800000000
[4357150208] tsan_acquire on 0x7b3800000000
counters: 1, workers: 1, iterations: 1
[4357150208] allocate task 0x7b3c00000000, parent = 0x0
[4357150208] creating task 0x7b3c00000000 with parent 0x0
[4357150208] tsan_release on 0x7b3c00000000 <<< new release edge
[139088221442048] tsan_acquire on 0x7b3c00000000
[4357150208] task 0x7b3800000000 waiting on task 0x7b3c00000000, going to sleep
[4357150208] tsan_release on 0x7b3800000000
[4357150208] tsan_release on 0x7b3800000000
[139088221442048] getting current executor 0x0
[139088221442048] tsan_release on 0x7b3c00000000
...
```
rdar://78932849
* Add static_cast<Job *>()
* Move TSan release edge to swift_task_enqueueGlobal()
Move the TSan release edge from `swift_task_create_commonImpl()` to
`swift_task_enqueueGlobalImpl()`. Task creation itself is not an event
that needs synchronization, but rather that task creation "happens
before" execution of that task on another thread.
This edge is usually added when the task is scheduled via
`swift_task_enqueue()` (which then usually calls
`swift_task_enqueueGlobal()`). However, not all task scheduling goes
through the `swift_task_enqueue()` funnel as some places call the more
specific `swift_task_enqueueGlobal()` directly. So let's annotate this
function (duplicate edges aren't harmful) to ensure we cover all
schedule events, including newly-created tasks (our original problem
here).
rdar://78932849
Co-authored-by: Julian Lettner <julian.lettner@apple.com>
This is to workaround a bug in llvm's codegen when emitting the
callee-pop stack adjustment on a regular return from a swiftasync
function (vs. a tail call).
Without the workaround we fail to emit the callee-pop stack adjustment
leading to a mis-aligned stack on return.
```
pop {r7, pc}
add sp, #16
```
Workaround for rdar://79726989
The `swift_task_create` entry point is our general runtime ABI for
launching tasks. Make the various Swift APIs sitting on top of it
always-emit-into-client to take them out of the ABI. This reduces the
number of ABI entry points and allows us to make more ABI-compatible
changes to the Swift side.
We're not actually performing the adjustments at the moment due to an
unrelated bug, and will want to perform them within
`swift_task_create_common` based on inheritContext and the given
priority.
Rather than using group task options constructed from the Swift parts
of the _Concurrency library and passed through `createAsyncTask`'s
options, introduce a separate builtin that always takes a group. Move
the responsibility for creating the options structure into IRGen, so
we don't need to expose the TaskGroupTaskOptionRecord type in Swift.
Introduce a builtin `createAsyncTask` that maps to `swift_task_create`,
and use that for the non-group task creation operations based on the
task-creation flags. `swift_task_create` and the thin function version
`swift_task_create_f` go through the dynamically-replaceable
`swift_task_create_common`, where all of the task creation logic is
present.
While here, move copying of task locals and the initial scheduling of
the task into `swift_task_create_common`, enabling by separate flags.
The flags that are useful for task creation are a bit different from
the flags that go on a job. Create a separate flag set for task
creation and use that in the API for `swift_task_create`. For now,
have the callers do the remapping.
Collapse the `group` parameter of this API into the task options, and
have existing callers set up the options appropriately. The goal for
this function is to become the centralized entry point for all task
creation, with an extensible interface.
The `createTaskGroup` builtin has changed its signature and now requires
a type metadata argument. Change the feature accordingly so that
compilers with the new and old versions have disjoint feature sets.
Fixes rdar://79561865.
introduce new options parameter to all task spawning
[Concurrency] ABI for asynclet start to accept options
[Concurrency] fix unittest usages of changed task creation ABI
[Concurrency] introduce constants for parameter indexes in ownership
[Concurrency] fix test/SILOptimizer/closure_lifetime_fixup_concurrency.swift
I added Builtin.buildMainActorExecutor before, but because I never
implemented it correctly in IRGen, it's not okay to use it on old
versions, so I had to introduce a new feature only for it.
The shim dispatch queue class in the Concurrency runtime is rather
awful, but I couldn't think of a reasonable alternative without
just entirely hard-coding the witness table in the runtime.
It's not ABI, at least.
This type can be put to good use outside of the concurrency runtime. Reexport it from libswift_Concurrency so it can still be found there as well.
rdar://79352061
