When TMO is enabled, change IRGen to pass the newly introduced runtime function `swift_coroFrameAlloc` (and pass an additional argument — the hash value) instead of `malloc` when it inserts calls to `coro_id_retcon_once`.
The hashValue is computed based on the current function name (computed in `getDiscriminatorForString`)
rdar://141235957
rdar://141575655
In partial application forwarder emission, we were missing a check for indirect results.
When results are being returned indirectly, we have to return the error indirectly as well.
For types like `Atomic` and `Mutex`, we want to know that even though they are
technically bitwise-takable, they differ from other bitwise-takable types until
this point because they are not also "bitwise-borrowable"; while borrowed,
they are pinned in memory, so they cannot be passed by value as a borrowed
parameter, unlike copyable bitwise-takable types. Add a bit to the value witness
table flags to record this.
Note that this patch does not include any accompanying runtime support for
propagating the flag into runtime-instantiated type metadata. There isn't yet
any runtime functionality that varies based on this flag, so that can
be implemented separately.
rdar://136396806
So call the destroy on the closing type instead.
Amends the concepts areFieldsABIAccessible/isABIAccessible to take metadata
accessibility of non-copyable types into account.
rdar://133990500
Those functions are effectively outlined functions with an alwaysinline
attribute. By removing their debug info and relying on the inliner to propagate
the call site location to the inlined instructions, we restore the "original"
locations as if the function had never been outlined.
This is technically relying on an implementation detail of the inliner, but it
seems to be the simplest way of addressing this issue.
Those functions are effectively outlined functions with an alwaysinline
attribute. By removing their debug info and relying on the inliner to propagate
the call site location to the inlined instructions, we restore the "original"
locations as if the function had never been outlined.
This is technically relying on an implementation detail of the inliner, but it
seems to be the simplest way of addressing this issue.
The patch adds lowering of partial_apply instructions for coroutines.
This pattern seems to trigger a lot of type mismatch errors in IRGen, because
coroutine functions are not substituted in the same way as regular functions
(see the patch 07f03bd2 "Use pattern substitutions to consistently abstract
yields" for more details).
Other than that, lowering of partial_apply for coroutines is straightforward: we
generate another coroutine that captures arguments passed to the partial_apply
instructions. It calls the original coroutine for yields (first return) and
yields the resulting values. Then it calls the original function's continuation
for return or unwind, and forwards them to the caller as well.
After IRGen, LLVM's Coroutine pass transforms the generated coroutine (along with
all other coroutines) and eliminates llvm.coro.* intrinsics. LIT tests check
LLVM IR after this transformation.
Co-authored-by: Anton Korobeynikov <anton@korobeynikov.info>
Co-authored-by: Arnold Schwaighofer <aschwaighofer@apple.com>
rdar://129359370
Second part of direct error support. This implements direct errors for async functions. Instead of always returning typed errors indirectly, we are returning them directly when possible.
This corresponds to the parameter-passing convention of the Itanium C++
ABI, in which the argument is passed indirectly and possibly modified,
but not destroyed, by the callee.
@in_cxx is handled the same way as @in in callers and @in_guaranteed in
callees. OwnershipModelEliminator emits the call to destroy_addr that is
needed to destroy the argument in the caller.
rdar://122707697
Although I don't plan to bring over new assertions wholesale
into the current qualification branch, it's entirely possible
that various minor changes in main will use the new assertions;
having this basic support in the release branch will simplify that.
(This is why I'm adding the includes as a separate pass from
rewriting the individual assertions)
Call `swift_clearSensitive` after destroying or taking "sensitive" struct types.
Also, support calling C-functions with "sensitive" parameters or return values. In SIL, sensitive types are address-only and so are sensitive parameters/return values.
Though, (small) sensitive C-structs are passed directly to/from C-functions. We need re-abstract such parameter and return values for C-functions.
Fix a bug in expandExternalSignatureTypes where it wasn't annotating a function call parameter type with sret when the result was being returned indirectly.
The bug was causing calls to ObjC methods that return their results indirectly to crash.
Additionally, fix the return type for C++ constructors computed in expandExternalSignatureTypes. Previously, the return type was always void even on targets that require constructors to return this (e.g., Apple arm64), which was causing C++ constructor thunks to be emitted needlessly.
Resolves rdar://121618707
KeyPath's getter/setter/hash/equals functions have their own calling
convention, which receives generic arguments and embedded indices from a
given KeyPath argument buffer.
The convention was previously implemented by:
1. Accepting an argument buffer as an UnsafeRawPointer and casting it to
indices tuple pointer in SIL.
2. Bind generic arguments info from the given argument buffer while emitting
prologue in IRGen by creating a new forwarding thunk.
This 2-phase lowering approach was not ideal, as it blocked KeyPath
projection optimization [^1], and also required having a target arch
specific signature lowering logic in SIL-level [^2].
This patch centralizes the KeyPath accessor calling convention logic to
IRGen, by introducing `@convention(keypath_accessor_XXX)` convention in
SIL and lowering it in IRGen. This change unblocks the KeyPath projection
optimization while capturing subscript indices, and also makes it easier
to support WebAssembly target.
[^1]: https://github.com/apple/swift/pull/28799
[^2]: https://forums.swift.org/t/wasm-support/16087/21
Reformatting everything now that we have `llvm` namespaces. I've
separated this from the main commit to help manage merge-conflicts and
for making it a bit easier to read the mega-patch.
This is phase-1 of switching from llvm::Optional to std::optional in the
next rebranch. llvm::Optional was removed from upstream LLVM, so we need
to migrate off rather soon. On Darwin, std::optional, and llvm::Optional
have the same layout, so we don't need to be as concerned about ABI
beyond the name mangling. `llvm::Optional` is only returned from one
function in
```
getStandardTypeSubst(StringRef TypeName,
bool allowConcurrencyManglings);
```
It's the return value, so it should not impact the mangling of the
function, and the layout is the same as `std::optional`, so it should be
mostly okay. This function doesn't appear to have users, and the ABI was
already broken 2 years ago for concurrency and no one seemed to notice
so this should be "okay".
I'm doing the migration incrementally so that folks working on main can
cherry-pick back to the release/5.9 branch. Once 5.9 is done and locked
away, then we can go through and finish the replacement. Since `None`
and `Optional` show up in contexts where they are not `llvm::None` and
`llvm::Optional`, I'm preparing the work now by going through and
removing the namespace unwrapping and making the `llvm` namespace
explicit. This should make it fairly mechanical to go through and
replace llvm::Optional with std::optional, and llvm::None with
std::nullopt. It's also a change that can be brought onto the
release/5.9 with minimal impact. This should be an NFC change.
Instead of passing `IRGenFunction`, pass the `IRGenModule` and the `IRBuilder`.
This makes enum creation not dependent on the presence of a function.
NFC, just refactoring.
Fix the type of the `alloca` created by `GenPack`'s for type metadata
and witness tables by fixing its callee, `emitDynamicAlloca` to always
return a `StackAddress` whose `Address`' type is the one specified by
the caller.
rdar://109540863
And use the new bit to ensure we don't try to lower move-only types
with common layout value witness surrogates. Take a bit in the runtime
value witness flags to represent types that are not copyable.
Noncopyable types aren't really "POD", but the bit is still useful to track
whether a noncopyable type has a no-op destroy operation, so rename the
existing bit to be more specific within IRGen's implementation.
Don't rename it in the runtime or Builtin names yet, since doing so will
require a naming transition for compatibility.
rdar://105837040
* WIP: Store layout string in type metadata
* WIP: More cases working
* WIP: Layout strings almost working
* Add layout string pointer to struct metadata
* Fetch bytecode layout strings from metadata in runtime
* More efficient bytecode layout
* Add support for interpreted generics in layout strings
* Layout string instantiation, take and more
* Remove duplicate information from layout strings
* Include size of previous object in next objects offset to reduce number of increments at runtime
* Add support for existentials
* Build type layout strings with StructBuilder to support target sizes and metadata pointers
* Add support for resilient types
* Properly cache layout strings in compiler
* Generic resilient types working
* Non-generic resilient types working
* Instantiate resilient type in layout when possible
* Fix a few issues around alignment and signing
* Disable generics, fix static alignment
* Fix MultiPayloadEnum size when no extra tag is necessary
* Fixes after rebase
* Cleanup
* Fix most tests
* Fix objcImplementattion and non-Darwin builds
* Fix BytecodeLayouts on non-Darwin
* Fix Linux build
* Fix sizes in linux tests
* Sign layout string pointers
* Use nullptr instead of debug value
Adjust the IRGen to always set the calling convention and then the
noexcept bit. This makes it easier to quickly scan through the
attributes for the async calls which was useful for some Concurrency
related work on Windows.
- SILPackType carries whether the elements are stored directly
in the pack, which we're not currently using in the lowering,
but it's probably something we'll want in the final ABI.
Having this also makes it clear that we're doing the right
thing with substitution and element lowering. I also toyed
with making this a scalar type, which made it necessary in
various places, although eventually I pulled back to the
design where we always use packs as addresses.
- Pack boundaries are a core ABI concept, so the lowering has
to wrap parameter pack expansions up as packs. There are huge
unimplemented holes here where the abstraction pattern will
need to tell us how many elements to gather into the pack,
but a naive approach is good enough to get things off the
ground.
- Pack conventions are related to the existing parameter and
result conventions, but they're different on enough grounds
that they deserve to be separated.
