We cannot use spare bits or other overlapping storage layout tricks with fundamentally
address-only enums, and we can take advantage of this to do borrowing switches or other
in-place projections without copying the value. However, for resilient enums, the
implementation may use spare bit packing, but the type must be handled address-only
outside of its defining module, and we didn't have a way to express that with
borrowing switch. Optimization passes have also been running into problems with the
complexity that we were using `unchecked_take_enum_data_addr` sometimes as a pure
operation. This patch splits the instruction into three:
- `unchecked_inplace_enum_data_addr` represents a nondestructive in-place enum
projection. It is only allowed for enums whose projection operation is
nondestructive.
- `unchecked_take_enum_data_addr` represents a destructive enum projection,
invalidating the enum and leaving the payload to be further consumed.
This matches the current instruction's semantics.
- `unchecked_borrow_enum_data_addr` represents a borrowing enum projection.
The instruction takes a second operand for "scratch" space, which the
enum representation may be copied into in order to avoid invalidating the
enum value, so the result is dependent on the lifetime of both the
original enum and the scratch buffer. This allows for borrowing switches
over resilient enums.
`unchecked_borrow_enum_data_addr` is implemented by taking advantage of the
"address-only enums can't do spare bit optimization" property at runtime.
We inspect the operand type's bitwise-borrowability from its metadata. If
the type is bitwise-borrowable, then we are allowed to bitwise-copy the
enum to the scratch space and apply the projection to the scratch space,
preserving the original value. If the type is not bitwise-borrowable, then
we cannot use spare bit optimization in its layout, so we apply the
projection in-place.
Fixes rdar://174952822.
The biggest complexity here is dealing with the improper nesting of the
allocation for the argument, and I'm not very proud of the solution I found,
but for a one-off builtin, I think it'll work.
You can see in the patch that I obviously wrote this to include the
task-local builtins, but unfortunately they don't work for this because
they don't actually have a use/def relationship. That will need to be
follow-up work.
Includes a test that the defer special case applies to the priority
escalation builtin.
This interacts with my previous commits to ensure that we mark other
allocation operations as non-nested when we would otherwise need to move
the finishAsyncLet operation.
This can be used to get the SILDeclRef for all of the various apply site
kinds. It is a composition of getCalleeOrigin + special code for handling
FunctionRefs and MethodInst. It essentially lets one abstract over the function
ref/method inst code.
This is necessary because we need to model its stack-allocation
behavior, although I'm not yet doing that in this patch because
StackNesting first needs to be taught to not try to move the
deallocation.
I'm not convinced that `async let` *should* be doing a stack allocation,
but it undoubtedly *is* doing a stack allocation, and until we have an
alternative to that, we will need to model it properly.
functions to compute them directly without a TypeLowering object, and
change a lot of getTypeLowering call sites to just use that.
There is one subtle change here that I think is okay: SILBuilder used to
use different TypeExpansionContexts when inserting into a global:
- getTypeLowering() always used a minimal context when inserting into
a global
- getTypeExpansionContext() always returned a maximal context for the
module scope
The latter seems more correct, as AFAIK global initializers are never
inlinable. If they are, we probably need to configure the builder with
an actual context properly rather than making global assumptions.
This is incremental progress towards computing this for most types
without a TypeLowering, and hopefully eventually removing TL entirely.
This prevents simplification and SILCombine passes to remove (alive) `mark_dependence_addr`.
The instruction is conceptually equivalent to
```
%v = load %addr
%d = mark_dependence %v on %base
store %d to %addr
```
Therefore the address operand has to be defined as writing to the address.
To determine whether an instruction may require pack metadata, the types
of its operands are examined.
Previously, the top level type was checked for having a pack in its
signature, and the whole type was checked for having a type anywhere in
its layout (via TypeLowering). This didn't account for the case where
the metadata was required for a resilient type which uses a pack in its
signature.
Here, during type lowering, a type having a pack in its signature is
counted towards the type having a pack.
Fixes a compiler crash.
rdar://147207926
It is like `zeroInitializer`, but does not actually initialize the memory.
It only indicates to mandatory passes that the memory is going to be initialized.
This API only makes sense for a scoped borrow-introducer such as:
- reborrow
- owned mark_dependence
Borrowing operands that forward guaranteed values do not have scope-ending uses.
If the base value of a mark_dependence is an address, that memory location must be initialized at the mark_dependence.
This requires that the mark_dependence is considered to read from the base address.
This fixes an OSSA verification bug introduced here:
commit 79b649854b
Author: Meghana Gupta <meghanavgupta@gmail.com>
Date: Wed Jan 22 01:24:49 2025
Fix operand ownership of mark_dependence [nonescaping] of address values
The bug only showed up with mark_dependence [nonescaping], which means mainly
affects `~Escapable` types. Adddressors happened to work because SILGen was
emitting a borrow scope around them.
Rather than reverting the change above, this fix migrates mark_dependence
[nonescaping] to an implicit borrow scope.
Fixes rdar://144199759 (Assert: mark_dependence [nonescaping]:
ownership incompatible with an owned value)
In OSSA, the `partial_apply [on_stack]` instruction produces a value
with odd characteristics that correspond to the fact that it is lowered
to a stack-allocating instruction. Among these characteristics is the
fact that copies of such values aren't load bearing.
When visiting the lifetime-ending uses of a `partial_apply [on_stack]`
the lifetime ending uses of (transitive) copies of the partial_apply
must be considered as well. Otherwise, the borrow scope it defins may be
incorrectly short:
```
%closure = partial_apply %fn(%value) // borrows %value
%closure2 = copy_value %closure
destroy_value %closure // does _not_ end borrow of %value!
...
destroy_value %closure2 // ends borrow of %value
...
destroy_value %value
```
Furthermore, _only_ the final such destroys actually count as the real
lifetime ends. At least one client (OME) relies on
`visitOnStackLifetimeEnds` visiting at most a single lifetime end on any
path.
Rewrite the utility to use PrunedLiveness, tracking only destroys of
copies and forwards. The final destroys are the destroys on the
boundary.
rdar://142636711
The problem with `is_escaping_closure` was that it didn't consume its operand and therefore reference count checks were unreliable.
For example, copy-propagation could break it.
As this instruction was always used together with an immediately following `destroy_value` of the closure, it makes sense to combine both into a `destroy_not_escaped_closure`.
It
1. checks the reference count and returns true if it is 1
2. consumes and destroys the operand
This enables access enforcement analysis to classify a dynamic begin_access in
access patterns (such as the one below) involving a throwing function as not
having nested conflicts.
```
struct Stack {
var items : [UInt8]
mutating func pop() throws -> UInt8 {
guard let item = items.popLast() else { throw SomeErr.err }
return item
}
...
}
class Container {
private var ref : Stack
@inline(never)
internal func someMethod() throws {
try ref.pop()
}
...
}
```
rdar://141182074
In case the control flow ends in a dead-end block there can be begin-borrow instructions which have no corresponding end-borrow uses.
After duplicating such a block, the re-borrow flags cannot be recomputed correctly for inserted phi arguments.
Therefore just disable duplicating such blocks, e.g. in jump-threading.
When its operand has coroutine kind `yield_once_2`, a `begin_apply`
instruction produces an additional value representing the storage
allocated by the callee. This storage must be deallocated by a
`dealloc_stack` on every path out of the function. Like any other stack
allocation, it must obey stack discipline.
SILInstruction::clone doesn't know how to clone instructions that produce
the archetype uuid. SILCloner is equipped to handle such instructions.
Optimizations like LoopRotate use SILInstruction::clone and will be
incorrect for such instructions.
rdar://130047619
Although I don't plan to bring over new assertions wholesale
into the current qualification branch, it's entirely possible
that various minor changes in main will use the new assertions;
having this basic support in the release branch will simplify that.
(This is why I'm adding the includes as a separate pass from
rewriting the individual assertions)
Emil Pedersen
Joe Groff
John McCall
Meghana Gupta
Michael Gottesman
swift-ci
Anthony Latsis
eeckstein
nate-chandler
Nate Chandler
Andrew Trick
Meghana Gupta
Arnold Schwaighofer
Slava Pestov
Ben Barham
Tim Kientzle