We cannot use spare bits or other overlapping storage layout tricks with fundamentally
address-only enums, and we can take advantage of this to do borrowing switches or other
in-place projections without copying the value. However, for resilient enums, the
implementation may use spare bit packing, but the type must be handled address-only
outside of its defining module, and we didn't have a way to express that with
borrowing switch. Optimization passes have also been running into problems with the
complexity that we were using `unchecked_take_enum_data_addr` sometimes as a pure
operation. This patch splits the instruction into three:
- `unchecked_inplace_enum_data_addr` represents a nondestructive in-place enum
projection. It is only allowed for enums whose projection operation is
nondestructive.
- `unchecked_take_enum_data_addr` represents a destructive enum projection,
invalidating the enum and leaving the payload to be further consumed.
This matches the current instruction's semantics.
- `unchecked_borrow_enum_data_addr` represents a borrowing enum projection.
The instruction takes a second operand for "scratch" space, which the
enum representation may be copied into in order to avoid invalidating the
enum value, so the result is dependent on the lifetime of both the
original enum and the scratch buffer. This allows for borrowing switches
over resilient enums.
`unchecked_borrow_enum_data_addr` is implemented by taking advantage of the
"address-only enums can't do spare bit optimization" property at runtime.
We inspect the operand type's bitwise-borrowability from its metadata. If
the type is bitwise-borrowable, then we are allowed to bitwise-copy the
enum to the scratch space and apply the projection to the scratch space,
preserving the original value. If the type is not bitwise-borrowable, then
we cannot use spare bit optimization in its layout, so we apply the
projection in-place.
Fixes rdar://174952822.
Out of SILGen, we'll get the non-indirect SSA for throwing
the error. AddressLowering then converts a `throw` into
`throw_addr` to match the function convention. Similarly, a
try_apply gets rewritten to pass the error address to the
error successor block.
resolves rdar://158171053
We are going to need to add more flags to the various checked cast
instructions. Generalize the CastingIsolatedConformances bit in all of
these SIL instructions to an "options" struct that's easier to extend.
Precursor to rdar://152335805.
When performing a dynamic cast to an existential type that satisfies
(Metatype)Sendable, it is unsafe to allow isolated conformances of any
kind to satisfy protocol requirements for the existential. Identify
these cases and mark the corresponding cast instructions with a new flag,
`[prohibit_isolated_conformances]` that will be used to indicate to the
runtime that isolated conformances need to be rejected.
A begin_apply token may be used by operands that do not end the coroutine:
mark_dependence.
We need an API that gives us only the coroutine-ending uses. This blocks
~Escapable accessors.
end_borrow is considered coroutine-ending even though it does not actually
terminate the coroutine.
We cannot simply ask isLifetimeEnding, because end_apply and abort_apply do not
end any lifetime.
When checking whether an instruction is contained in a liveness
boundary, a pointer to a DeadEndBlocks instance must always be passed.
When the pointer is null, it is only checked that the instruction occurs
within the direct live region. When the pointer is non-null, it is
checked whether the instruction occurs within the region obtained by
extending the live region up to the availability boundary within
dead-end regions that are adjacent to the non-lifetime-ending portion of
the liveness boundary.
This corresponds to the parameter-passing convention of the Itanium C++
ABI, in which the argument is passed indirectly and possibly modified,
but not destroyed, by the callee.
@in_cxx is handled the same way as @in in callers and @in_guaranteed in
callees. OwnershipModelEliminator emits the call to destroy_addr that is
needed to destroy the argument in the caller.
rdar://122707697
Create two versions of the following functions:
isConsumedParameter
isGuaranteedParameter
SILParameterInfo::isConsumed
SILParameterInfo::isGuaranteed
SILArgumentConvention::isOwnedConvention
SILArgumentConvention::isGuaranteedConvention
These changes will be needed when we add a new convention for
non-trivial C++ types as the functions will return different answers
depending on whether they are called for the caller or the callee. This
commit doesn't change any functionality.
Although I don't plan to bring over new assertions wholesale
into the current qualification branch, it's entirely possible
that various minor changes in main will use the new assertions;
having this basic support in the release branch will simplify that.
(This is why I'm adding the includes as a separate pass from
rewriting the individual assertions)
The copy operator has been implemented and doesn't use it. Remove
`Builtin.copy` and `_copy` as much as currently possible.
Source compatibility requires that `_copy` remain in the stdlib. It is
deprecated here and just uses the copy operator.
Handling old swiftinterfaces requires that `Builtin.copy` be defined.
Redefine it here as a passthrough--SILGen machinery will produce the
necessary copy_addr.
rdar://127502242
For years, optimizer engineers have been hitting a common bug caused by passes
assuming all SILValues have a parent function only to be surprised by SILUndef.
Generally we see SILUndef not that often so we see this come up later in
testing. This patch eliminates that problem by making SILUndef uniqued at the
function level instead of the module level. This ensures that it makes sense for
SILUndef to have a parent function, eliminating this possibility since we can
define an API to get its parent function.
rdar://123484595
In the fullness of time, AddressLowering should never see such
instructions because they will be eliminated after diagnostics. Until
that time, though they need to be handled.
Resolves numerous regressions when enabling AddressLowering early in the
pipeline.
Previously, if a value incoming into a phi had storage which itself was
a use-projection out of some other storage, PhiStorageOptimizer bailed
out. The result was unnecessary "moves" (i.e. `copy_addr [take] [init]`
instructions).
Here, this bailout is removed. In order to do this, it is necessary to
find (1) all values whose storage recursively project out of an incoming
value (such a value may have storage which is either a use _or_ a def
projection) and (2) the block which dominates the defs of all these
values.
Together, these values are used to compute liveness to determine
interference. Previously, the live region was that between the uses of
an incoming value and its defining block. Now, it is that between the
uses of any of the values found in (1) and the dominating block found in
(2).
If a value's storage is a use projection out of some use,
AddressLowering correctly does not move into that storage when
initializing the composing use of the operand for the user into whose
storage it projects.
Previously, however, it was only checked that the value's storage was a
use projection at all. When initializing the composing use of the
operand of some _other_ user of the value (i.e. a user into whose
storage it does _not_ project), the result was a missing move.
Here, this is fixed by only skipping the move when the value's storage
is a projection _and_ it projects into the storage of the user of the
operand at hand.
When determining where the "latest opening instruction" is, consider not
just `OpenedArchetypeType`s but any `LocalArchetypeType` which includes
`PackArchetypeType`s.
When rewriting arguments, the index used is into the callee's argument
list. For full applies, that is identical to the index into the
instruction's argument list. For partial applies, it is not.
Previously, though, the index was used as if it were an index into the
instruction's argument list to get an argument ref. Here, use instead
the newly added convenience to get the argument ref using the index into
the callee's arguments.
Full apply instructions have the same number of arguments as the callee
has parameters. Partial apply instructions have some number less than
or equal to the number of callee parameters.
In opaque values mode, all arguments are passed to a partial_apply (just
like all other flavors of apply) directly. AddressLowering needs to
rewrite the operands whose convention is indirect as it does for other
applies.
A function which returns a value of opaque result type
func f() -> some P { S() }
has a lowered, substituted signature
@convention(thin) () -> @out @_opaqueReturnTypeOf("$s4main1fQryF", 0) opaque
featuring an _opaqueReturnTypeOf attr. The SILFunctionArgument for that
@out result, however, must be of the type that the opaque result
substitutes to in the expansion context of the function.
Previously, `end_borrow`s were rewritten last in order to be able to
find them when inserting `end_borrow`s on behalf of newly created
`load_borrow`s. Generalize this to rewriting all lifetime-ending users
last. This is necessary for the lifetime utilities used by `isLoadCopy`
to remain accurate when rewriting a `copy_value` previously determined
to be from a load-copy pair.
Projections may involve opened archetypes. They must be dominated by
the instructions which open those archetypes.
When determining the earliest storage point for a value and the point at
which to insert its projections, look for such obstructions in the
projection chain. The first one found is the earliest storage point.
The pass rewrites opaque values in reverse post order of their
definitions. When an opaque value is rewritten, both its definition and
its uses are rewritten. When a def or a use is rewritten, in order to
materialize an address with which to rewrite, projection instructions
are created. Previously, these projections were created at the site of
the def or use. Some of these projection instructions may be reused when
rewriting later opaque values.
As a result, it's possible to have two opaque values `A` and `B` (which
are rewritten in that order) such that rewriting a use of `A` which
occurs "after" the def (or a use) of `B` creates a projection `P` which
is then used by that "earlier" def (or use) of `B`:
```
A =
B = // relies on P
use B
use A // creates some projection P
```
When rewriting the def (or that use, respectively) of `B`, the
projection which was created for the use of `A` will be reused. And
previously, the projection would be created at the use of A. But that
projection instruction was "after" the place where it is used when
rewriting the def or use of `B`. That's invalid!
To address this, rather than creating projections at the instruction
being rewritten, instead create them "as early as possible". The
locations where they will be created are chosen so as to dominate all
uses.
Moved the assert that the value which opens an archetype exists before
the call to get its defining instruction which would be an NPE if the
value were in fact null.
Promoted the functionality from createStackAllocation to a named member
AddressLoweringState::getLatestOpeningInst so that it can be employed
elsewhere.
When determining whether a copy_value is part of a copy->store pair, if
the value being copied is guaranteed, it is checked whether the store is
within the lifetime of all its guaranteed roots. If one of those
guaranteed roots is a function argument, the store is certainly within
the lifetime, so exit early.
An extract may have users that have escaping operand ownership.
Consequently, we can't rely on findInnerTransitiveGuaranteedUses as used
by emitEndBorrows.
A destructure may have users that have escaping operand ownership.
Consequently, we can't rely on findInnerTransitiveGuaranteedUses as used
by emitEndBorrows.
When a `load_borrow` is created on behalf of a `begin_apply`, it's
possible that it may have escaping users. When it does,
`findInnerTransitiveGuaranteedUses` returns `false` and the uses it
finds are incomplete. As a result the boundary computed in
`emitEndBorrows` will be incomplete.
Joe Groff
Slava Pestov
Kavon Farvardin
Doug Gregor
Andrew Trick
Nate Chandler
Akira Hatanaka
Tim Kientzle
Michael Gottesman
nate-chandler