When multi-block `alloc_stack`s of enum type with users are promoted,
lifetimes of stored values are completed to compensate for the legality
of not `destroy_addr`ing the `alloc_stack` in blocks where no
non-trivial value is known to be stored (the none case block of a
`switch_enum` of a load_borrow of the `alloc_stack`, e.g.).
Here, phis created during promotion are lifetime completed as well.
Previously, the value was computed and destroyed during
`promoteAllocationInBlock`. Make the value available in
`StackAllocationPromoter`'s top-level routine (`::run`) in preparation
for using it there.
Because getLexicalValueForStore returns "null" when
lexicalLifetimeEnsured is false, there's no need to check first whether
lexicalLifetimeEnsured is true before calling getLexicalValueForStore
and only taking action if a non-"null" value is returned.
@guaranteed stored values will never be added to a phi in mem2reg because
any uses of store borrowed locations have to be accessed via the store borrow return address
and since we ban address phis altogether we will never have input sil which necessitates this.
But, the DJ-edges based algorithm for inserting phi blocks in mem2reg can insert unnecessary phis
which are removed later.
Ensure fixPhiPredBlock handles both owned and guaranteed stored values correctly for this reason.
Previously it was disabled because we have incomplete address lifetimes for such types,
and transforming to value form in mem2reg would expose verification errors due to incompleteness.
Enable this case here and fixup the lifetimes using the new lifetime completion utility.
Previously, when blocks were added to the worklist, only blocks which
were users of the `alloc_stack` instruction were considered. For
"guaranteed alloc_stacks" (`store_borrow` locations), that resulted in
not processing blocks which contained uses of the `store_borrow` but not
the `alloc_stack`. When such a user was an `end_borrow`, the effect was
that no `end_borrow` was created for the newly introduced
`begin_borrow [lexical]`.
Fix this by adding blocks with users of the `store_borrow` to the
worklist.
Currently, memory locations whose type is empty (`SILType::isEmpty`) are
regarded as viable sources for loads.
Previously, though, Mem2Reg only handled loads from empty types formed
only by tupling. Here, support is added for types formed also by
struct'ing. As before, this entails recursively instantiating the empty
types until reaching the innermost empty types (which aggregate nothing)
and then aggregating the resulting instances.
rdar://106224845
Previously, LiveValues consisted always of three values: the value which
was stored, the borrow, and the copy. For store_borrows, there never
was a copy. Treating the two different scenarios as if they were the
same was confusing already. It was only get when we switch to
representing owned lexical lifetimes with move_values.
The type Optional<Ty *> implied that there was a meaningful distinction
between None and Some(nullptr) but that was not the case here. Replaced
it with a bare Ty *.
No need to bind to the StoreBorrowInst to get the source because it's
already bound to the local variable `stored`. And no need to bind to a
more specific type to find the next instruction.
Extend the definition of isGuaranteedLexicalValue--by means of which
Mem2Reg determines whether to borrow introducing a begin_borrow
[lexical] of a value which is store_borrow'd to an alloc_stack
[lexical]--to include every guaranteed lexical value.
Because lexical borrows are already avoided for store_borrows of lexical
values, the function is already misnamed: it's not that Mem2Reg should
necessarily _add_ a lexical lifetime, but rather that it should ensure
that there is one. Considering that we should do the same for owned
lexical values, the renaming will remain appropriate later. Finally,
name it so that it can switch from being a boolean to returning a
tristate (none, guaranteed, owned) when that becomes necessary (as it
will when we need to distinguish among the states to determine what phis
look like).
Now that StackAllocationPromoter::initializationPoints maps to either a
StoreInst or a StoreBorrowInst, there is no longer a subtype of
SILInstruction * at which the BlockToInstMap could be specialized, so
just eliminate the template argument and erase some angle brackets.
Add `deletableInstructions()` and `reverseDeletableInstructions()` in SILBasicBlock.
It allows deleting instructions while iterating over all instructions of the block.
This is a replacement for `InstructionDeleter::updatingRange()`.
It's a simpler implementation than the existing `UpdatingListIterator` and `UpdatingInstructionIteratorRegistry`, because it just needs to keep the prev/next pointers for "deleted" instructions instead of the iterator-registration machinery.
It's also safer, because it doesn't require to delete instructions via a specific instance of an InstructionDeleter (which can be missed easily).
`getValue` -> `value`
`getValueOr` -> `value_or`
`hasValue` -> `has_value`
`map` -> `transform`
The old API will be deprecated in the rebranch.
To avoid merge conflicts, use the new API already in the main branch.
rdar://102362022
Instead of checking if the end_borrow is ending the lifetime of the store_borrow of the asi under consideration,
this code was checking if the store_borrow source is the runningValue which is incorrect in cases where a store_borrow
src to another destination gets replaced during mem2reg. This PR fixes the issue.
With the change to include `SmallVector.h` directly in `LLVM.h` rather
than forward declaring in the only case it matters (ie. Clang <= 5),
these fixes are no longer needed. Since defaulted version is preferred
when there's no better choice (which is presumably the case if that's
how they were originally added), use it instead. Some uses were instead
changed to add `llvm::` so remove that too.
Already, load [take]s of struct_element_addr|tuple_element_addr
projections resulted in Mem2Reg bailing. Expand that to include load
[take]s involving unchecked_addr_cast.
To handle load [take]s of (struct|tuple)_element_addr projections, it
would be necessary to replace the running value with a value obtained
from the original product by recursive destructuring, replacing the
value at the load [take]n address with undef, and then restructuring.
To handle load [take]s of cast projections, it would be necessary to use
unchecked_value_cast instead of unchecked_bitwise_cast. But we would
need to still use unchecked_bitwise_cast in the case of load [copy]
because otherwise we would lose the original value--unchecked_value_cast
forwards ownership, and not all casts can be reversed (because they may
narrow).
For now, just bail out in the face of these complex load [take]s.
When optimizing an enum `store` to an `alloc_stack`, require that all uses are in the same block.
Otherwise it could be a `switch_enum` of an optional where the none-case does not have a destroy of the enum value.
After transforming such an `alloc_stack`, the value would leak in the none-case block.
It fixes the same OSSA verification error as done for TempRValueOpt in a previous commit.
Otherwise they will never compare the same. Also restore the test that was
updated for the previous commit to its old state. I did this to ensure that each
commit would compile successfully and so that I could show the test associated
with this commit.
The reason why I am doing this is that we are going to be enabling lexical
lifetimes early in the pipeline so that I can use it for the move operator's
diagnostics.
To make it easy for passes to know whether or not they should support lexical
lifetimes, I included a query on SILOptions called
supportsLexicalLifetimes. This will return true if the pass (given the passed in
option) should insert the lexical lifetime flag. This ensures that passes that
run in both pipelines (e.x.: AllocBoxToStack) know whether or not to set the
lexical lifetime flag without having to locally reason about it.
This is just chopping off layers of a larger patch I am upstreaming.
NOTE: This is technically NFC since it leaves the default alone of not inserting
lexical lifetimes at all.
Thanks to the lack of critical edges in SIL, if a block B dominated by P
has a successor S which is not dominated by P, then B must have only a
single successor. Used this fact to replace a loop over successors to a
call to getSinglePredecessor.
Also added an assertion that, in the notation above, B is dominated by
P.
Previously, it was asserted that any single-block allocation which had
valid memory after all instructions in the block were visited terminated
in unreachable. That assertion was false--while all paths through that
block must end in unreachable, the block itself need not be terminated
with an unreachable inst. Here, that is corrected by walking forward
from the block until blocks terminated by unreachables or blocks not
dominated by the block containing the alloc_stack are reached. In the
first case, the lifetime is ended just before the unreachable inst. In
the second, the lifetime is ended just before the branch to such a
successor block (i.e. just before the branch to a block which is not
dominated by the block containing the alloc_stack).
