To eliminate copies which become newly spurious, Mem2Reg canonicalizes
the lifetimes of values that are stored and of newly introduced phis
after rewriting.
It's possible, however, for the values that are stored to be deleted
during canonicalization if a value and its copy are both stored to the
address. Such values must not be canonicalized. So check whether
values have been erased before canonicalizing them.
rdar://113762355
Corrected the comments describing what the
`promoteSingleBlockAllocation` and `promoteMultiBlockAllocation`
functions do since they were no longer accurate.
Previously there were the functions `promoteSingleAllocation` and
`removeSingleBlockAllocation`. The distinction between these based on
the name alone is a bit subtle. Shortened the name of
`promoteSingleAllocation` to `promoteAllocation`.
Reformatting everything now that we have `llvm` namespaces. I've
separated this from the main commit to help manage merge-conflicts and
for making it a bit easier to read the mega-patch.
This is phase-1 of switching from llvm::Optional to std::optional in the
next rebranch. llvm::Optional was removed from upstream LLVM, so we need
to migrate off rather soon. On Darwin, std::optional, and llvm::Optional
have the same layout, so we don't need to be as concerned about ABI
beyond the name mangling. `llvm::Optional` is only returned from one
function in
```
getStandardTypeSubst(StringRef TypeName,
bool allowConcurrencyManglings);
```
It's the return value, so it should not impact the mangling of the
function, and the layout is the same as `std::optional`, so it should be
mostly okay. This function doesn't appear to have users, and the ABI was
already broken 2 years ago for concurrency and no one seemed to notice
so this should be "okay".
I'm doing the migration incrementally so that folks working on main can
cherry-pick back to the release/5.9 branch. Once 5.9 is done and locked
away, then we can go through and finish the replacement. Since `None`
and `Optional` show up in contexts where they are not `llvm::None` and
`llvm::Optional`, I'm preparing the work now by going through and
removing the namespace unwrapping and making the `llvm` namespace
explicit. This should make it fairly mechanical to go through and
replace llvm::Optional with std::optional, and llvm::None with
std::nullopt. It's also a change that can be brought onto the
release/5.9 with minimal impact. This should be an NFC change.
When `debug_value`s are visited, if their operand is the stack address,
they are rewritten as `debug_value`s of the stored value, provided it is
known.
Previously, the check for whether the operand is the stack address,
however, just compared the `debug_value`'s operand with the
`alloc_stack`. For owned `alloc_stack`s (i.e. those that are not
"store_borrow locations"), that was correct. For guaranteed
`alloc_stack`s (i.e. those that are "store_borrow locations"), however,
this failed to recognize the the values produced by the `store_borrow`
instructions (which amount to aliases for the `alloc_stack`) as the
stack address.
Here, this is fixed by checking whether the `debug_value`'s operand is
either (1) the `alloc_stack` itself or (2) some `store_borrow` whose
destination is the `alloc_stack`.
rdar://109894792
When multi-block `alloc_stack`s of enum type with users are promoted,
lifetimes of stored values are completed to compensate for the legality
of not `destroy_addr`ing the `alloc_stack` in blocks where no
non-trivial value is known to be stored (the none case block of a
`switch_enum` of a load_borrow of the `alloc_stack`, e.g.).
Here, phis created during promotion are lifetime completed as well.
Previously, the value was computed and destroyed during
`promoteAllocationInBlock`. Make the value available in
`StackAllocationPromoter`'s top-level routine (`::run`) in preparation
for using it there.
Because getLexicalValueForStore returns "null" when
lexicalLifetimeEnsured is false, there's no need to check first whether
lexicalLifetimeEnsured is true before calling getLexicalValueForStore
and only taking action if a non-"null" value is returned.
@guaranteed stored values will never be added to a phi in mem2reg because
any uses of store borrowed locations have to be accessed via the store borrow return address
and since we ban address phis altogether we will never have input sil which necessitates this.
But, the DJ-edges based algorithm for inserting phi blocks in mem2reg can insert unnecessary phis
which are removed later.
Ensure fixPhiPredBlock handles both owned and guaranteed stored values correctly for this reason.
Previously it was disabled because we have incomplete address lifetimes for such types,
and transforming to value form in mem2reg would expose verification errors due to incompleteness.
Enable this case here and fixup the lifetimes using the new lifetime completion utility.
Previously, when blocks were added to the worklist, only blocks which
were users of the `alloc_stack` instruction were considered. For
"guaranteed alloc_stacks" (`store_borrow` locations), that resulted in
not processing blocks which contained uses of the `store_borrow` but not
the `alloc_stack`. When such a user was an `end_borrow`, the effect was
that no `end_borrow` was created for the newly introduced
`begin_borrow [lexical]`.
Fix this by adding blocks with users of the `store_borrow` to the
worklist.
Currently, memory locations whose type is empty (`SILType::isEmpty`) are
regarded as viable sources for loads.
Previously, though, Mem2Reg only handled loads from empty types formed
only by tupling. Here, support is added for types formed also by
struct'ing. As before, this entails recursively instantiating the empty
types until reaching the innermost empty types (which aggregate nothing)
and then aggregating the resulting instances.
rdar://106224845
Previously, LiveValues consisted always of three values: the value which
was stored, the borrow, and the copy. For store_borrows, there never
was a copy. Treating the two different scenarios as if they were the
same was confusing already. It was only get when we switch to
representing owned lexical lifetimes with move_values.
The type Optional<Ty *> implied that there was a meaningful distinction
between None and Some(nullptr) but that was not the case here. Replaced
it with a bare Ty *.
No need to bind to the StoreBorrowInst to get the source because it's
already bound to the local variable `stored`. And no need to bind to a
more specific type to find the next instruction.
Extend the definition of isGuaranteedLexicalValue--by means of which
Mem2Reg determines whether to borrow introducing a begin_borrow
[lexical] of a value which is store_borrow'd to an alloc_stack
[lexical]--to include every guaranteed lexical value.
Because lexical borrows are already avoided for store_borrows of lexical
values, the function is already misnamed: it's not that Mem2Reg should
necessarily _add_ a lexical lifetime, but rather that it should ensure
that there is one. Considering that we should do the same for owned
lexical values, the renaming will remain appropriate later. Finally,
name it so that it can switch from being a boolean to returning a
tristate (none, guaranteed, owned) when that becomes necessary (as it
will when we need to distinguish among the states to determine what phis
look like).
Now that StackAllocationPromoter::initializationPoints maps to either a
StoreInst or a StoreBorrowInst, there is no longer a subtype of
SILInstruction * at which the BlockToInstMap could be specialized, so
just eliminate the template argument and erase some angle brackets.
Add `deletableInstructions()` and `reverseDeletableInstructions()` in SILBasicBlock.
It allows deleting instructions while iterating over all instructions of the block.
This is a replacement for `InstructionDeleter::updatingRange()`.
It's a simpler implementation than the existing `UpdatingListIterator` and `UpdatingInstructionIteratorRegistry`, because it just needs to keep the prev/next pointers for "deleted" instructions instead of the iterator-registration machinery.
It's also safer, because it doesn't require to delete instructions via a specific instance of an InstructionDeleter (which can be missed easily).
`getValue` -> `value`
`getValueOr` -> `value_or`
`hasValue` -> `has_value`
`map` -> `transform`
The old API will be deprecated in the rebranch.
To avoid merge conflicts, use the new API already in the main branch.
rdar://102362022
Instead of checking if the end_borrow is ending the lifetime of the store_borrow of the asi under consideration,
this code was checking if the store_borrow source is the runningValue which is incorrect in cases where a store_borrow
src to another destination gets replaced during mem2reg. This PR fixes the issue.
With the change to include `SmallVector.h` directly in `LLVM.h` rather
than forward declaring in the only case it matters (ie. Clang <= 5),
these fixes are no longer needed. Since defaulted version is preferred
when there's no better choice (which is presumably the case if that's
how they were originally added), use it instead. Some uses were instead
changed to add `llvm::` so remove that too.
