When we have a non trivial enum lexical stack location, and we store a
none value, after mem2reg we can end up with an ownership error
because of the leaking move_value [lexical].
This change avoids creating move_value [lexical] for such stores.
Do the same when we have store_borrow of none values, even though there
is no ownership error in this case, handle it like store for consistency
Although I don't plan to bring over new assertions wholesale
into the current qualification branch, it's entirely possible
that various minor changes in main will use the new assertions;
having this basic support in the release branch will simplify that.
(This is why I'm adding the includes as a separate pass from
rewriting the individual assertions)
For years, optimizer engineers have been hitting a common bug caused by passes
assuming all SILValues have a parent function only to be surprised by SILUndef.
Generally we see SILUndef not that often so we see this come up later in
testing. This patch eliminates that problem by making SILUndef uniqued at the
function level instead of the module level. This ensures that it makes sense for
SILUndef to have a parent function, eliminating this possibility since we can
define an API to get its parent function.
rdar://123484595
Mem2Reg may materialize the unique instance of empty types when
promoting an address of that empty type. Previously, it was required
that the top-most type in the aggregate be empty. This failed to handle
the case where a projection of empty type from a non-empty aggregate was
promoted.
For example:
```
%addr = alloc_stack $((), C)
%empty_addr = tuple_element_addr $addr : $*((), C), 0
%addr = load %empty_addr : $*()
```
where `C` is some non-empty type.
Here, that case is handled by using `undef` for each non-empty field in
the projected-from aggregate.
In the example,
```
%empty = tuple ()
%tuple = tuple (%empty : $(), undef : $C)
%empty_again = tuple_extract %tuple : $((), C), 0
```
rdar://122417297
We allow none values to be stored to a non-trivial enum.
For such store_borrow, `LiveValues::forValues` used
an Owned storage type, but endLexicalLifetime expected Guaranteed
storage type, leading to a compiler crash.
For store_borrow, use the LiveValues::forGuaranteed and for store
use LiveValues::forOwned to avoid this.
Fixes rdar://114390472
To eliminate copies which become newly spurious, Mem2Reg canonicalizes
the lifetimes of values that are stored and of newly introduced phis
after rewriting.
It's possible, however, for the values that are stored to be deleted
during canonicalization if a value and its copy are both stored to the
address. Such values must not be canonicalized. So check whether
values have been erased before canonicalizing them.
rdar://113762355
Corrected the comments describing what the
`promoteSingleBlockAllocation` and `promoteMultiBlockAllocation`
functions do since they were no longer accurate.
Previously there were the functions `promoteSingleAllocation` and
`removeSingleBlockAllocation`. The distinction between these based on
the name alone is a bit subtle. Shortened the name of
`promoteSingleAllocation` to `promoteAllocation`.
Reformatting everything now that we have `llvm` namespaces. I've
separated this from the main commit to help manage merge-conflicts and
for making it a bit easier to read the mega-patch.
This is phase-1 of switching from llvm::Optional to std::optional in the
next rebranch. llvm::Optional was removed from upstream LLVM, so we need
to migrate off rather soon. On Darwin, std::optional, and llvm::Optional
have the same layout, so we don't need to be as concerned about ABI
beyond the name mangling. `llvm::Optional` is only returned from one
function in
```
getStandardTypeSubst(StringRef TypeName,
bool allowConcurrencyManglings);
```
It's the return value, so it should not impact the mangling of the
function, and the layout is the same as `std::optional`, so it should be
mostly okay. This function doesn't appear to have users, and the ABI was
already broken 2 years ago for concurrency and no one seemed to notice
so this should be "okay".
I'm doing the migration incrementally so that folks working on main can
cherry-pick back to the release/5.9 branch. Once 5.9 is done and locked
away, then we can go through and finish the replacement. Since `None`
and `Optional` show up in contexts where they are not `llvm::None` and
`llvm::Optional`, I'm preparing the work now by going through and
removing the namespace unwrapping and making the `llvm` namespace
explicit. This should make it fairly mechanical to go through and
replace llvm::Optional with std::optional, and llvm::None with
std::nullopt. It's also a change that can be brought onto the
release/5.9 with minimal impact. This should be an NFC change.
When `debug_value`s are visited, if their operand is the stack address,
they are rewritten as `debug_value`s of the stored value, provided it is
known.
Previously, the check for whether the operand is the stack address,
however, just compared the `debug_value`'s operand with the
`alloc_stack`. For owned `alloc_stack`s (i.e. those that are not
"store_borrow locations"), that was correct. For guaranteed
`alloc_stack`s (i.e. those that are "store_borrow locations"), however,
this failed to recognize the the values produced by the `store_borrow`
instructions (which amount to aliases for the `alloc_stack`) as the
stack address.
Here, this is fixed by checking whether the `debug_value`'s operand is
either (1) the `alloc_stack` itself or (2) some `store_borrow` whose
destination is the `alloc_stack`.
rdar://109894792
When multi-block `alloc_stack`s of enum type with users are promoted,
lifetimes of stored values are completed to compensate for the legality
of not `destroy_addr`ing the `alloc_stack` in blocks where no
non-trivial value is known to be stored (the none case block of a
`switch_enum` of a load_borrow of the `alloc_stack`, e.g.).
Here, phis created during promotion are lifetime completed as well.
Previously, the value was computed and destroyed during
`promoteAllocationInBlock`. Make the value available in
`StackAllocationPromoter`'s top-level routine (`::run`) in preparation
for using it there.
Because getLexicalValueForStore returns "null" when
lexicalLifetimeEnsured is false, there's no need to check first whether
lexicalLifetimeEnsured is true before calling getLexicalValueForStore
and only taking action if a non-"null" value is returned.
@guaranteed stored values will never be added to a phi in mem2reg because
any uses of store borrowed locations have to be accessed via the store borrow return address
and since we ban address phis altogether we will never have input sil which necessitates this.
But, the DJ-edges based algorithm for inserting phi blocks in mem2reg can insert unnecessary phis
which are removed later.
Ensure fixPhiPredBlock handles both owned and guaranteed stored values correctly for this reason.
Previously it was disabled because we have incomplete address lifetimes for such types,
and transforming to value form in mem2reg would expose verification errors due to incompleteness.
Enable this case here and fixup the lifetimes using the new lifetime completion utility.
Previously, when blocks were added to the worklist, only blocks which
were users of the `alloc_stack` instruction were considered. For
"guaranteed alloc_stacks" (`store_borrow` locations), that resulted in
not processing blocks which contained uses of the `store_borrow` but not
the `alloc_stack`. When such a user was an `end_borrow`, the effect was
that no `end_borrow` was created for the newly introduced
`begin_borrow [lexical]`.
Fix this by adding blocks with users of the `store_borrow` to the
worklist.
Currently, memory locations whose type is empty (`SILType::isEmpty`) are
regarded as viable sources for loads.
Previously, though, Mem2Reg only handled loads from empty types formed
only by tupling. Here, support is added for types formed also by
struct'ing. As before, this entails recursively instantiating the empty
types until reaching the innermost empty types (which aggregate nothing)
and then aggregating the resulting instances.
rdar://106224845
