Already, load [take]s of struct_element_addr|tuple_element_addr
projections resulted in Mem2Reg bailing. Expand that to include load
[take]s involving unchecked_addr_cast.
To handle load [take]s of (struct|tuple)_element_addr projections, it
would be necessary to replace the running value with a value obtained
from the original product by recursive destructuring, replacing the
value at the load [take]n address with undef, and then restructuring.
To handle load [take]s of cast projections, it would be necessary to use
unchecked_value_cast instead of unchecked_bitwise_cast. But we would
need to still use unchecked_bitwise_cast in the case of load [copy]
because otherwise we would lose the original value--unchecked_value_cast
forwards ownership, and not all casts can be reversed (because they may
narrow).
For now, just bail out in the face of these complex load [take]s.
When optimizing an enum `store` to an `alloc_stack`, require that all uses are in the same block.
Otherwise it could be a `switch_enum` of an optional where the none-case does not have a destroy of the enum value.
After transforming such an `alloc_stack`, the value would leak in the none-case block.
It fixes the same OSSA verification error as done for TempRValueOpt in a previous commit.
Otherwise they will never compare the same. Also restore the test that was
updated for the previous commit to its old state. I did this to ensure that each
commit would compile successfully and so that I could show the test associated
with this commit.
The reason why I am doing this is that we are going to be enabling lexical
lifetimes early in the pipeline so that I can use it for the move operator's
diagnostics.
To make it easy for passes to know whether or not they should support lexical
lifetimes, I included a query on SILOptions called
supportsLexicalLifetimes. This will return true if the pass (given the passed in
option) should insert the lexical lifetime flag. This ensures that passes that
run in both pipelines (e.x.: AllocBoxToStack) know whether or not to set the
lexical lifetime flag without having to locally reason about it.
This is just chopping off layers of a larger patch I am upstreaming.
NOTE: This is technically NFC since it leaves the default alone of not inserting
lexical lifetimes at all.
Thanks to the lack of critical edges in SIL, if a block B dominated by P
has a successor S which is not dominated by P, then B must have only a
single successor. Used this fact to replace a loop over successors to a
call to getSinglePredecessor.
Also added an assertion that, in the notation above, B is dominated by
P.
Previously, it was asserted that any single-block allocation which had
valid memory after all instructions in the block were visited terminated
in unreachable. That assertion was false--while all paths through that
block must end in unreachable, the block itself need not be terminated
with an unreachable inst. Here, that is corrected by walking forward
from the block until blocks terminated by unreachables or blocks not
dominated by the block containing the alloc_stack are reached. In the
first case, the lifetime is ended just before the unreachable inst. In
the second, the lifetime is ended just before the branch to such a
successor block (i.e. just before the branch to a block which is not
dominated by the block containing the alloc_stack).
Previously, if it was determined that a proactive phi was unnecessary,
it was removed, along with the phis for the lifetime and the original
value of which the proactive phi was a copy. The uses of only one of
the three phis (namely, the proactive phi) was RAUW undef. In the case
where the only usage of the phi was to branch back to the block that
took the phi as an argument, that was a problem. Here, that is fixed by
giving all three phis the same treatment. To avoid duplicating code,
that treatment is pulled out into a new lambda.
This was exposed by adding lifetime versions of some OSSA versions of
mem2reg tests that had been missed previously.
Previously, the flag was a LangOptioins. That didn't make much sense because
this isn't really a user-facing behavior. More importantly, as a member
of that type type it couldn't be accessed when setting up pass
pipelines. Here, the flag is moved to SILOptions.
Previously, Mem2Reg would delete write-only alloc_stacks. That is
incorrect for lexical alloc_stacks--if a var was assigned but never
read, we still want to keep it alive for the duration of that var's
lexical scope.
Previously, the lexical borrow scopes that were introduced to replace
lexical stack allocations were tied to the uses of the stored value.
That meant that the borrow scope could be both too long and also too
short. It could be too long if there were uses of the value after the
dealloc stack and it would be too short if there were not uses of the
value while the value was still stored into the stack allocation.
Here, instead, the lexical borrow scopes are tied to the storage of a
value into the stack allocation. A value's lexical borrow scope begins
when the storage is initialied with the value; its lexical borrow scope
ends when the storage is deinitialized. That corresponds to the range
during which a var in the Swift source has a particular value assigned
to it.
Mem2Reg's implementation is split into a few steps:
(1) visiting the instructions in a basic block which contains all uses
of an alloc_stack
(2.a) visiting the instructions in each basic block which contains a use
of the alloc_stack
(2.b) adding phis
(2.c) using the last stored values as arguments to the new outgoing phi
arguments
(2.c) replacing initial uses of the storage with the new incoming phi
arguments
And here, (1) amounts to a special case of (2.a).
During (1) and (2.a):
(a) lexical borrow scopes are begun after store instructions for the
values that were stored
(b) when possible, lexical borrow scopes are ended before instructions
that deinitialize memory
- destroy_addr
- store [assign]
- load [take]
For (1), that is enough to create valid borrow scopes.
For (2), there are two complications:
(a) Borrow scopes that are begun may not be ended (when visiting a
single block's instructions).
For example, when visiting
bb1:
store %instance to [init] %addr
br bb2
a borrow scope is started after the store but cannot be ended.
(b) There may not be enough information available to end borrow scopes
when visiting instructions that would typically introduce borrow
scopes.
For example, when visiting
bb2:
%copy = load [copy] %addr
%instance = load [take] %addr
br bb3
there is not enough information available to end the borrow scope
that should be ended before the load [take]
To resolve these issues, both sorts of instructions are tracked. For
(a), in StackAllocationPromoter::initializationPoints. For (b), in
StackAllocationPromoter::deinitializationPoints. Finally, a new
step is added:
(2.d) StackAllocationPromoter::endLexicalLifetimes does a forward CFG
walk starting from the out-edge of each of the blocks which began
but did not end lexical lifetimes. At an out-edge, we do a check
regarding unreachables is done, and we may end the borrow scope.
Otherwise, the walk continues to the in-edge of each successor.
At an in-edge, we look for an instruction from (b) (in
unprecedentedDeinitializations) above. If one is found, then we
end the borrow scope before that instruction. Otherwise, the walk
continues to the out-edge of the block.
The ubuntu 18.04 Linux builder fails to build when the SmallVector does
not include the number of elements. This is a known issue and is
incorrect, but that is the version of clang on that OS.
The definition of SmallVector is already available transitively since
there are values of that type, I've just made it explicit.
llvm::SmallVector has a default parameter for the number of elements
while swift::SmallVector doesn't. I've gone ahead and made it explicitly
use the llvm::SmallVector to receive that.
SILGen turns vars into alloc_boxes. When possible, AllocBoxToStack
turns those into alloc_stacks. In order to preserve the lexical
lifetime of those vars, the alloc_stacks are annotated with the
[lexical] attribute. When Mem2Reg runs, it promotes the loads from
and stores into those alloc_stacks to uses of registers. In order to
preserve the lexical lifetime during that transformation, lexical borrow
scopes must be introduces that encode the same lifetime as the
alloc_stacks did. Here, that is done.
This patch replace all in-memory objects of DebugValueAddrInst with
DebugValueInst + op_deref, and duplicates logics that handles
DebugValueAddrInst with the latter. All related check in the tests
have been updated as well.
Note that this patch neither remove the DebugValueAddrInst class nor
remove `debug_value_addr` syntax in the test inputs.
With such alloc_stack we can pass over-consumed values to a
proactively added phi. Even though such a SIL does not have
issues at runtime, they raise an ownership verification error.
Mem2Reg creates phis proactively that may be unnecessary.
Unnecessary phis are those without uses or operands to other
proactive phis that are unnecessary.
Even though this does not translate to a real issue at runtime, we can
see ownership verification errors. This PR identifies such phis and deletes them.
Track in-use iterators and update them both when instructions are
deleted and when they are added.
Safe iteration in the presence of arbitrary changes now looks like
this:
for (SILInstruction *inst : deleter.updatingRange(&bb)) {
modify(inst);
}
Fix innumerable latent bugs with iterator invalidation and callback invocation.
Removes dead code earlier and chips away at all the redundant copies the compiler generates.
We previously were computing this eagerly meaning that if we did not actually
need the DomTreeLevel map, we would calculate it and additionally incur a
relatively large malloc for the DenseMap (which stores by default IIRC 64 key
value pairs).
Now, we do it lazily ensuring that we only compute this if we actually need it
(when promoting non-single block stack allocations).
We have for a long time talked about creating a scope like data structure for
use in the SILOptimizer. The discussion was whether or not to reuse the
infrastructure in SILGen that does this already. There were concerns about doing
so since the code in the SILOptimizer and SILGen can work differently.
With that in mind, I added a small AssertingScope class and built on top of that
a composition SIL level class called SILOptScope that one can use to add various
cleanups. One is able to both destructively pop at end of scope and pop along
early exits.
At an implementation level, I kept it simple and:
1. Represented a scope as a stack of Optional<Cleanup> which are just a wrapper
around a std::function. The Optional is so that we can invalidate a cleanup.
2. Based all of these scopes around the idea that the user of the scope must
invalidate the scope by hand. If not, the scope object will assert at the end
of its RAII scope.
3. Rather than creating a whole class hierarchy, I just used std::function
closures to keep things simple.
... with a fix for a non-assert build crash: I used the wrong ilist type for SlabList. This does not explain the crash, though. What I think happened here is that llvm miscompiled and put the llvm_unreachable from the Slab's deleteNode function unconditionally into the SILModule destructor.
Now by using simple_ilist, there is no need for a deleteNode at all.
This is a pretty old style of code that we are trying to move away from.
Specifically, we use to always store a global SILBuilder and manually manipulate
its insertion point/other state when we moved places. This was very bugprone so
we instead now create new SILBuilders when we want to create instructions at a
new insertion point and pass down a SILBuilderContext for global state (like the
list of inserted instructions/etc) to each SILBuilder.
I went through and updated all of the code to now follow this pattern.
This code organization already existed in the file in content, but all of the
routines were mixed together. This splits the code by topic providing to the eye
easy code locality when reading.
This is code that was written at the very beginning of Swift and has some early
style remaining interspirsed with newer swift style code. Since I am going to be
touching this and I don't expect other people to touch it for a while, I am
fixing up the formatting/reorganizing rather than leaving the style inconsistent.
This PR simplifies the handling of store [assign] in Mem2Reg by always
converting it to a store [init].
Also fixes an edge case where store [assign] in a non entry block was
not the first store of an alloc_stack with uses in multiple blocks.
This removes the ambiguity when casting from a SingleValueInstruction to SILNode, which makes the code simpler. E.g. the "isRepresentativeSILNode" logic is not needed anymore.
Also, it reduces the size of the most used instruction class - SingleValueInstruction - by one pointer.
Conceptually, SILInstruction is still a SILNode. But implementation-wise SILNode is not a base class of SILInstruction anymore.
Only the two sub-classes of SILInstruction - SingleValueInstruction and NonSingleValueInstruction - inherit from SILNode. SingleValueInstruction's SILNode is embedded into a ValueBase and its relative offset in the class is the same as in NonSingleValueInstruction (see SILNodeOffsetChecker).
This makes it possible to cast from a SILInstruction to a SILNode without knowing which SILInstruction sub-class it is.
Casting to SILNode cannot be done implicitly, but only with an LLVM `cast` or with SILInstruction::asSILNode(). But this is a rare case anyway.
This makes it easier to understand conceptually why a ValueOwnershipKind with
Any ownership is invalid and also allowed me to explicitly document the lattice
that relates ownership constraints/value ownership kinds.
