Recognize lexical borrows as nested when their borrowee's guaranteed
reference roots are all lexical borrows.
Addresses the following regressions
Breadcrumbs.MutatedUTF16ToIdx.Mixed 188 882 +369.1% **0.21x**
Breadcrumbs.MutatedIdxToUTF16.Mixed 230 926 +302.6% **0.25x**
seen when enabling lexical lifetimes in the standard library.
In preparation for adding OwnershipLiveness.
Rename Simple LiveRangeSummary to LiveRangeSummary.
Add initializeDefNode helpers to avoid confusion about the argument
type.
Add defBegin/defEnd iterators in MultiDefPrunedLiveness.
These APIs are essential for complete OSSA liveness analysis. The
existing ad-hoc OSSA logic always misses some of the cases handled by
these new utilities. We need to start replacing that ad-hoc logic with
new utilities built on top of these APIs to define away potential
latent bugs.
Add FIXMEs to the inverse API: visitAdjacentBorrowsOfPhi. It should
probably be redesigned in terms of these new APIs.
Factors a mess of code in MemAccessUtils to handle forwarding
instruction types into a simpler utility. This utility is also needed
for ownership APIs, which need to be extended to handle these cases.
Add TermInst::forwardedOperand.
Add SILArgument::forwardedTerminatorResultOperand. This API will be
moved into a proper TerminatorResult abstraction.
Remove getSingleTerminatorOperand, which could be misused because it's
not necessarilly forwarding ownership.
Remove the isTransformationTerminator API, which is not useful or well
defined.
Rewrite several instances of complex logic to handle block arguments
with the simple terminator result API. This defines away potential
bugs where we don't detect casts that perform implicit conversion.
Replace uses of the SILPhiArgument type and code that explicitly
handle block arguments. Control flow is irrelevant in these
situations. SILPhiArgument needs to be deleted ASAP. Instead, use
simple APIs like SILArgument::isTerminatorResult(). Eventually this
will be replaced by a TerminatorResult type.
Computing simple liveness is distinct from computing transitive
liveness. But for safety and consistency, always handle the first
level of liveness transitively. This way, computeSimple can be used on
guaranteed values that need OSSA lifetime fixup.
Simple liveness just means that *inner* borrow and address scopes are
assumed to be complete.
This utility is still only used conservatively because OSSA lifetime
completion is not yet enabled. But, in the future, computeSimple can
be used in the common case.
Start using consistent terminolfy in ownership utils.
A transitive use set follows transitive uses within an ownership
lifetime. It does not rely on complete inner scopes. An extended use
set is not necessarilly transitive but does look across
lifetime-ending uses: copies of owned values and/or reborrows of
guaranteed values. Whether lifetime extension refers to copies or
reborrow is context dependent.
The API for computing simple liveness now returns a
SimpleLiveRangeSummary. Callers need to decide how to handle reborrows
and pointer escapes. If either condition exists then the resulting
liveness does not necessarily encapsulate the definition's ownership.
Fixes some number of latent bugs w.r.t. liveness clients.
This fixes ScopedAddressValue::computeLiveness in unreachable code scenarios.
For example:
%storeBorrow = store_borrow %_ to %adr
%loadBorrow = load_borrow %storeBorrow
apply %f(%loadBorrow) : $@convention(thin) (...) -> Never
Now recursively process uses of load_borrow as if they are address
uses.
Ultimately, this would be more efficiently handled by a recursive
lifetime completion utility which would fixup the load_borrow scope
before computing the store_borrow liveness.
Fixes rdar://99874173: unreachable assert "No user in LiveWithin block"
First restore the basic PrunedLiveness abstraction to its original
intention. Move code outside of the basic abstraction that polutes the
abstraction and is fundamentally wrong from the perspective of the
liveness abstraction.
Most clients need to reason about live ranges, including the def
points, not just liveness based on use points. Add a PrunedLiveRange
layer of types that understand where the live range is
defined. Knowing where the live range is defined (the kill set) helps
reliably check that arbitrary points are within the boundary. This
way, the client doesn't need to be manage this on its own. We can also
support holes in the live range for non-SSA liveness. This makes it
safe and correct for the way liveness is now being used. This layer
safety handles:
- multiple defs
- instructions that are both uses and defs
- dead values
- unreachable code
- self-loops
So it's no longer the client's responsibility to check these things!
Add SSAPrunedLiveness and MultiDefPrunedLiveness to safely handle each
situation.
Split code that I can't figure out into
DiagnosticPrunedLiveness. Hopefully it will be deleted soon.
Fix the utilities used by LexicalDestroyHoisting that finds all uses
to report a "PointerEscape". We can't rely on lifetime analysis when
such a use is present.
Add ScopedAddressOperand and ScopedAddressValue abstraction utilities
Introduce verification for store_borrow to validate its uses are correctly enclosed in their scope.
Include end_borrow/end_access as implicit uses while validating a borrow introducer
Add flow sensitive verifier rule for store_borrow/end_borrow pair
Make sure store_borrow is always to an alloc_stack
Make sure uses to store borrow location are via its return address only
Andy some time ago already created the new API but didn't go through and update
the old occurences. I did that in this PR and then deprecated the old API. The
tree is clean, so I could just remove it, but I decided to be nicer to
downstream people by deprecating it first.
The new utility, given an phi, visits all adjacent phis (i.e. arguments
to the same block) which are (potentially iterated) reborrows of a value
reaching the given phi.
These instructions have the following attributes:
1. copyably_to_moveonlywrapper takes in a 'T' and maps it to a '@moveOnly
T'. This is semantically used when initializing a new moveOnly binding from a
copyable value. It semantically destroys its input @owned value and returns a
brand new independent @owned @moveOnly value. It also is used to convert a
trivial copyable value with type 'Trivial' into an owned non-trivial value of
type '@moveOnly Trivial'. If one thinks of '@moveOnly' as a monad, this is how
one injects a copyable value into the move only space.
2. moveonlywrapper_to_copyable takes in a '@moveOnly T' and produces a new 'T'
value. This is a 'forwarding' instruction where at parse time, we only allow for
one to choose it to be [owned] or [guaranteed].
* moveonlywrapper_to_copyable [owned] is used to signal the end of lifetime of
the '@moveOnly' wrapper. SILGen inserts these when ever a move only value has
its ownership passed to a situation where a copyable value is needed. Since it
is consuming, we know that the no implicit copy checker will ensure that if we
need a copy for it, the program will emit a diagnostic.
* moveonlywrapper_to_copyable [guaranteed] is used to pass a @moveOnly T value
as a copyable guaranteed parameter with type 'T' to a function. In the case of
using no-implicit-copy checking this is always fine since no-implicit-copy is a
local pattern. This would be an error when performing no escape
checking. Importantly, this instruction also is where in the case of an
@moveOnly trivial type, we convert from the non-trivial representation to the
trivial representation.
Some important notes:
1. In a forthcoming commit, I am going to rebase the no implicit copy checker on
top of these instructions. By using '@moveOnly' in the type system, we can
ensure that later in the SIL pipeline, we can have optimizations easily ignore
the code.
2. Be aware of is that due to SILGen only emitting '@moveOnly T' along immediate
accesses to the variable and always converts to a copyable representation when
calling other code, we can simply eliminate from the IR all moveonly-ness from
the IR using a lowering pass (that I am going to upstream). In the evil scheme
we are accomplishing here, we perform lowering of trivial values right after
ownership lowering and before diagnostics to simplify the pipeline.
On another note, I also fixed a few things in SILParsing around getASTType() vs
getRawASTType().
This fix unblocks unrelated optimizer commits. A unit test will be
introduced with those commits.
The RAUW utility does not correctly handle reborrows. It is in the
middle of being rewritten. For now, simply bail out since this isn't
an important case to optimize.
Add NonTypeDependentOperandToValue predicate for composability.
Add a getNonTypeDependentOperandValues(), which can be used as a functor.
The skipTypeDependentOperands parameter complicated the API.
Previously, visitTransitiveEndBorrows took BorrowedValues. However,
there is at least one kind of borrow--namely,
unchecked_ownership_conversion insts--that is not currently permitted by
the BorrowedValue API. The long term fix is to make BorrowedValue
handle such instructions. For now, change visitTransitiveEndBorrows to
take SILValues so that unchecked_ownership_conversion can be passed to
the API.
rdar://87985420
Replaced findInnerTransitiveGuaranteeedUsesOfBorrowedValue with
findExtendedUsesOfSimpleBorrowedValue. Starting from a borrowed value,
it finds all extended (i.e., seeing through copies) uses of the borrow
and its projections within the simple (i.e. without considering
reborrowing) borrow scope.
More bugs related to dead-end blocks and OSSA lifetimes that aren't
well formed because of that.
Independently, I'm working on prohibiting ill-formed OSSA even in the
presence of dead-end blocks. That makes these workarounds unnecessary,
but we urgently need a narrow fix here.
SemanticARCOptVisitor::performGuaranteedCopyValueOptimization was
converting this SIL
%borrow = begin_borrow %copiedValue
%copy = copy_value %borrow
%borrowCopy = begin_borrow %copy
end_borrow %borrow
end_borrow %borrowCopy
destroy_value %copy
// something something
unreachable
into
%borrow = begin_borrow %copiedValue
%innerBorrow = begin_borrow %borrow
end_borrow %borrow
end_borrow %innerBorrow
// something something
unreachable
Dead-end blocks are simply irrelevant for this
optimization. Unfortunately, there were multiple layers of attempted
workarounds that were hiding the real problem, except in rare cases.
Thanks Nate Chandler for reducing the test.
During copy propagation (for which -enable-copy-propagation must still
be passed), also try to shrink borrow scopes by hoisting end_borrows
using the newly added ShrinkBorrowScope utility.
Allow end_borrow instructions to be hoisted over instructions that are
not deinit barriers for the value which is borrowed. Deinit barriers
include uses of the value, loads of memory, loads of weak references
that may be zeroed during deinit, and "synchronization points".
rdar://79149830
This instruction is similar to a copy_addr except that it marks a move of an
address that has to be checked. In order to keep the memory lifetime verifier
happy, the semantics before the checker runs are the mark_unresolved_move_addr is
equivalent to copy_addr [init] (not copy_addr [take][init]).
The use of this instruction is that Mandatory Inlining converts builtin "move"
to a mark_unresolved_move_addr when inlining the function "_move" (the only
place said builtin is invoked).
This is then run through a special checker (that is later in this PR) that
either proves that the mark_unresolved_move_addr can actually be a move in which
case it converts it to copy_addr [take][init] or if it can not be a move, emit
an error and convert the instruction to a copy_addr [init]. After this is done
for all instructions, we loop back through again and emit an error on any
mark_unresolved_move_addr that were not processed earlier allowing for us to
know that we have completeness.
NOTE: The move kills checker for addresses is going to run after Mandatory
Inlining, but before predictable memory opts and friends.
The reason why I am doing this is that currently checked_cast_br of an AnyObject
may return a different value due to boxing. As an example, this can occur when
performing a checked_cast_br of a __SwiftValue(AnyHashable(Klass())).
To model this in SIL, I added to OwnershipForwardingMixin a bit of information
that states if the instruction directly forwards ownership with a default value
of true. In checked_cast_br's constructor though I specialize this behavior if
the source type is AnyObject and thus mark the checked_cast_br as not directly
forwarding its operand. If an OwnershipForwardingMixin directly forwards
ownership, one can assume that if it forwards ownership, it will always do so in
a way that ensures that each forwarded value is rc-identical to whatever result
it algebraically forwards ownership to. So in the context of checked_cast_br, it
means that the source value is rc-identical to the argument of the success and
default blocks.
I added a verifier check to CheckedCastBr that makes sure that it can never
forward guaranteed ownership and have a source type of AnyObject.
In SemanticARCOpts, I modified the code that builds extended live ranges of
owned values (*) to check if a OwnershipForwardingMixin is directly
forwarding. If it is not directly forwarding, then we treat the use just as an
unknown consuming use. This will then prevent us from converting such an owned
value to guaranteed appropriately in such a case.
I also in SILGen needed to change checked_cast_br emission in SILGenBuilder to
perform an ensurePlusOne on the input operand (converting it to +1 with an
appropriate cleanup) if the source type is an AnyObject. I found this via the
verifier check catching this behavior from SILGen when compiling libswift. This
just shows how important IR verification of invariants can be.
(*) For those unaware, SemanticARCOpts contains a model of an owned value and
all forwarding uses of it called an OwnershipLiveRange.
rdar://85710101
