It was originally convenient for exclusivity optimization to treat
boxes specially. We wanted to know that the 'Box' kind was always
uniquely identified. But that's not really important. And now that
AccessedStorage is being used more generally, the inconsistency is
problematic.
A consistent model is also must easier to understand and explain.
This also make the implementation of the utility simpler and more powerful.
Functional changes:
isRCIdentical will look through mark_dependence and mark_uninitialized.
findReferenceRoot is used consistently everywhere increasing analysis precision.
Generalize the AccessUseDefChainCloner in MemAccessUtils. It was
always meant to work this way, just needed a client.
Add a new API AccessUseDefChainCloner::canCloneUseDefChain().
Add a bailout for begin_borrow and mark_dependence. Those
projections may appear on an access path, but they can't be
individually cloned without compensating.
Delete InteriorPointerAddressRebaseUseDefChainCloner.
Add a check in OwnershipRAUWHelper for canCloneUseDefChain.
Add test cases for begin_borrow and mark_dependence.
AccessPath was treating init_enum_data_addr as an address base, which
is not ideal. It should be able to identify the underlying enum object
as the base. This issue was caught by LoadBorrowImmutabilityChecker
during SIL verification.
Instead handle init_enum_data_addr as a access projection that does
not affect the access path. I expect this SIL pattern to disappear
with SIL opaque values, but it still needs to be handled properly
after lowering addresses.
Functionality changes:
- any user of AccessPath now sees enum initialization stores as writes
to the underlying enum object
- SILGen now generates begin/end access markers for enum
initialization patterns. (Originally, we did not "see through"
init_enum_data_addr because we didn't want to generate these
markers, but that behavior was inconsistent and problematic).
Fixes rdar://70725514 fatal error encountered during compilation;
Unknown instruction: init_enum_data_addr)
For class storage AccessedStorage is now close to what some passes use
for RC identity, but it still does not look past wrapping references
in an Optional.
Compute 'isLet' from the VarDecl that is available when constructing
AccessedStorage so we don't need to recover the VarDecl for the base
later.
This generally makes more sense and is more efficient, but it will be
necessary when we look past class casts when finding the reference root.
Add AccesssedStorage::compute and computeInScope to mirror AccessPath.
Allow recovering the begin_access for Nested storage.
Adds AccessedStorage.visitRoots().
Things that have come up recently but are somewhat blocked on this:
- Moving AccessMarkerElimination down in the pipeline
- SemanticARCOpts correctness and improvements
- AliasAnalysis improvements
- LICM performance regressions
- RLE/DSE improvements
Begin to formalize the model for valid memory access in SIL. Ignoring
ownership, every access is a def-use chain in three parts:
object root -> formal access base -> memory operation address
AccessPath abstracts over this path and standardizes the identity of a
memory access throughout the optimizer. This abstraction is the basis
for a new AccessPathVerification.
With that verification, we now have all the properties we need for the
type of analysis requires for exclusivity enforcement, but now
generalized for any memory analysis. This is suitable for an extremely
lightweight analysis with no side data structures. We currently have a
massive amount of ad-hoc memory analysis throughout SIL, which is
incredibly unmaintainable, bug-prone, and not performance-robust. We
can begin taking advantage of this verifably complete model to solve
that problem.
The properties this gives us are:
Access analysis must be complete over memory operations: every memory
operation needs a recognizable valid access. An access can be
unidentified only to the extent that it is rooted in some non-address
type and we can prove that it is at least *not* part of an access to a
nominal class or global property. Pointer provenance is also required
for future IRGen-level bitfield optimizations.
Access analysis must be complete over address users: for an identified
object root all memory accesses including subobjects must be
discoverable.
Access analysis must be symmetric: use-def and def-use analysis must
be consistent.
AccessPath is merely a wrapper around the existing accessed-storage
utilities and IndexTrieNode. Existing passes already very succesfully
use this approach, but in an ad-hoc way. With a general utility we
can:
- update passes to use this approach to identify memory access,
reducing the space and time complexity of those algorithms.
- implement an inexpensive on-the-fly, debug mode address lifetime analysis
- implement a lightweight debug mode alias analysis
- ultimately improve the power, efficiency, and maintainability of
full alias analysis
- make our type-based alias analysis sensistive to the access path
Distinguish ref_tail_addr storage from the other storage classes.
We didn't have this originally because be don't expect a begin_access
to directly operate on tail storage. It could occur after inlining, at
least with static access markers. More importantly it helps ditinguish
regular formal accesses from other unidentified access, so we probably
should have always had this.
At any rate, it's particularly important when AccessedStorage is
generalized to arbitrary memory access.
The immediate motivation is to add an AccessPath utility, which will
need to distinguish tail storage.
In the process, rewrite AccessedStorage::isDistinct. This could have a
large positive impact on exclusivity performance.
Prepare to reuse this visitor for an AccessPath utility.
Remove visitIncomplete. Add visitCast and visitPathComponent.
Handle phis in a separate visitor. This simplifies the main
visitor. In the long-term, we may be able to eliminate the pointer-phi
visitor entirely. For now, this lets us enforce that all phi paths
follow the same access path.
For use outside access enforcement passes.
Add isUniquelyIdentifiedAfterEnforcement.
Rename functions for clarity and generality.
Rename isUniquelyIdentifiedOrClass to isFormalAccessBase.
Rename findAccessedStorage to identifyFormalAccess.
Rename findAccessedStorageNonNested to findAccessedStorage.
Part of generalizing the utility for use outside the access
enforcement passes.
Assertion failed:
(accessedAddress == getAccessedAddress(accessedAddress) &&
"caller must find the address root"), function isLetAddress,
file /Users/rjmccall/dev/swift/swift/lib/SIL/Utils/MemAccessUtils.cpp,
line 63.
Teach the getAccessedAddress utility to iterate through nested access
markers with projections interposed.
Fixes <rdar://problem/61464370>
Crash in SILOptimizer/access_marker_verify.swift
The API was accidentally undefined, presumably because I checked in
the wrong code or there was a bad merge. The API will be used by
upcoming commits.
Meanwhile, getAccessedAddress was not stripping access markers, which
means some analysis may have been too conservative.
This fix could expose issues by making existing analyses more effective.
The only reason why BranchPropagatedUser existed was because early on in SIL, we
weren't sure if cond_br should be able to handle non-trivial values in
ossa. Now, we have reached the point where we have enough experience to make the
judgement that it is not worth having in the representation due to it not
holding its weight.
Now that in ToT we have banned cond_br from having non-trivial operands in ossa,
I can just eliminate BranchPropagatedUser and replace it with the operands that
we used to construct them!
A few notes:
1. Part of my motiviation in doing this is that I want to change LiveRange to
store operands instead of instructions. This is because we are interested in
being able to understand the LiveRange at a use granularity in cases where we
have multiple operands. While doing this, I discovered that I needed
SILInstructions to use the Linear Lifetime Checker. Then I realized that now was
the time to just unwind BranchPropagatedUser.
2. In certain places in SemanticARCOpts, I had to do add some extra copies to
transform arrays of instructions from LiveRange into their operand form. I am
going to remove them in a subsequent commit when I change LiveRange to work on
operands. I am doing this split to be incremental.
3. I changed isSingleInitAllocStack to have an out array of Operand *. The only
user of this code is today in SemanticARCOpts and this information is fed to the
Linear Lifetime Checker, so I needed to do it.
This is in prepration for other bug fixes.
Clarify the SIL utilities that return canonical address values for
formal access given the address used by some memory operation:
- stripAccessMarkers
- getAddressAccess
- getAccessedAddress
These are closely related to the code in MemAccessUtils.
Make sure passes use these utilities consistently so that
optimizations aren't defeated by normal variations in SIL patterns.
Create an isLetAddress() utility alongside these basic utilities to
make sure it is used consistently with the address corresponding to
formal access. When this query is used inconsistently, it defeats
optimization. It can also cause correctness bugs because some
optimizations assume that 'let' initialization is only performed on a
unique address value.
Functional changes to Memory Behavior:
- An instruction with side effects now conservatively still has side
effects even when the queried value is a 'let'. Let values are
certainly sensitive to side effects, such as the parent object being
deallocated.
- Return the correct MemBehavior for begin/end_access markers.
If the lifetime of a value copied out of a let property in a class is dominated by the lifetime
of the base object it was copied from, then we don't need to copy at all, since the value of the
`let` is guaranteed as long as the base object is. `let` globals are in turn always guaranteed.
rdar://problem/54531607
The name doesn't mean "memory access" colloquially. It specifically
refers to memory operations that take part in formal access scopes
marked by begin_access/end_access instructions. If you aren't working
with those instructions it really doesn't make sense to use it.
Add `llvm_unreachable` to mark covered switches which MSVC does not
analyze correctly and believes that there exists a path through the
function without a return value.
1. During identifyAccess, determine if there are either any
identical accesses or an accesses that aren't already marked
no_nested_storage. If there are neither, then skip the subsequent
data flow analysis.
2. In the new StorageSet, indicate whether identical storage was
seen elsewhere in the function. During dataflow, only add an access
to the out-of-scope access set if was marked as having identical
storage with another access.
3. During data flow, don't track in scope conflicts for
instructions already marked [no_nested_conflict].
- code simplification critical for comprehension
- substantially improves the overhead of AccessedStorage comparison
- as a side effect improves precision of analysis in some cases
AccessedStorage is meant to be an immutable value type that identifies
a storage location with minimal representation. It is used in many global
interprocedural data structures.
The RefElementAddress instruction that it was derived from does not
contribute to the uniqueness of the storage location. It doesn't
belong here. It was being used to create a ProjectionPath, which is an
extremely inneficient way to compare access paths.
Just delete all the code related to that extra field.
Cleaning up in preparation for making changes that improve
compile-time issues in AccessEnforcementOpts.
This is a simple but important enum with a handful of cases. The cases
need to be easily referenced from the header. Don't define them in a
separate .def. Remove the visitor biolerplate because it doesn't serve
any purpose.
This enum is meant to be used with covered switches. The enum cases do
not have their own types, so there's no performance reason to use a
Visitor pattern.
It should not be possible to add a case to this enum without carefully
considering the impact on the encoding of this class and the impact on
each and every one of the uses. We always want covered switches at the
use sites.
This is a major improvement in readability and usability both in the
definition of the class and in the one place where a visitor was used.
This adds a mostly flow-insensitive analysis that runs before the
dominator-based transformations. The analysis is simple and efficient
because it only needs to track data flow of currently in-scope
accesses. The original dominator tree walk remains simple, but it now
checks the flow insensitive analysis information to determine general
correctness. This is now correct in the presence of all kinds of nested
static and dynamic nested accesses, call sites, coroutines, etc.
This is a better compromise than:
(a) disabling the pass and taking a major performance loss.
(b) converting the pass itself to full-fledged data flow driven
optimization, which would be more optimal because it could remove
accesses when nesting is involved, but would be much more expensive
and complicated, and there's no indication that it's useful.
The new approach is also simpler than adding more complexity to
independently handle to each of many issues:
- Nested reads followed by a modify without a false conflict.
- Reads nested within a function call without a false conflict.
- Conflicts nested within a function call without dropping enforcement.
- Accesses within a generalized accessor.
- Conservative treatment of invalid storage locations.
- Conservative treatment of unknown apply callee.
- General analysis invalidation.
Some of these issues also needed to be considered in the
LoopDominatingAccess sub-pass. Rather than fix that sub-pass, I just
integrated it into the main pass. This is a simplification, is more
efficient, and also handles nested loops without creating more
redundant accesses. It is also generalized to:
- hoist non-uniquely identified accesses.
- Avoid unnecessarily promoting accesses inside the loop.
With this approach we can remove the scary warnings and caveats in the
comments.
While doing this I also took the opportunity to eliminate quadratic
behavior, make the domtree walk non-recursive, and eliminate cutoff
thresholds.
Note that simple nested dynamic reads to identical storage could very
easily be removed via separate logic, but it does not fit with the
dominator-based algorithm. For example, during the analysis phase, we
could simply mark the "fully nested" read scopes, then convert them to
[static] right after the analysis, removing them from the result
map. I didn't do this because I don't know if it happens in practice.
This is the first in a sequence of patches that implement various optimizations
to transform load [copy] into load_borrow.
The optimization works by looking for a load [copy] that:
1. Only has destroy_value as consuming users. This implies that we do not need
to pass off the in memory value at +1 and that we can use a +0 value.
2. Is loading from a memory location that is never written to or only written to
after all uses of the load [copy].
and then RAUW the load [copy] with a load_borrow and convertes the destroy_value
to end_borrow.
NOTE: I also a .def file for AccessedStorage so we can do visitors over the
kinds. The reason I want to do this is to ensure that people update these
optimizations if we add new storage kinds.
The SIL verifier was asserting while attempting to prove that all
formal accesses are well-formed. This is necessary because
unrecognized access could lead to invalid whole-module optimization.
We would like to eliminate address-phis from SIL, but there are still
optimizer passes that produce them. For now, the best we can do is
hope to recover the original base of each phi value and prove they are
actually the same value. They should be because the optimizer
produced the phi through block cloning.
Fixes <rdar://problem/46114512> SIL verification failed: Unknown
formal access pattern: storage
