AccessPath was treating init_enum_data_addr as an address base, which
is not ideal. It should be able to identify the underlying enum object
as the base. This issue was caught by LoadBorrowImmutabilityChecker
during SIL verification.
Instead handle init_enum_data_addr as a access projection that does
not affect the access path. I expect this SIL pattern to disappear
with SIL opaque values, but it still needs to be handled properly
after lowering addresses.
Functionality changes:
- any user of AccessPath now sees enum initialization stores as writes
to the underlying enum object
- SILGen now generates begin/end access markers for enum
initialization patterns. (Originally, we did not "see through"
init_enum_data_addr because we didn't want to generate these
markers, but that behavior was inconsistent and problematic).
Fixes rdar://70725514 fatal error encountered during compilation;
Unknown instruction: init_enum_data_addr)
For class storage AccessedStorage is now close to what some passes use
for RC identity, but it still does not look past wrapping references
in an Optional.
Compute 'isLet' from the VarDecl that is available when constructing
AccessedStorage so we don't need to recover the VarDecl for the base
later.
This generally makes more sense and is more efficient, but it will be
necessary when we look past class casts when finding the reference root.
Add AccesssedStorage::compute and computeInScope to mirror AccessPath.
Allow recovering the begin_access for Nested storage.
Adds AccessedStorage.visitRoots().
Things that have come up recently but are somewhat blocked on this:
- Moving AccessMarkerElimination down in the pipeline
- SemanticARCOpts correctness and improvements
- AliasAnalysis improvements
- LICM performance regressions
- RLE/DSE improvements
Begin to formalize the model for valid memory access in SIL. Ignoring
ownership, every access is a def-use chain in three parts:
object root -> formal access base -> memory operation address
AccessPath abstracts over this path and standardizes the identity of a
memory access throughout the optimizer. This abstraction is the basis
for a new AccessPathVerification.
With that verification, we now have all the properties we need for the
type of analysis requires for exclusivity enforcement, but now
generalized for any memory analysis. This is suitable for an extremely
lightweight analysis with no side data structures. We currently have a
massive amount of ad-hoc memory analysis throughout SIL, which is
incredibly unmaintainable, bug-prone, and not performance-robust. We
can begin taking advantage of this verifably complete model to solve
that problem.
The properties this gives us are:
Access analysis must be complete over memory operations: every memory
operation needs a recognizable valid access. An access can be
unidentified only to the extent that it is rooted in some non-address
type and we can prove that it is at least *not* part of an access to a
nominal class or global property. Pointer provenance is also required
for future IRGen-level bitfield optimizations.
Access analysis must be complete over address users: for an identified
object root all memory accesses including subobjects must be
discoverable.
Access analysis must be symmetric: use-def and def-use analysis must
be consistent.
AccessPath is merely a wrapper around the existing accessed-storage
utilities and IndexTrieNode. Existing passes already very succesfully
use this approach, but in an ad-hoc way. With a general utility we
can:
- update passes to use this approach to identify memory access,
reducing the space and time complexity of those algorithms.
- implement an inexpensive on-the-fly, debug mode address lifetime analysis
- implement a lightweight debug mode alias analysis
- ultimately improve the power, efficiency, and maintainability of
full alias analysis
- make our type-based alias analysis sensistive to the access path
Distinguish ref_tail_addr storage from the other storage classes.
We didn't have this originally because be don't expect a begin_access
to directly operate on tail storage. It could occur after inlining, at
least with static access markers. More importantly it helps ditinguish
regular formal accesses from other unidentified access, so we probably
should have always had this.
At any rate, it's particularly important when AccessedStorage is
generalized to arbitrary memory access.
The immediate motivation is to add an AccessPath utility, which will
need to distinguish tail storage.
In the process, rewrite AccessedStorage::isDistinct. This could have a
large positive impact on exclusivity performance.
Prepare to reuse this visitor for an AccessPath utility.
Remove visitIncomplete. Add visitCast and visitPathComponent.
Handle phis in a separate visitor. This simplifies the main
visitor. In the long-term, we may be able to eliminate the pointer-phi
visitor entirely. For now, this lets us enforce that all phi paths
follow the same access path.
For use outside access enforcement passes.
Add isUniquelyIdentifiedAfterEnforcement.
Rename functions for clarity and generality.
Rename isUniquelyIdentifiedOrClass to isFormalAccessBase.
Rename findAccessedStorage to identifyFormalAccess.
Rename findAccessedStorageNonNested to findAccessedStorage.
Part of generalizing the utility for use outside the access
enforcement passes.
Assertion failed:
(accessedAddress == getAccessedAddress(accessedAddress) &&
"caller must find the address root"), function isLetAddress,
file /Users/rjmccall/dev/swift/swift/lib/SIL/Utils/MemAccessUtils.cpp,
line 63.
Teach the getAccessedAddress utility to iterate through nested access
markers with projections interposed.
Fixes <rdar://problem/61464370>
Crash in SILOptimizer/access_marker_verify.swift
The API was accidentally undefined, presumably because I checked in
the wrong code or there was a bad merge. The API will be used by
upcoming commits.
Meanwhile, getAccessedAddress was not stripping access markers, which
means some analysis may have been too conservative.
This fix could expose issues by making existing analyses more effective.
The only reason why BranchPropagatedUser existed was because early on in SIL, we
weren't sure if cond_br should be able to handle non-trivial values in
ossa. Now, we have reached the point where we have enough experience to make the
judgement that it is not worth having in the representation due to it not
holding its weight.
Now that in ToT we have banned cond_br from having non-trivial operands in ossa,
I can just eliminate BranchPropagatedUser and replace it with the operands that
we used to construct them!
A few notes:
1. Part of my motiviation in doing this is that I want to change LiveRange to
store operands instead of instructions. This is because we are interested in
being able to understand the LiveRange at a use granularity in cases where we
have multiple operands. While doing this, I discovered that I needed
SILInstructions to use the Linear Lifetime Checker. Then I realized that now was
the time to just unwind BranchPropagatedUser.
2. In certain places in SemanticARCOpts, I had to do add some extra copies to
transform arrays of instructions from LiveRange into their operand form. I am
going to remove them in a subsequent commit when I change LiveRange to work on
operands. I am doing this split to be incremental.
3. I changed isSingleInitAllocStack to have an out array of Operand *. The only
user of this code is today in SemanticARCOpts and this information is fed to the
Linear Lifetime Checker, so I needed to do it.
This is in prepration for other bug fixes.
Clarify the SIL utilities that return canonical address values for
formal access given the address used by some memory operation:
- stripAccessMarkers
- getAddressAccess
- getAccessedAddress
These are closely related to the code in MemAccessUtils.
Make sure passes use these utilities consistently so that
optimizations aren't defeated by normal variations in SIL patterns.
Create an isLetAddress() utility alongside these basic utilities to
make sure it is used consistently with the address corresponding to
formal access. When this query is used inconsistently, it defeats
optimization. It can also cause correctness bugs because some
optimizations assume that 'let' initialization is only performed on a
unique address value.
Functional changes to Memory Behavior:
- An instruction with side effects now conservatively still has side
effects even when the queried value is a 'let'. Let values are
certainly sensitive to side effects, such as the parent object being
deallocated.
- Return the correct MemBehavior for begin/end_access markers.
If the lifetime of a value copied out of a let property in a class is dominated by the lifetime
of the base object it was copied from, then we don't need to copy at all, since the value of the
`let` is guaranteed as long as the base object is. `let` globals are in turn always guaranteed.
rdar://problem/54531607
The name doesn't mean "memory access" colloquially. It specifically
refers to memory operations that take part in formal access scopes
marked by begin_access/end_access instructions. If you aren't working
with those instructions it really doesn't make sense to use it.
Add `llvm_unreachable` to mark covered switches which MSVC does not
analyze correctly and believes that there exists a path through the
function without a return value.
1. During identifyAccess, determine if there are either any
identical accesses or an accesses that aren't already marked
no_nested_storage. If there are neither, then skip the subsequent
data flow analysis.
2. In the new StorageSet, indicate whether identical storage was
seen elsewhere in the function. During dataflow, only add an access
to the out-of-scope access set if was marked as having identical
storage with another access.
3. During data flow, don't track in scope conflicts for
instructions already marked [no_nested_conflict].
- code simplification critical for comprehension
- substantially improves the overhead of AccessedStorage comparison
- as a side effect improves precision of analysis in some cases
AccessedStorage is meant to be an immutable value type that identifies
a storage location with minimal representation. It is used in many global
interprocedural data structures.
The RefElementAddress instruction that it was derived from does not
contribute to the uniqueness of the storage location. It doesn't
belong here. It was being used to create a ProjectionPath, which is an
extremely inneficient way to compare access paths.
Just delete all the code related to that extra field.
Cleaning up in preparation for making changes that improve
compile-time issues in AccessEnforcementOpts.
This is a simple but important enum with a handful of cases. The cases
need to be easily referenced from the header. Don't define them in a
separate .def. Remove the visitor biolerplate because it doesn't serve
any purpose.
This enum is meant to be used with covered switches. The enum cases do
not have their own types, so there's no performance reason to use a
Visitor pattern.
It should not be possible to add a case to this enum without carefully
considering the impact on the encoding of this class and the impact on
each and every one of the uses. We always want covered switches at the
use sites.
This is a major improvement in readability and usability both in the
definition of the class and in the one place where a visitor was used.
This adds a mostly flow-insensitive analysis that runs before the
dominator-based transformations. The analysis is simple and efficient
because it only needs to track data flow of currently in-scope
accesses. The original dominator tree walk remains simple, but it now
checks the flow insensitive analysis information to determine general
correctness. This is now correct in the presence of all kinds of nested
static and dynamic nested accesses, call sites, coroutines, etc.
This is a better compromise than:
(a) disabling the pass and taking a major performance loss.
(b) converting the pass itself to full-fledged data flow driven
optimization, which would be more optimal because it could remove
accesses when nesting is involved, but would be much more expensive
and complicated, and there's no indication that it's useful.
The new approach is also simpler than adding more complexity to
independently handle to each of many issues:
- Nested reads followed by a modify without a false conflict.
- Reads nested within a function call without a false conflict.
- Conflicts nested within a function call without dropping enforcement.
- Accesses within a generalized accessor.
- Conservative treatment of invalid storage locations.
- Conservative treatment of unknown apply callee.
- General analysis invalidation.
Some of these issues also needed to be considered in the
LoopDominatingAccess sub-pass. Rather than fix that sub-pass, I just
integrated it into the main pass. This is a simplification, is more
efficient, and also handles nested loops without creating more
redundant accesses. It is also generalized to:
- hoist non-uniquely identified accesses.
- Avoid unnecessarily promoting accesses inside the loop.
With this approach we can remove the scary warnings and caveats in the
comments.
While doing this I also took the opportunity to eliminate quadratic
behavior, make the domtree walk non-recursive, and eliminate cutoff
thresholds.
Note that simple nested dynamic reads to identical storage could very
easily be removed via separate logic, but it does not fit with the
dominator-based algorithm. For example, during the analysis phase, we
could simply mark the "fully nested" read scopes, then convert them to
[static] right after the analysis, removing them from the result
map. I didn't do this because I don't know if it happens in practice.
This is the first in a sequence of patches that implement various optimizations
to transform load [copy] into load_borrow.
The optimization works by looking for a load [copy] that:
1. Only has destroy_value as consuming users. This implies that we do not need
to pass off the in memory value at +1 and that we can use a +0 value.
2. Is loading from a memory location that is never written to or only written to
after all uses of the load [copy].
and then RAUW the load [copy] with a load_borrow and convertes the destroy_value
to end_borrow.
NOTE: I also a .def file for AccessedStorage so we can do visitors over the
kinds. The reason I want to do this is to ensure that people update these
optimizations if we add new storage kinds.
The SIL verifier was asserting while attempting to prove that all
formal accesses are well-formed. This is necessary because
unrecognized access could lead to invalid whole-module optimization.
We would like to eliminate address-phis from SIL, but there are still
optimizer passes that produce them. For now, the best we can do is
hope to recover the original base of each phi value and prove they are
actually the same value. They should be because the optimizer
produced the phi through block cloning.
Fixes <rdar://problem/46114512> SIL verification failed: Unknown
formal access pattern: storage
This silences the instances of the warning from Visual Studio about not all
codepaths returning a value. This makes the output more readable and less
likely to lose useful warnings. NFC.
Sometions we don't have a projection path for each ref_elem_addr - fixes a compiler crash in Deferred in the Source Compatibility Suite
When checking if two storage locations are distinct from one another, play it safe in such a case.
Consider a class ‘C’ with distinct fields ‘A’ and ‘B’
And consider we are accessing C.A and C.B inside a loop
LICM well not hoist the exclusivity checking outside of the loop because isDistinctFrom(C.A, C.B) returns false.
This is because the helper function bails if isUniquelyIdentified returns false (which is the case in class kinds)
Same with all other potential access enforcement optimizations.
This PR resolves that
For now, the accessors have been underscored as `_read` and `_modify`.
I'll prepare an evolution proposal for this feature which should allow
us to remove the underscores or, y'know, rename them to `purple` and
`lettuce`.
`_read` accessors do not make any effort yet to avoid copying the
value being yielded. I'll work on it in follow-up patches.
Opaque accesses to properties and subscripts defined with `_modify`
accessors will use an inefficient `materializeForSet` pattern that
materializes the value to a temporary instead of accessing it in-place.
That will be fixed by migrating to `modify` over `materializeForSet`,
which is next up after the `read` optimizations.
SIL ownership verification doesn't pass yet for the test cases here
because of a general fault in SILGen where borrows can outlive their
borrowed value due to being cleaned up on the general cleanup stack
when the borrowed value is cleaned up on the formal-access stack.
Michael, Andy, and I discussed various ways to fix this, but it seems
clear to me that it's not in any way specific to coroutine accesses.
rdar://35399664
Now that SILGen change adds Unsafe access markers to addressors and
materializeForSet, we can use that as a sentinel to enable strict
verification everywhere.
* SILModule::isVisibleExternally utility for VarDecls.
* Fix the SIL parser so it doesn't drop global variable decls.
This information was getting lost in SIL printing/parsing.
Some passes rely on it. Regardless of whether passes should rely on it,
it is totally unacceptable for the SIL passes to have subtle differences
in behavior depending on the frontend mode. So, if we don't want passes
to rely on global variable decls, that needs to be enforced by the API
independent of how the frontend is invoked or how SIL is serialized.
* Use custom DemangleOptions to lookup global variable identifiers.
* SILGlobalVariable utilities.
Move the utilities that are required for recognizing SILGlobalVariable access
into SILGlobalVariable.[h|cpp].
Structural SIL properties that are assumed by the optimizer, and thus required
for SIL verification, should never be embedded in SILOptimizer passes, or even
in SILOptimizer/Utils. Structural SIL properties need to be defined in
/SIL. They are as much part of the SIL language as the opcode list.
These particular utilities are required for working with SILGlobalVariables, and
will be used by a whole-module access enforcement optimization.
The primary API for recognizing SIL globals is `getVariableOfGlobalInit`. It is
required to find the association between the addressor SILFunction marked
[global_init], and the SILGlobalVariable being addressed.
Other helper APIs expose more details about the addressor's SIL patterns and are
useful for transforming the initializer itself into an optimized form.
Use AccessedStorageAnalysis to find access markers with no nested conflicts.
This optimization analyzes the scope of each access to determine
whether it contains a potentially conflicting access. If not, then it
can be demoted to an instantaneous check, which still catches
conflicts on any enclosing outer scope.
This removes up to half of the runtime calls associated with
exclusivity checking.
