Commit Graph

350 Commits

Author SHA1 Message Date
Meghana Gupta
90c5b80b0b [NFC] Add some debug msgs in LoopUnroll 2023-12-13 10:38:12 -08:00
Erik Eckstein
cc83a7d414 add the "array.copy_into_vector" array-semantic 2023-12-09 18:49:57 +01:00
Meghana Gupta
86b651330b Revert "Merge pull request #69807 from apple/revert-69450-uninarrayfix"
This reverts commit cabb5e109f, reversing
changes made to 09688abb02.
2023-12-06 08:29:50 -08:00
Erik Eckstein
0897d8a720 MemoryLifetimeVerifier: use CalleeCache instead of AliasAnalysis
To verify if a function may read from an indirect argument, don't use AliasAnalysis.
Instead use the CalleeCache to get the list of callees of an apply instruction.
Then use a simple call-back into the swift Function to check if a callee has any relevant memory effect set.

This avoids a dependency from SIL to the Optimizer.
It fixes a linker error when building some unit tests in debug.
2023-12-01 19:20:18 +01:00
Erik Eckstein
0d596fd8c7 ABCOpts: fix a miscompile in case a value is added to the array index
For example: `array[i + x]`. In this case the addition of `x` was misinterpreted as pre-increment of the induction variable

rdar://118026862
2023-11-20 11:20:55 +01:00
Doug Gregor
8c079c3e43 Teach ForEachLoopUnroll to deal with error basic blocks without arguments 2023-11-13 14:51:14 -08:00
Mishal Shah
e8de333daf Revert "Add a mark_dependence while emitting SIL for uninitialized array allocation " 2023-11-12 09:43:13 -08:00
Meghana Gupta
61ce3a3af2 Handle mark_dependence in ForEachLoopUnroll 2023-10-27 14:19:21 -07:00
Erik Eckstein
4db8130625 LoopRotate: ignore single-block loops where the "backedge" block only contains trivial instructions
This avoids that LoopRotate increases code size without any benefit.

rdar://117362048
2023-10-24 11:19:28 +02:00
Evan Wilde
309aed4925 Add SmallSetVector replacement
llvm::SmallSetVector changed semantics
(https://reviews.llvm.org/D152497) resulting in build failures in Swift.
The old semantics allowed usage of types that did not have an
`operator==` because `SmallDenseSet` uses `DenseSetInfo<T>::isEqual` to
determine equality. The new implementation switched to using
`std::find`, which internally uses `operator==`. This type is used
pretty frequently with `swift::Type`, which intentionally deletes
`operator==` as it is not the canonical type and therefore cannot be
compared in normal circumstances.

This patch adds a new type-alias to the Swift namespace that provides
the old semantic behavior for `SmallSetVector`. I've also gone through
and replaced usages of `llvm::SmallSetVector` with the
`Swift::SmallSetVector` in places where we're storing a type that
doesn't implement or explicitly deletes `operator==`. The changes to
`llvm::SmallSetVector` should improve compile-time performance, so I
left the `llvm::SmallSetVector` where possible.
2023-07-25 12:28:27 -07:00
Saleem Abdulrasool
55531ea5d9 Merge pull request #67085 from hjyamauchi/licm
Don't try to hoist/sink {begin,end}_access when the loop has no exits
2023-07-05 20:58:05 -07:00
Hiroshi Yamauchi
cd5275a12f Don't try to hoist/sink {begin,end}_access when the loop has no exits
This fixes compiler crash with message `LICM: Could not perform must-sink instruction`.

Fixes: #67084
2023-07-02 13:04:14 -07:00
Evan Wilde
250082df25 [NFC] Reformat all the LLVMs
Reformatting everything now that we have `llvm` namespaces. I've
separated this from the main commit to help manage merge-conflicts and
for making it a bit easier to read the mega-patch.
2023-06-27 09:03:52 -07:00
Evan Wilde
f3ff561c6f [NFC] add llvm namespace to Optional and None
This is phase-1 of switching from llvm::Optional to std::optional in the
next rebranch. llvm::Optional was removed from upstream LLVM, so we need
to migrate off rather soon. On Darwin, std::optional, and llvm::Optional
have the same layout, so we don't need to be as concerned about ABI
beyond the name mangling. `llvm::Optional` is only returned from one
function in
```
getStandardTypeSubst(StringRef TypeName,
                     bool allowConcurrencyManglings);
```
It's the return value, so it should not impact the mangling of the
function, and the layout is the same as `std::optional`, so it should be
mostly okay. This function doesn't appear to have users, and the ABI was
already broken 2 years ago for concurrency and no one seemed to notice
so this should be "okay".

I'm doing the migration incrementally so that folks working on main can
cherry-pick back to the release/5.9 branch. Once 5.9 is done and locked
away, then we can go through and finish the replacement. Since `None`
and `Optional` show up in contexts where they are not `llvm::None` and
`llvm::Optional`, I'm preparing the work now by going through and
removing the namespace unwrapping and making the `llvm` namespace
explicit. This should make it fairly mechanical to go through and
replace llvm::Optional with std::optional, and llvm::None with
std::nullopt. It's also a change that can be brought onto the
release/5.9 with minimal impact. This should be an NFC change.
2023-06-27 09:03:52 -07:00
Meghana Gupta
1e89ad6035 Sink address projections in ArrayPropertyOpt so that there are no address phis 2023-06-22 10:55:02 -07:00
Meghana Gupta
6c299ad15c Merge pull request #65573 from meg-gupta/abcoptfix
Handle a gap while matching SIL patterns for hoisting array bound checks
2023-05-10 10:59:22 -07:00
Meghana Gupta
270125b8ee Handle a gap while matching SIL patterns for hoisting array bound checks 2023-05-09 02:36:49 -07:00
Erik Eckstein
47a7acd4a7 LICM: hoist builtin "once" calls out of loops
`builtin "once"` calls are a result of inlining global accessors
2023-05-08 21:23:36 +02:00
Erik Eckstein
4b33b99ee2 MemoryLifetimeVerifier: be more precise with indirect function arguments.
Optimizations can rely on alias analysis to know that an in-argument (or parts of it) is not actually read.
We have to do the same in the verifier: if alias analysis says that an in-argument is not read, there is no need that the memory location is initialized.

Fixes a false verifier error.
rdar://106806899
2023-05-03 14:33:45 +02:00
Slava Pestov
1b721069d7 SILOptimizer: Teach loop unrolling about more kinds of increment operations 2023-04-12 18:27:50 -04:00
Erik Eckstein
8c05024ea6 SIL: move the SILInstruction::MemoryBehavior enum out of SILInstruction into the swift namespace 2023-03-21 15:33:09 +01:00
Meghana Gupta
49d9ba5751 Enable LoopRotate on header blocks with forwarding instructions
We now have support for guaranteed forwarding phis.

Fixes rdar://86210259
2023-03-15 12:53:06 -07:00
Erik Eckstein
7d8bf37e5e change to the new llvm::Optional APIs
This is a follow-up of https://github.com/apple/swift/pull/62217
2023-01-25 09:18:36 +01:00
Erik Eckstein
ae7e5b5d90 LICM: add limits for the number of alias checks to avoid large compile times in corner cases
Avoid quadratic complexity in corner cases. Usually, these limits will not be exceeded.
2023-01-04 13:36:44 +01:00
Erik Eckstein
814d890946 Use the new side effects in LICM 2022-12-21 17:41:46 +01:00
Meghana Gupta
81351daf16 Fix creation of end_borrows of store_borrows in ForEachLoopUnroll
end_borrow should not be inserted after dealloc_stack of the destination
2022-12-14 02:08:38 -08:00
Meghana Gupta
e23fe89129 LoopRotate: Fix avoiding copy_value hoisted to the pre-header
copy_value is marked as having no side effects.
Teach LoopRotate to avoid moving it to the pre header and instead duplicate it while rotating the loop.
2022-12-05 09:55:19 -08:00
Erik Eckstein
ab1b343dad use new llvm::Optional API
`getValue` -> `value`
`getValueOr` -> `value_or`
`hasValue` -> `has_value`
`map` -> `transform`

The old API will be deprecated in the rebranch.
To avoid merge conflicts, use the new API already in the main branch.

rdar://102362022
2022-11-21 19:44:24 +01:00
Allan Shortlidge
0921480486 SILOptimizer: Break circular dependency with SIL library by moving canDuplicate().
Fixes a cycle introduced by https://github.com/apple/swift/pull/61051.
2022-10-27 16:12:05 -07:00
Meghana Gupta
fdf4d13b8a Revert "Avoid generating address phis in ArrayPropertyOpt"
This reverts commit 17285c006f.
2022-10-25 11:10:52 -07:00
Josh Soref
730b16c569 Spelling siloptimizer
* access
* accessed
* accesses
* accessor
* acquiring
* across
* activated
* additive
* address
* addresses'
* aggregated
* analysis
* and
* appropriately
* archetype
* argument
* associated
* availability
* barriers
* because
* been
* beginning
* belongs
* beneficial
* blocks
* borrow
* builtin
* cannot
* canonical
* canonicalize
* clazz
* cleanup
* coalesceable
* coalesced
* comparisons
* completely
* component
* computed
* concrete
* conjunction
* conservatively
* constituent
* construct
* consuming
* containing
* covered
* creates
* critical
* dataflow
* declaration
* defined
* defining
* definition
* deinitialization
* deliberately
* dependencies
* dependent
* deserialized
* destroy
* deterministic
* deterministically
* devirtualizes
* diagnostic
* diagnostics
* differentiation
* disable
* discipline
* dominate
* dominates
* don't
* element
* eliminate
* eliminating
* elimination
* embedded
* encounter
* epilogue
* epsilon
* escape
* escaping
* essential
* evaluating
* evaluation
* evaluator
* executing
* existential
* existentials
* explicit
* expression
* extended
* extension
* extract
* for
* from
* function
* generic
* guarantee
* guaranteed
* happened
* heuristic
* however
* identifiable
* immediately
* implementation
* improper
* include
* infinite
* initialize
* initialized
* initializer
* inside
* instruction
* interference
* interferes
* interleaved
* internal
* intersection
* intractable
* intrinsic
* invalidates
* irreducible
* irrelevant
* language
* lifetime
* literal
* looks
* materialize
* meaning
* mergeable
* might
* mimics
* modification
* modifies
* multiple
* mutating
* necessarily
* necessary
* needsmultiplecopies
* nonetheless
* nothing
* occurred
* occurs
* optimization
* optimizing
* original
* outside
* overflow
* overlapping
* overridden
* owned
* ownership
* parallel
* parameter
* paths
* patterns
* pipeline
* plottable
* possible
* potentially
* practically
* preamble
* precede
* preceding
* predecessor
* preferable
* preparation
* probably
* projection
* properties
* property
* protocol
* reabstraction
* reachable
* recognized
* recursive
* recursively
* redundant
* reentrancy
* referenced
* registry
* reinitialization
* reload
* represent
* requires
* response
* responsible
* retrieving
* returned
* returning
* returns
* rewriting
* rewritten
* sample
* scenarios
* scope
* should
* sideeffects
* similar
* simplify
* simplifycfg
* somewhat
* spaghetti
* specialization
* specializations
* specialized
* specially
* statistically
* substitute
* substitution
* succeeds
* successful
* successfully
* successor
* superfluous
* surprisingly
* suspension
* swift
* targeted
* that
* that our
* the
* therefore
* this
* those
* threshold
* through
* transform
* transformation
* truncated
* ultimate
* unchecked
* uninitialized
* unlikely
* unmanaged
* unoptimized key
* updataflow
* usefulness
* utilities
* villain
* whenever
* writes

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
2022-10-03 18:31:33 -04:00
Meghana Gupta
d5f6b12830 Avoid generating address phis in LoopUnroll 2022-09-12 11:25:55 -07:00
Meghana Gupta
17285c006f Avoid generating address phis in ArrayPropertyOpt 2022-09-05 23:42:29 -07:00
Meghana Gupta
9007fca45b Get rid of address to pointer conversion in LoopRotate which is present just to avoid the address phi absence assert 2022-09-05 23:42:28 -07:00
Meghana Gupta
2e3be7da07 Avoid generating address phis in LoopRotate 2022-09-05 23:42:20 -07:00
Meghana Gupta
80414ca44b [NFC] Reorganize LoopRotate 2022-09-05 23:41:51 -07:00
Meghana Gupta
b6cf3cd7a1 Fix store_borrow generation and pattern matching in foreach loop unroll optimization 2022-08-16 15:08:22 -07:00
Michael Gottesman
1e6187c4f4 [sil] Update all usages of old API SILValue::getOwnershipKind() in favor of new ValueBase::getOwnershipKind().
Andy some time ago already created the new API but didn't go through and update
the old occurences. I did that in this PR and then deprecated the old API. The
tree is clean, so I could just remove it, but I decided to be nicer to
downstream people by deprecating it first.
2022-07-26 11:46:23 -07:00
Andrew Trick
c4079179ee Trivial fix for an LICM assertion in projectLoadValue
This seems to compile correctly in release builds. But it does go
through an llvm_unreachable path, so really isn't safe to leave unfixed.

When the accessPath has an offset, propagate it through the recursive
calls. This may have happened when the offset was moved outside of the
sequence of path indices. The code rebuilds a path from the indices
without adding back the offset.

LICM asserts during projectLoadValue when it needs to rematerialize a
loaded value within the loop using projections and the loop-invariant
address is an index_addr.

Basically:

  %a = index_addr %4 : $*Wrapper, %1 : $Builtin.Word
  store %_ to %a : $*Wrapper
  br loop:

loop:
  %f = struct_element_addr %a
  %v = load %f : $Value
  %s = struct $Wrapper (%v : $Value)
  store %s to %a : $*Wrapper

Where the store inside the loop is deleted. And where the load is
hoisted out of the loop, but now loads $Wrapper instead of $Value.

Fixes rdar://92191909 (LICM assertion: isSubObjectProjection(), MemAccessUtils.h, line 1069)
2022-04-22 16:07:12 -07:00
Nate Chandler
3e8df52e4c [LoopRotate] Don't phi forwarded guaranteed value.
A guaranteed value produced by a begin_borrow can't be both used as
an operand of an ownership forwarding-instruction and also reborrowed by
being used as a phi argument.

Avoid that by stopping rotation when encountering a header block
containing an ownership-forwarding instruction whose forwarded ownership
kind is guaranteed; such a rotation may result in using both the
original guaranteed value and the resulting guaranteed value as phi
arguments.
2021-12-07 15:54:12 -08:00
Andrew Trick
e85228491d Rename AccessedStorage to AccessStorage
to be consistent with AccessPath and AccessBase.

Otherwise, the arbitrary name difference adds constant friction.
2021-09-21 23:18:24 -07:00
Meghana Gupta
64095a6d6b Fix use of MemAcessUtils in LICM (#38784)
AccessPathWithBase::compute can return a valid access path with unidentified base.
In such cases, we cannot LICM stores, because there is no base address to check if it is invariant
2021-08-06 18:34:55 -07:00
Andrew Trick
0f88e0f3cc Rewrite instruction deletion logic in many passes
Fix innumerable latent bugs with iterator invalidation and callback invocation.

Removes dead code earlier and chips away at all the redundant copies the compiler generates.
2021-06-02 07:38:27 -07:00
Erik Eckstein
d2fc6eb3b5 AliasAnalysis: make AliasAnalysis a function analysis and simplify the cache keys
Instead of caching alias results globally for the module, make AliasAnalysis a FunctionAnalysisBase which caches the alias results per function.
Why?
* So far the result caches could only grow. They were reset when they reached a certain size. This was not ideal. Now, they are invalidated whenever the function changes.
* It was not possible to actually invalidate an alias analysis result. This is required, for example in TempRValueOpt and TempLValueOpt (so far it was done manually with invalidateInstruction).
* Type based alias analysis results were also cached for the whole module, while it is actually dependent on the function, because it depends on the function's resilience expansion. This was a potential bug.

I also added a new PassManager API to directly get a function-base analysis:
    getAnalysis(SILFunction *f)

The second change of this commit is the removal of the instruction-index indirection for the cache keys. Now the cache keys directly work on instruction pointers instead of instruction indices. This reduces the number of hash table lookups for a cache lookup from 3 to 1.
This indirection was needed to avoid dangling instruction pointers in the cache keys. But this is not needed anymore, because of the new delayed instruction deletion mechanism.
2021-05-26 21:57:54 +02:00
Erik Eckstein
ef306ca9e0 SILOptimizer: fix some missing header file include directives
NFC
2021-05-26 21:57:54 +02:00
Erik Eckstein
2a17d9a480 ArrayBoundCheckOpts: introduce a limit for the maximum dominator tree recursion depth
This is a quick fix for a stack overflow in case of very large functions.
TODO: Ideally this algorithm would be implemented as an iterative worklist algorithm.

rdar://77563057
2021-05-07 18:13:08 +02:00
Azoy
9ed732f0ab Introduce isDecl and getDeclType
fix enum logic issue

fix tests

guard against null types
2021-04-20 02:22:16 -04:00
Meghana Gupta
1f89d9ff89 Verify critical edges when -sil-verify-all is enabled 2021-03-03 23:45:56 -08:00
Andrew Trick
66752e9724 OSSA: Rewrite address cloning code to fix issues.
Generalize the AccessUseDefChainCloner in MemAccessUtils. It was
always meant to work this way, just needed a client.

Add a new API AccessUseDefChainCloner::canCloneUseDefChain().

Add a bailout for begin_borrow and mark_dependence. Those
projections may appear on an access path, but they can't be
individually cloned without compensating.

Delete InteriorPointerAddressRebaseUseDefChainCloner.

Add a check in OwnershipRAUWHelper for canCloneUseDefChain.

Add test cases for begin_borrow and mark_dependence.
2021-02-24 22:18:21 -08:00
Michael Gottesman
248292b9da [cowarrayopt] Instead of always recomputing the loop we are analyzings exiting blocks when checking for dominance, use a cache.
Saves a bunch of compile time when compiling the stdlib. Specifically, when
compiling the iOS 32 bit stdlib + sil-verify-all, this shaves off ~20% of the
compile time.
2021-02-24 11:13:52 -08:00