Increase inlining benefits for functions with more than 5 arguments and / or results.
We assume that each argument beyond these 5 would be passed on stack and therefore would incur a pair of load and store.
The old analysis pass doesn't take into account profile data, nor does
it consider post-dominance. It primarily dealt with _fastPath/_slowPath.
A block that is dominated by a cold block is itself cold. That's true
whether it's forwards or backwards dominance.
We can also consider a call to any `Never` returning function as a
cold-exit, though the block(s) leading up to that call may be executed
frequently because of concurrency. For now, I'm ignoring the concurrency
case and assuming it's cold. To make use of this "no return" prediction,
use the `-enable-noreturn-prediction` flag, which is currently off by
default.
Although I don't plan to bring over new assertions wholesale
into the current qualification branch, it's entirely possible
that various minor changes in main will use the new assertions;
having this basic support in the release branch will simplify that.
(This is why I'm adding the includes as a separate pass from
rewriting the individual assertions)
Compute, update and handle borrowed-from instruction in various utilities and passes.
Also, used borrowed-from to simplify `gatherBorrowIntroducers` and `gatherEnclosingValues`.
Replace those utilities by `Value.getBorrowIntroducers` and `Value.getEnclosingValues`, which return a lazily computed Sequence of borrowed/enclosing values.
LLVM is presumably moving towards `std::string_view` -
`StringRef::startswith` is deprecated on tip. `SmallString::startswith`
was just renamed there (maybe with some small deprecation inbetween, but
if so, we've missed it).
The `SmallString::startswith` references were moved to
`.str().starts_with()`, rather than adding the `starts_with` on
`stable/20230725` as we only had a few of them. Open to switching that
over if anyone feels strongly though.
Not inlined co-routines are so expensive that they should be inlined, unless they are really large.
So far co-routines didn't get any special treatment in the inliner, except generic co-routines.
With this change, even non-generic co-routines are treated as high-priority to inline.
rdar://117201823
For linear maps containing control-flow, closures (representing the pullbacks
of intermediate values) may be passed as arguments, however, they may be
hidden behind a branch-tracing enum (tracing execution flow of the original
function).
Such linear maps did not use to get inlining benefits as the compiler
could not see that the intermediate pullback closures were actually part
of the input.
This change modifies the inliner logic to correctly award inlining
benefits to linear maps containing control-flow, by checking if a
"callee" in the linear map actually traces back to an input closure that
was received as part of a branch-tracing enum input argument.
Fixes#68945
Reformatting everything now that we have `llvm` namespaces. I've
separated this from the main commit to help manage merge-conflicts and
for making it a bit easier to read the mega-patch.
This is phase-1 of switching from llvm::Optional to std::optional in the
next rebranch. llvm::Optional was removed from upstream LLVM, so we need
to migrate off rather soon. On Darwin, std::optional, and llvm::Optional
have the same layout, so we don't need to be as concerned about ABI
beyond the name mangling. `llvm::Optional` is only returned from one
function in
```
getStandardTypeSubst(StringRef TypeName,
bool allowConcurrencyManglings);
```
It's the return value, so it should not impact the mangling of the
function, and the layout is the same as `std::optional`, so it should be
mostly okay. This function doesn't appear to have users, and the ABI was
already broken 2 years ago for concurrency and no one seemed to notice
so this should be "okay".
I'm doing the migration incrementally so that folks working on main can
cherry-pick back to the release/5.9 branch. Once 5.9 is done and locked
away, then we can go through and finish the replacement. Since `None`
and `Optional` show up in contexts where they are not `llvm::None` and
`llvm::Optional`, I'm preparing the work now by going through and
removing the namespace unwrapping and making the `llvm` namespace
explicit. This should make it fairly mechanical to go through and
replace llvm::Optional with std::optional, and llvm::None with
std::nullopt. It's also a change that can be brought onto the
release/5.9 with minimal impact. This should be an NFC change.
Now that we handle inlined global initializers in LICM, CSE and the StringOptimization, we don't need to have a separate mid-level inliner pass, which treats global accessors specially.
A pass is skipped if no other pass changed the function since the previous run of the same pass.
Don't do this is if a pass depends on the function bodies of called functions, e.g. the inliner.
Other passes might change the callees, e.g. function signature opts, which makes it worth to run the inliner
again, even if the function itself didn't change.
Inline-always should only be used on relatively small functions. It must not be used on recursive functions.
Add a check that prevents that inlining of large @inline(__always) functions.
https://github.com/apple/swift/issues/64319
rdar://106655649
Now that in OSSA `partial_apply [on_stack]`s are represented as owned
values rather than stack locations, it is possible for their destroys to
violate stack discipline. A direct lowering of the instructions to
non-OSSA would violate stack nesting.
Previously, when inlining, it was assumed that non-coroutine callees
maintained stack discipline. And, when inlining an OSSA function into a
non-OSSA function, OSSA instructions were lowered directly. The result
was that stack discipline could be violated.
Here, when inlining a function in OSSA form into a function lowered out
of OSSA form, stack nesting is fixed up.
Previously, there was an -Xllvm option to verify after all inlining to a
particlar caller. That makes it a chore to track down which apply's
inlining resulted in invalid code. Here, a new option is added that
verifies after each run of the inliner.
`getValue` -> `value`
`getValueOr` -> `value_or`
`hasValue` -> `has_value`
`map` -> `transform`
The old API will be deprecated in the rebranch.
To avoid merge conflicts, use the new API already in the main branch.
rdar://102362022
There are examples of such functions, which were previously captured by `isPureCall` in the inliner. But the implementation of `isPureCall` was wrong.
With the new (and correct) side effect analysis we need to correctly handle such functions in the inlining heuristic.
Enable caller and callee to be printed as inlining runs. The printing
is filtered based on -sil-print-function/-sil-print-functions and
includes finer-grained info than those do already. The caller before
and after each callee is inlined can be printed as well as the callee
on its own as it exists when inlining occurs.
Add AccesssedStorage::compute and computeInScope to mirror AccessPath.
Allow recovering the begin_access for Nested storage.
Adds AccessedStorage.visitRoots().
This attribute allows to define a pre-specialized entry point of a
generic function in a library.
The following definition provides a pre-specialized entry point for
`genericFunc(_:)` for the parameter type `Int` that clients of the
library can call.
```
@_specialize(exported: true, where T == Int)
public func genericFunc<T>(_ t: T) { ... }
```
Pre-specializations of internal `@inlinable` functions are allowed.
```
@usableFromInline
internal struct GenericThing<T> {
@_specialize(exported: true, where T == Int)
@inlinable
internal func genericMethod(_ t: T) {
}
}
```
There is syntax to pre-specialize a method from a different module.
```
import ModuleDefiningGenericFunc
@_specialize(exported: true, target: genericFunc(_:), where T == Double)
func prespecialize_genericFunc(_ t: T) { fatalError("dont call") }
```
Specially marked extensions allow for pre-specialization of internal
methods accross module boundries (respecting `@inlinable` and
`@usableFromInline`).
```
import ModuleDefiningGenericThing
public struct Something {}
@_specializeExtension
extension GenericThing {
@_specialize(exported: true, target: genericMethod(_:), where T == Something)
func prespecialize_genericMethod(_ t: T) { fatalError("dont call") }
}
```
rdar://64993425
More specifically, if one wants to force emit /all/ opt-remarks on a function, mark it with:
```
@_semantics("optremark")
```
If one wants to emit opt-remarks only for a specific SIL pass (like lets say
sil-opt-remark-gen), one can write:
```
@_semantics("optremark.sil-opt-remark-gen")
```
I made the pattern matching strict so if you just put in a '.' or add additional
suffixes, it will not pattern match. I think that this is sufficient for a
prototyping tool.
This is useful if one wants to play around with opt-remarks when optimizing code
in Xcode or any IDE that can use serialized diagnostics.
In all of these cases, we already had a SILFunction and were just grabbing its
SILModule instead of passing it in. So this is just an NFC change.
The reason why I am doing this is so that I can force emit opt-remarks on
functions with the semantics attribute "optremark", so I need to be able to
access the SILFunction in the optimization remark infrastructure.
For use outside access enforcement passes.
Add isUniquelyIdentifiedAfterEnforcement.
Rename functions for clarity and generality.
Rename isUniquelyIdentifiedOrClass to isFormalAccessBase.
Rename findAccessedStorage to identifyFormalAccess.
Rename findAccessedStorageNonNested to findAccessedStorage.
Part of generalizing the utility for use outside the access
enforcement passes.
This became necessary after recent function type changes that keep
substituted generic function types abstract even after substitution to
correctly handle automatic opaque result type substitution.
Instead of performing the opaque result type substitution as part of
substituting the generic args the underlying type will now be reified as
part of looking at the parameter/return types which happens as part of
the function convention apis.
rdar://62560867
When inlining many functions in a very large basic block, the splitting of the block at the call sites is quadratic, when traversing in forward order.
Traversing backwards, fixes the problem.
rdar://problem/56268570
We need this anyways for -Onone and I want to do some experiments with running
this very early so I can expose more of the stdlib (modulo inlining) to the new
ownership optimizing passes.
I also changed how the inliner handles inlining around OSSA by changing it to
check early that if the caller is in ossa, then we only inline if all of the
callees that the caller calls are in ossa. The intention is to hopefully avoid
weird swings in code-size/perf due to the inliner heuristic's calculation being
artificially manipulated due to some callees not being available to inline (due
to this difference) when others are already available.
This reverts commit 66474ed5a2.
The original commit triggers a crash in IRGen: rdar://problem/59456064
Reverting for now until the IRGen issue is fixed.
The returned partial_apply of a thunk is most likely being optimized away if inlined.
Because some thunks cannot be specialized (e.g. if an opened existential is in the subsitution list), inline such thunks also in case they are generic.
https://bugs.swift.org/browse/SR-12115
rdar://problem/59061452
