The C++ `SILFunctionType` exposes both `getResults()` (formal results only)
and `getResultsWithError()` (formal + error). The Swift mirror previously
only had `results`, bridging to the with-error variant. Add `formalResults`
for the formal-only view, matching the C++ split.
Switch PackSpecialization's three result-iteration sites to `formalResults`.
The bridged `createSpecializedFunctionDeclaration` preserves the error
result on its own, so iterating with-error included it twice in the new
function's signature.
Also forward the original apply's `nothrow`/`noasync` flags to the
specialized apply, required for SIL verification of a plain apply calling
a function with an error result.
Fix lifetime diagnostics to consider an implicit initializer of a ~Escapable
type to be implicitly immortal. Required to handle Optional<~Escapable> stored
properties, such as:
struct Foo<Element: ~Escapable>: ~Escapable {
var element: Element?
@_lifetime(borrow c)
init<C>(c: borrowing C) {
// error: Lifetime-dependent variable 'self' escapes its scope
}
}
The fix is simply to remove a temporary safeguard that I put in place to
compensate for our incomplete closure lifetimes. We now have the complete
representation of lifetimes on closures, so don't need the safeguard.
Representationally, a function that returns a ~Escapable value but has no
dependendencies is immortal. This was always the intended design, but the
temporary safeguard treated these cases as implicitly bound to some local scope.
Removing this safeguard has the effect of:
- Variable intialization is immortal (it cannot depend on anything by
definition). The safety of the initializer is checked inside the implementation
of those expressions rather than the caller.
- Empty ~Escapable types have an implicit immortal initializer (why not?)
- Calls to a function with @_unsafeNonescapableResult but no @_lifetime
annotation produce an immortal value. This is reasonable, and we want to
deprecate this attribute as soon as possible anyway. It is not for general use.
This is currently blocking usage of BorrowingSequence, such as a hypothetical BorrowingSequenceMapSequenceIterator.
Fixes rdar://176561897 ([nonescapable] initialization of Optional fields reports
a lifetime escape)
Currently, AutoDiff Closure Specialization pass iterates over all VJP
instructions and checks each of them against a set of conditions which
`partial_apply` of pullback must satisfy.
This logic could be re-implemented w/o loop, checking conditions in
opposite direction, starting from `return` instruction and transitively
going to defining instructions of operands (`tuple` and `partial_apply`
for the desired pullback case).
Rather than doing a standard swift runtime cast to an existential, explicitly check for the conforming instruction classes, which is much faster.
The new `isFullApplySite` and `isReturnInstruction` casting utilities are used in the (very few) time critical places in the optimizer.
After toolchain builders are upgraded to a compiler version which includes the fix for this problem (https://github.com/swiftlang/swift/pull/88270), we don't need this workaround anymore and the regular `as`/`is` casts can be used again.
Now the runtime casts doesn't show up prominently in compile-time profiling data anymore - even with a host compiler which doesn't implement fast type checks, yet.
rdar://173916206
Passing a C++ object to the TSanInOutAccess builtin resulted in an extra
temporary copy. This copy was not optimized out because the semantics of this
builtin was not understood by the optimizer. Teaching the utils that this
intrinsic does not actually modify the object, does not escape it,
and does not read it lets the optimizer eliminate this copy.
Strictly speaking, the test code that uses interop is not safe/correct,
this is why it had a lifetime issue.
rdar://173921363
We cannot use spare bits or other overlapping storage layout tricks with fundamentally
address-only enums, and we can take advantage of this to do borrowing switches or other
in-place projections without copying the value. However, for resilient enums, the
implementation may use spare bit packing, but the type must be handled address-only
outside of its defining module, and we didn't have a way to express that with
borrowing switch. Optimization passes have also been running into problems with the
complexity that we were using `unchecked_take_enum_data_addr` sometimes as a pure
operation. This patch splits the instruction into three:
- `unchecked_inplace_enum_data_addr` represents a nondestructive in-place enum
projection. It is only allowed for enums whose projection operation is
nondestructive.
- `unchecked_take_enum_data_addr` represents a destructive enum projection,
invalidating the enum and leaving the payload to be further consumed.
This matches the current instruction's semantics.
- `unchecked_borrow_enum_data_addr` represents a borrowing enum projection.
The instruction takes a second operand for "scratch" space, which the
enum representation may be copied into in order to avoid invalidating the
enum value, so the result is dependent on the lifetime of both the
original enum and the scratch buffer. This allows for borrowing switches
over resilient enums.
`unchecked_borrow_enum_data_addr` is implemented by taking advantage of the
"address-only enums can't do spare bit optimization" property at runtime.
We inspect the operand type's bitwise-borrowability from its metadata. If
the type is bitwise-borrowable, then we are allowed to bitwise-copy the
enum to the scratch space and apply the projection to the scratch space,
preserving the original value. If the type is not bitwise-borrowable, then
we cannot use spare bit optimization in its layout, so we apply the
projection in-place.
Fixes rdar://174952822.
The [dynamic_lifetime] attribute represents that the stack location's initialization state is tracked dynamically via a boolean flag — it may be uninitialized at certain program points. TempLValueElimination pass can replace an alloc_stack [dynamic_lifetime] with a destination alloc_stack that does not have dynamic_lifetime. This results in invalid SIL which triggers verifier errors due to lifetime mismatches in some program paths in ossa.
This PR fixes this issue by bailing out of TempLValueElimination for alloc_stack [dynamic_lifetime] in ossa.
Resolves rdar://175097584
This optimization handles unconditional `cond_fail` instructions, i.e. `cond_fail`s with a non-zero `integer_literal` operand.
It cuts off the control flow after such a `cond_fail` by inserting an `unreachable` instruction.
However, this optimization cannot be done as instruction simplification, because it can leave OSSA lifetimes uncompleted.
Other simplification may depend on complete lifetimes.
Similar for constant folding failing casts: we also cannot insert an `unreachable` there.
Instead, do this optimization a new function pass (which can do lifetime completion).
Fixes a SIL verification error
rdar://173728487
## Summary
This PR fixes the ODR violation detailed in
https://github.com/swiftlang/swift/issues/87917
## Impl
When specializeClosure determines the specialized closure remains
generic (`isGeneric == true`), clone the body without applying type
substitutions, and use `partial_apply` with the substitution map at the
call site instead of `thin_to_thick_function`. The body stays generic
with type parameters like `$*Optional<Self>`, which is correct since all
callers share the same shared symbol and apply their own concrete types
through the substitution map on the `partial_apply`.
## Tests
Add SIL and executable tests demonstrating that
ConstantCapturePropagation produces an ODR violation when specializing
generic closures with different concrete type substitutions but the same
constant captures.
The SIL test provides a minimal reproducer: two callers instantiate the
same generic closure with UInt8 and Int respectively, both capturing
constant `radix=10`. With `-enable-sil-verify-all`, the verifier catches
the type mismatch in the specialized function body.
The executable test exercises the real-world scenario through
`FixedWidthInteger.init?(_:radix:)`, where the miscompile causes Int and
`Int32` parsing to return garbage values because the closure body was
baked with `UInt8` type metadata.
Fixes: https://github.com/swiftlang/swift/issues/87917
[Assisted-by](https://t.ly/Dkjjk): [Claude Opus
4.6](https://www.anthropic.com/news/claude-opus-4-6)
cc @drodriguez @kyulee-com
* convert "method"s to "thin" functions: We are removing arguments from the original function. If the removed argument is the "self" argument, the specialized function cannot be a "method" anymore.
* don't create `thin_to_thick_function` instructions for non-thin functions. Instead keep it a `partial_apply`
Fixes a SIL verifier crash.
rdar://172774069
We already move `debug_value` instructions into the liverange in such cases.
But we didn't do this if the `debug_value`'s operand is a projection of the `alloc_stack`.
The fix is to delete such `debug_value` instructions. It's not ideal, however this is a very rare case.
Also, we need to delete dead projection instructions.
Dead projection instructions can appear outside of the liverange in case they were only used by an (now deleted) `debug_value` or `destroy_addr` instruction.
Fixes a SIL ownership verification error
https://github.com/swiftlang/swift/issues/87980
rdar://172950559
For very large functions this optimization can run into noticeable quadratic behavior.
Therefore, ignore functions with more than 100000 SIL instructions.
This limit is large enough to not affect most of real-world SIL functions.
In non-OSSA we cannot insert an early `destroy_addr`, because a loaded value could be retained later, e.g.
```
%1 = load %destination
... // we cannot insert a `destroy_addr %destination` here!
stores to %temp
strong_retain %1
```
fixes a miscompile
rdar://172223667
On encountering a cond_fail with differing message, merge the accumulated cond_fail instructions and begin new merge set.
Partially resolves rdar://164947648
* All SIL modifications must go through a `MutatingContext`. Therefore replace the simple setter for `isNested` with `set(isNested:, context)`
* It's better to add a `isNested` parameter for `Builder.createPartialApply` than to set it after each construction of a `partial_apply`, which can easily be missed.
Type-dependent operands can appear outside the liverange of a value and therefore must be ignored.
This bug caused MandatoryDestroyHoisting to insert wrong destroys.
Fixes a SIL verification error and/or a mis-compile
rdar://170510052
This is needed if the guaranteed argument is replaced by a copied owned value. If the argument has any use which is not compatible with "owned" ownership, we need a borrow scope.
* insert borrow scopes for specialized guaranteed arguments: This is needed if the guaranteed argument is replaced by a copied owned value. If the argument has any use which is not compatible with "owned" ownership, we need a borrow scope.
* prevent creating a specialized function with more than one "isolated" parameters
`fix_lifetime` has memory-write effects defined.
However, in TempRValueElimination we don't shrink lifetimes. Therefore we can safely ignore this instruction.
rdar://168840965
So far the optimization just handled the case where all uses of the alloc_stack are in the same basic block.
Now we can handle arbitrary liveranges of the alloc_stack.
Also, remove the destroy-hoisting part of the algorithm because this is already handle by the dedicated DestroyAddrHoisting pass
Do this even if the function then contains references to other functions with wrong linkage.
MandatoryPerformanceOptimization fixes the linkage afterwards.
This is similar to what we already do with de-virtualizing class and witness methods: https://github.com/swiftlang/swift/pull/76032
rdar://168171608
Wihtout this change an alloc_stack instruction that is defined in a different
basic block than its use could result in instructions that are not dominated by
its operands. In an asserts build this is caught by the SIL verifier, but in a
non-asserts build it can crash the compiler.
Thanks to Erik Eckstein for the actual implementation of the fix!
rdar://168622625
As RedundantLoadElimination processing the loads in reverse control flow order, the replaced loads might accumulate quite a lot of users.
This happens if the are many loads from the same location in a row.
To void quadratic complexity in `uses.replaceAll`, we swap both load instructions and move the uses from the `existingLoad` (which usually has a small number of uses) to this load - and delete the `existingLoad`.
This came up in the Sema/large_int_array.swift.gyb test, which tests a 64k large Int array.
With this fix, the compile time gets down from 3 minutes to 5 seconds.
I also changed the test:
* run the compiler with the timeout script to detect build time regressions
* re-enabled the SIL verifier because the problem there is already fixed (https://github.com/swiftlang/swift/pull/86781)
* run the compiler with -O to also test the whole optimizer pipeline and not only the mandatory pipeline
* assigned the array to a real variable (instead of `_`) to not let the optimizer remove the whole array too early
* run the compiler with and without `-parse-as-library` because this makes a huge difference how the global array is being generated
If a value is "copied" to the stack location via a `load` and `store` instruction pair and the source location is written or de-allocated between both instructions,
the optimization generated wrong SIL.
The fix is to make sure that writes to the source locations are always checked between the `load` and `store`.
rdar://168595700
This new OSSA invariant simplifies many optimizations because they don't have to take care of the corner case of incomplete lifetimes in dead-end blocks.
The implementation basically consists of these changes:
* add the lifetime completion utility
* add a flag in SILFunction which tells optimization that they need to run the lifetime completion utility
* let all optimizations complete lifetimes if necessary
* enable the ownership verifier to check complete lifetimes
These two new invariants eliminate corner cases which caused bugs if optimization didn't handle them.
Also, it will significantly simplify lifetime completion.
The implementation basically consists of these changes:
* add a flag in SILFunction which tells optimization if they need to take care of infinite loops
* add a utility to break infinite loops
* let all optimizations remove unreachable blocks and break infinite loops if necessary
* add verification to check the new SIL invariants
The new `breakIfniniteLoops` utility breaks infinite loops in the control flow by inserting an "artificial" loop exit to a new dead-end block with an `unreachable`.
It inserts a `cond_br` with a `builtin "infinite_loop_true_condition"`:
```
bb0:
br bb1
bb1:
br bb1 // back-end branch
```
->
```
bb0:
br bb1
bb1:
%1 = builtin "infinite_loop_true_condition"() // always true, but the compiler doesn't know
cond_br %1, bb2, bb3
bb2: // new back-end block
br bb1
bb3: // new dead-end block
unreachable
```
Make sure to create `destroy_value` with correct dead_end flags.
Especially, in a dead-end region where the original value's lifetime was ended with a `destroy_value [dead_end]`, the optimization must not re-create a destroy without the dead_end flag.
^ Conflicts:
^ test/SILOptimizer/mandatory-destroy-hoisting.sil
Make sure to create `destroy_value` with correct dead_end flags.
Especially, in a dead-end region where the original value's lifetime was ended with a `destroy_value [dead_end]`, the optimization must not re-create a destroy without the dead_end flag.
^ Conflicts:
^ test/SILOptimizer/destroy-hoisting.sil
This reverts commit b1eb70bf45.
The original PR (#85244) was reverted (#85836) due to rdar://165667449
This version fixes the aforementioned issue, and (potentially) improves overall
debug info retention by salvaging info in erase(instructionIncludingDebugUses:),
rather than inserting many explicit salvageDebugInfo calls throughout the code
to make the test cases pass.
Bridging: Make salvageDebugInfo a method of MutatingContext
This feels more consistent than making it a method of Instruction.
DebugInfo: Salvage from trivially dead instructions
This handles the case we were checking in constantFoldBuiltin, so we do not need to salvage debug info there any more.
Usually `load [take]` is not hoisted anyway, because there must be another instruction in the loop which re-initializes the loaded memory location.
However, this is not necessarily true for alloc_stack locations with dynamic lifetime, e.g. a pack-loop which is specialized for a single pack element (at runtime the loop is only executed once).
Fixes a SIL ownership verifier crash
rdar://167504916
The conformances `Type: Comparable` and `EnumCase: Equatable` were
previously introduced in #85757 for implementing AutoDiff closure
specialization logic. This patch moves the latter directly to the SIL
module. The conformance `Type: Comparable` is deleted, and `sort(by:)`
is used instead of `sort` to mimic the same behavior in tests. These
changes address warnings mentioned in the comment:
https://github.com/swiftlang/swift/pull/85757#issuecomment-3681636278
Aidan Hall
Ben Cohen
elsa
Andrew Trick
Daniil Kovalev
Erik Eckstein
Gábor Horváth
Gabor Horvath
Joe Groff
Meghana Gupta
Peter Rong
Meghana Gupta
John McCall
Adrian Prantl
Aidan Hall
AZero13