Commit Graph

542 Commits

Author SHA1 Message Date
elsa 83d4291709 CSE Optimizer Pass rewrite (#88248)
Resolves rdar://173862129
2026-05-15 19:11:10 +01:00
Andrew Trick f41daa2087 Merge pull request #88968 from atrick/fix-makeborrow-escape
LifetimeDependenceDiagnostics: add Builtin.makeBorrow support
2026-05-08 17:20:46 -07:00
Andrew Trick 1ed083be0c LifetimeDependenceDiagnostics: add Builtin.makeBorrow support
Code like this currently looks like a lifetime escape:

  struct Ref<T: ~Copyable & ~Escapable>: ~Escapable {
    private let ref: Builtin.Borrow<T>

    @_lifetime(borrow target)
    init(_ target: borrowing T) {
      self.ref = Builtin.makeBorrow(target)
    }
  }

This is blocking the implementation of `Ref<~Escapable>`.

Fixes rdar://176564359 ([nonescapable] support Builtin.makeBorrow in
lifetime diagnostics)
2026-05-08 12:26:42 -07:00
Meghana Gupta ebd4d11225 Merge pull request #88847 from meg-gupta/fixeaborrow
Update SIL utilities for address results from borrow accessors
2026-05-07 20:57:06 -07:00
Gábor Horváth cfeda37650 Merge pull request #88622 from Xazax-hun/tsan-effects-modeled
[SILOptimizer] Correctly compute the effects of TSan instrumentation
2026-05-06 12:20:31 +01:00
Meghana Gupta ff7724900f Fix EscapeAnalysis for address results of borrow accessors
Update escape walker to recognize that the returned address from borrow accessors
are dependent on the parameter.
2026-05-05 12:28:23 -07:00
Erik Eckstein 78340fd42d Optimizer: de-virtualize deinits of non-copyable InlineArray elements
This is especially important for Embedded Swift because non de-virtualized deinits result in IRGen crashes.

Fixes a compiler crash in embedded
rdar://175984319
2026-05-05 07:27:13 +02:00
Erik Eckstein 7204a1524b EscapeUtils: use isFullApplySite instead of listing all conforming instruction classes 2026-04-30 17:53:39 +02:00
Erik Eckstein d28549af34 Optimizer: workaround for fast type casting to the FullApplySite and ReturnInstruction protocols
Rather than doing a standard swift runtime cast to an existential, explicitly check for the conforming instruction classes, which is much faster.
The new `isFullApplySite` and `isReturnInstruction` casting utilities are used in the (very few) time critical places in the optimizer.

After toolchain builders are upgraded to a compiler version which includes the fix for this problem (https://github.com/swiftlang/swift/pull/88270), we don't need this workaround anymore and the regular `as`/`is` casts can be used again.

Now the runtime casts doesn't show up prominently in compile-time profiling data anymore - even with a host compiler which doesn't implement fast type checks, yet.

rdar://173916206
2026-04-30 17:53:39 +02:00
Gabor Horvath 12b6809813 [SILOptimizer] Correctly compute the effects of TSan instrumentation
Passing a C++ object to the TSanInOutAccess builtin resulted in an extra
temporary copy. This copy was not optimized out because the semantics of this
builtin was not understood by the optimizer. Teaching the utils that this
intrinsic does not actually modify the object, does not escape it,
and does not read it lets the optimizer eliminate this copy.

Strictly speaking, the test code that uses interop is not safe/correct,
this is why it had a lifetime issue.

rdar://173921363
2026-04-30 11:21:17 +01:00
Joe Groff 097b0d3400 SIL: Split unchecked_*_enum_data_addr according to ownership and effects.
We cannot use spare bits or other overlapping storage layout tricks with fundamentally
address-only enums, and we can take advantage of this to do borrowing switches or other
in-place projections without copying the value. However, for resilient enums, the
implementation may use spare bit packing, but the type must be handled address-only
outside of its defining module, and we didn't have a way to express that with
borrowing switch. Optimization passes have also been running into problems with the
complexity that we were using `unchecked_take_enum_data_addr` sometimes as a pure
operation. This patch splits the instruction into three:

- `unchecked_inplace_enum_data_addr` represents a nondestructive in-place enum
  projection. It is only allowed for enums whose projection operation is
  nondestructive.
- `unchecked_take_enum_data_addr` represents a destructive enum projection,
  invalidating the enum and leaving the payload to be further consumed.
  This matches the current instruction's semantics.
- `unchecked_borrow_enum_data_addr` represents a borrowing enum projection.
  The instruction takes a second operand for "scratch" space, which the
  enum representation may be copied into in order to avoid invalidating the
  enum value, so the result is dependent on the lifetime of both the
  original enum and the scratch buffer. This allows for borrowing switches
  over resilient enums.

`unchecked_borrow_enum_data_addr` is implemented by taking advantage of the
"address-only enums can't do spare bit optimization" property at runtime.
We inspect the operand type's bitwise-borrowability from its metadata. If
the type is bitwise-borrowable, then we are allowed to bitwise-copy the
enum to the scratch space and apply the projection to the scratch space,
preserving the original value. If the type is not bitwise-borrowable, then
we cannot use spare bit optimization in its layout, so we apply the
projection in-place.

Fixes rdar://174952822.
2026-04-27 15:40:37 -07:00
Meghana Gupta e260417aae Merge pull request #88278 from meg-gupta/fixea
Fix escape analysis for pointer_to_address
2026-04-03 12:53:30 -07:00
Andrew Trick d889cac8e5 Merge pull request #88212 from atrick/fix-builtin-borrow-ossa
Fix Builtin.Borrow<T> OSSA representation
2026-04-03 12:09:24 -07:00
Meghana Gupta 7b2d5fb212 Fix escape analysis for pointer_to_address
When a `pointer_to_address` reinterprets the type (i.e. the address type
differs from the originating `address_to_pointer`), the accumulated
projection path is no longer meaningful. Use `anyValueFields` to be
conservative in both the walk-down and walk-up directions.

Without this fix, escape analysis could incorrectly conclude that a store
to an `alloc_stack` does not escape when it is read back through a
type-reinterpreting `pointer_to_address`, causing DeadStoreElimination
to illegally remove the store.

rdar://171841812
2026-04-02 11:21:45 -07:00
Andrew Trick 09c5e7912d Fix Builtin.Borrow<T> OSSA representation
Builtin.Borrow<T> does not introduce a new borrow scope at the level of
SILValue. A SILValue that introduces a borrow scope is expected to be tracked by
OSSA--it cannot produce a trivial value. That borrow scope is expected to have a
scope-ending operation on all paths.

It's not clear how OSSA utilities and passes will handle the previous
implementation which doesn't honor those invariants. Fixing this is needed to
rely on Builtin.Borrow<T> more extensively.

Instead, treat the make_borrow instruction like an instantenous use. It produces
a Builtin.Borrow<T> value with ownership None. That value is completely ignored
by OSSA and does not contribute to liveness of the incoming value.

This is at least a coherent representation, but still incredibly dangerous. The
validity of SIL depends on the programmer using Builtin.makeBorrow
correctly. Add a TODO for implementing verification.

Fixes rdar://173800290 (Fix Builtin.Borrow<T> OSSA representation)
2026-04-01 23:39:55 -07:00
Andrew Trick 401d2ee072 Fix DeinitDevirtualizer to handle non-generic deinitializer function
Avoid passing an invalid substitution map into createApply.

Required conditions (from the bug report):
1. `~Copyable` struct with user-defined `deinit`
2. Nested inside a constrained extension where all generic parameters are concrete
3. A `destroy_value` or `destroy_addr` of the struct's value exists in some function
4. DeinitDevirtualizer runs (release-mode optimization pipeline)

Fixes rdar://173803881 ([GH:#88213] [SILOptimizer] DeinitDevirtualizer assertion
failure with ~Copyable type in all-concrete constrained extension)

Suggested by: coenttb
2026-04-01 20:56:42 -07:00
Erik Eckstein 3c5e08d5a6 embedded: correctly compute base protocol conformances when specializing witness tables
This didn't work in case the base conformance is both, specialized and inherited

rdar://170096756
2026-03-10 13:22:59 +01:00
Erik Eckstein b449b8c7a1 SIL: fix PartialApplyInst.isNested
* All SIL modifications must go through a `MutatingContext`. Therefore replace the simple setter for `isNested` with `set(isNested:, context)`
* It's better to add a `isNested` parameter for `Builder.createPartialApply` than to set it after each construction of a `partial_apply`, which can easily be missed.
2026-03-09 18:04:37 +01:00
John McCall 374e3d37f0 Allow partial_apply [on_stack] to be flagged [non_nested]. 2026-03-06 03:15:28 -05:00
Andrew Trick 6a9875c73e LifetimeDependenceUtils: add support for mutable captures
A mutable capture is always escaping. Naturally, you can't mutably capture a
non-Escapable value. Nonetheless, code may override the lifetime of the captured
value as immortal, knowing that the closure does not actually escape.

'_overrideLifetime(capturedThing, copying: ())' quiets the diagnostics
on the caller side. But, when diagnosing the closure body itself, lifetime
analysis is upset by the boxed non-Escaping value. Simply add a case
to LocalVariableAccessMap to recognize this as a valid case so that diagnostics
can ignore it.

Fixes rdar://170592353 (error: lifetime-dependent variable 'overriddenLifetime'
escapes its scope)
2026-02-20 09:12:38 -08:00
eeckstein 2bf3979f95 Merge pull request #86974 from eeckstein/simplification-pass
Optimizer: don't remove unused `destructure_struct` of a non-copyable value
2026-02-05 07:26:49 +01:00
Erik Eckstein 9f0a9d77c0 DeinitDevirtualizer: don't devirtualize destroy_value [dead_end]
It doesn't make sense to de-virtualize `destroy_value [dead_end]` because such operations are no-ops anyway.
This is especially important for Embedded Swift, because introducing calls to generic deinit functions after the mandatory pipeline can result in IRGen crashes.

rdar://168171608
2026-02-04 09:10:24 +01:00
Erik Eckstein 7ea1756fa4 Optimizer: don't remove unused destructure_struct of a non-copyable value
Because `destructure_struct` acts as lifetime-ending use. Removing this instruction results in an ownership error.
This can happen if a non-copyable struct only has trivial fields.

Fixes a verifier crash
https://github.com/swiftlang/swift/issues/86901
rdar://169285436
2026-02-04 08:47:42 +01:00
Erik Eckstein 6dde4c942e MandatoryPerformanceOptimizations: force de-virtualizing value-type deinits
Do this even if the function then contains references to other functions with wrong linkage.
MandatoryPerformanceOptimization fixes the linkage afterwards.
This is similar to what we already do with de-virtualizing class and witness methods: https://github.com/swiftlang/swift/pull/76032

rdar://168171608
2026-02-02 19:36:48 +01:00
Erik Eckstein 5cd70731da LifetimeCompletion/DiagnoseUnreachable: fix a wrong "will never be executed" warning
Make sure that debug locations are correct so that instruction which are inserted by DI after a no-return call doesn't cause DiagnoseUnreachable to print a wrong warning.
Also, add `extend_lifetime` to the list of excluded instructions in DiagnoseUnreachable.
2026-01-30 11:52:49 +01:00
Erik Eckstein bdf5a70258 DeinitDevirtualizer: don't create an invalid function_ref in a serialized function.
Serialized functions may not reference non-public functions.

Fixes a compiler crash
rdar://169022052
2026-01-28 13:10:30 +01:00
Joe Groff 9a417d4c28 Handle Borrow instructions in LifetimeCompletion 2026-01-23 10:03:23 -08:00
Joe Groff 5f49668577 Builtin.dereferenceBorrow should maintain the memory location of addressable values. 2026-01-23 08:02:09 -08:00
Joe Groff b11e8c985a SIL: Handle dereference_borrow returns in SILGenCleanup.
Make sure an `end_borrow` is emitted on the `dereference_borrow` so that
SILGenCleanup recognizes this pattern and lowers it to a `return_borrow`
as it does for returned `load_borrow`s. Add support to the Swift implementation
of `BorrowingInstruction` for `DereferenceBorrow`.
2026-01-23 08:02:09 -08:00
Joe Groff bc53d4ecb4 SIL: Handle Borrow instructions in more utility visitors. 2026-01-23 08:02:09 -08:00
Erik Eckstein 18063707b5 Optimizer: enable complete OSSA lifetimes throughout the pass pipeline
This new OSSA invariant simplifies many optimizations because they don't have to take care of the corner case of incomplete lifetimes in dead-end blocks.

The implementation basically consists of these changes:
* add the lifetime completion utility
* add a flag in SILFunction which tells optimization that they need to run the lifetime completion utility
* let all optimizations complete lifetimes if necessary
* enable the ownership verifier to check complete lifetimes
2026-01-22 17:41:48 +01:00
Erik Eckstein 0f0aa0c17b Optimizer: require that there are no unreachable blocks and infinite loops in OSSA
These two new invariants eliminate corner cases which caused bugs if optimization didn't handle them.
Also, it will significantly simplify lifetime completion.

The implementation basically consists of these changes:
* add a flag in SILFunction which tells optimization if they need to take care of infinite loops
* add a utility to break infinite loops
* let all optimizations remove unreachable blocks and break infinite loops if necessary
* add verification to check the new SIL invariants

The new `breakIfniniteLoops` utility breaks infinite loops in the control flow by inserting an "artificial" loop exit to a new dead-end block with an `unreachable`.
It inserts a `cond_br` with a `builtin "infinite_loop_true_condition"`:
```
bb0:
  br bb1
bb1:
  br bb1              // back-end branch
```
->
```
bb0:
  br bb1
bb1:
  %1 = builtin "infinite_loop_true_condition"() // always true, but the compiler doesn't know
  cond_br %1, bb2, bb3
bb2:                  // new back-end block
  br bb1
bb3:                  // new dead-end block
  unreachable
```
2026-01-22 17:41:23 +01:00
Erik Eckstein 2860c2df77 Optimizer: add DominatorTree.getImmediateDominator and DominatorTree.getDominanceOrder 2026-01-22 17:41:22 +01:00
Daniil Kovalev 1f77138afe [AutoDiff] Closure specialization: specialize branch tracing enums (#85757)
This patch contains part of the changes intended to resolve #68944.

1. Closure info gathering logic.
2. Branch tracing enum specialization logic.
3. Specialization of branch tracing enum basic block arguments in VJP.
4. Specialization of branch tracing enum payload basic block arguments
in pullback.

Note that mangling-related logic is implemented in C++ since at this
moment we have no Swift bridged for that.

Here is a simplified example of how branch tracing enum (BTE)
specialization looks like.

Before specialization:

```
enum $_AD__xxx {
  case bb0(((Float) -> Float))
}

func vjp(...) {
  // ...
  %foo      = function_ref $foo         : (Float, Float) -> Float
  %pa1      = partial_apply %foo(%arg1) : (Float) -> Float
  %payload1 = tuple (%pa1)              : ((Float) -> Float)
  %bte      = enum $_AD__xxx.bb0!enumelt, %payload1
  // ...
}

func pullback(%bte, ...) {
  // ...
  %payload2 = unchecked_enum_data %bte, $_AD__xxx.bb0!enumelt : ((Float) -> Float)
  %pa2      = tuple_extract %payload2, 0                      : (Float) -> Float
  %res      = apply %pa2(%arg2)                               : Float
  // ...
}
```

After specialization:

```
enum $_AD__xxx_spec_bb0_0 {
  case bb0(((Float)))
}

func vjp(...) {
  // ...
  %captured1 = tuple (%arg1)      : (Float)
  %payload1  = tuple (%captured1) : ((Float))
  %bte_spec  = enum $_AD__xxx_spec_bb0_0.bb0!enumelt, %payload1
  // ...
}

func pullback_spec(%bte_spec, ...) {
  // ...
  %payload2  = unchecked_enum_data %bte, $_AD__xxx_spec_bb0_0.bb0!enumelt : ((Float))
  %captured2 = tuple_extract %payload2, 0                                 : (Float)
  %arg1      = tuple_extract %captured2, 0                                : Float
  %foo       = function_ref $foo                                          : (Float, Float) -> Float
  %res       = apply %foo(%arg2, %arg1)                                   : Float
  // ...
}
```
2025-12-21 00:33:50 +00:00
Meghana Gupta 5c4ce2f942 Merge pull request #86023 from meg-gupta/miscborrowfixes
Minor fixes to borrow accessors
2025-12-16 20:03:19 -08:00
Meghana Gupta 0c5fe56b7f Bailout on address apply result while computng argument escaping effects 2025-12-15 13:02:13 -08:00
Erik Eckstein d8a85693c8 Optimizer: convert the FunctionUsesDumper pass to a test 2025-12-15 10:01:41 +01:00
Erik Eckstein 6ef1064a92 Optimizer: add ModuleTest 2025-12-15 10:01:41 +01:00
Erik Eckstein 751e52cdc3 Optimizer: convert the DeadEndBlockDumper pass to a test 2025-12-15 10:01:40 +01:00
Erik Eckstein dc38a2226e Optimizer: convert the AliasInfoDumper and MemBehaviorDumper passes to tests 2025-12-15 09:09:41 +01:00
Erik Eckstein abc40b3b55 Optimizer: convert the EscapeInfoDumper from a pass to a test 2025-12-15 09:09:40 +01:00
Andrew Trick d609889e87 Merge pull request #85934 from atrick/fix-closure-liveness
Fix InteriorUseWalker: consider partial_apply [on_stack] an escape
2025-12-11 15:05:21 -08:00
Joe Groff 4479b49fd8 SIL: Handle address ReturnInsts from borrow accessors as yielding uses.
This avoids a spurious lifetime error when an address-only borrow accessor returns a
non-`Escapable` value from a non-`Escapable` aggregate.
2025-12-10 11:44:41 -08:00
Andrew Trick f7c3c6f437 Fix InteriorUseWalker: consider partial_apply [on_stack] an escape
Fixes a bug in MandatoryDestroyHoisting where a captured value is destroyed
before a copy of the closure.

On-stack closures can be copied, and all copied uses must be within the borrow
scope of the captured operand. This is just like any other non-Escapable value,
so treat it as such by checking `Value.mayEscape` rather than `Type.Escapable`.

Originally, I wanted to make it illegal to copy of partial_apply [on_stack], but
it looks like we still allow it.

I would rather not complicate any logic yet with special handling for this
case. To fix any performance concerns, we might be able to simplify the
representation instead by banning copy_value on on-stack closures.

Fixes rdar://165850554 swift-frontend crash: While running "CopyPropagation" -
Invalid SIL provided to OSSACompleteLifetime?!)
2025-12-09 14:38:04 -08:00
eeckstein d0f8ccef28 Merge pull request #85857 from eeckstein/fix-vtable-specialization
Embedded: allow generic class constructors
2025-12-05 20:17:52 +01:00
Erik Eckstein 8b496f668f Embedded: allow generic class constructors
For example:

```
  public class C: MyClass {
    public init(p: some P) {
      // ...
    }
  }
```

Constructors are not called via the vtable (except "required" constructors).
Therefore we can allow generic constructors.

https://github.com/swiftlang/swift/issues/78150
rdar://138576752
2025-12-05 11:35:00 +01:00
Aidan Hall b1eb70bf45 Revert "[DebugInfo] Salvage more in -O builds"
This reverts commit a95d2979f9.

rdar://165667449
2025-12-04 15:39:39 +00:00
Arnold Schwaighofer 36a3c6e611 Merge pull request #85644 from aschwaighofer/wip_embedded_exits_with_eriks_changes_v1
[embedded] Fix associated type conformances for specialized witness tables
2025-12-01 16:40:31 -08:00
Aidan Hall 96dca43eb9 Merge pull request #85244 from aidan-hall/fixing-debug-info-rebase
Retain more debug info in optimized builds
2025-12-01 20:49:40 +00:00
Arnold Schwaighofer 0c882d42ce Merge pull request #85625 from aschwaighofer/wip_embedded_cast_tuples
[embedded] Allow casting to tuples and tighten the check which existentials we support in embedded with existentials
2025-12-01 08:27:43 -08:00