Commit Graph

230 Commits

Author SHA1 Message Date
Dario Rexin 6af54cd164 [IRGen] Return large loadable types as @guaranteed_address
rdar://167713693

The LoadableByAddress pass recognizes large loadable types in borrow accessors and returns them by @guaranteed_address.
This prevents stack allocations and unnecessary copies.

Co-authored-by: Meghana Gupta <meghana_gupta@apple.com>
2026-01-29 13:33:47 -08:00
Arnold Schwaighofer 4f19313f14 Merge pull request #86148 from aschwaighofer/large_types_reg2mem_large_inline_array_compile_time_fix
LargeTypesReg2Mem: Avoid building an explosion schema just to count the number of registers for very large types
2025-12-22 15:11:21 -08:00
Arnold Schwaighofer f02d91395a Add some comments to constants 2025-12-19 12:30:33 -08:00
Arnold Schwaighofer 6c930432ad LargeTypesReg2Mem: Avoid building an explosion schema just to count the number of registers for very large types
Computing the explosion schema for large types can be expensive.

E.g the following example would spend multiple seconds to compute the
explosion scheme for the array type. It is not worth adding the
complexity to have a special case explosion schema that handles types
like that since exploding a value like that is going to be detrimental.

```
struct Test {
  private var values: [0xfffffff of [UInt32]] =
  InlineArray(repeating: [])
}
```

rdar://165202495
2025-12-19 12:07:48 -08:00
Erik Eckstein daa8850a77 LoadableByAddress: handle the unchecked_bitwise_cast instruction
Fixes a compiler crash in LoadableByAddress.
Unfortunately I don't have a test case for this.
2025-12-19 17:44:01 +01:00
Slava Pestov 819738c83e AST: Rename mapTypeIntoContext() => mapTypeIntoEnvironment(), mapTypeOutOfContext() => mapTypeOutOfEnvironment() 2025-11-12 14:48:19 -05:00
Arnold Schwaighofer 78a3393f72 Add more information to backtraces from the LoadableByAddress pass
Should not change functionality outside of displaying more information in backtraces.
2025-10-24 06:31:07 -07:00
Arnold Schwaighofer 1e8e3b82df LargeTypesReg2Mem: Peephole load followed by single user to reuse the load's address
For example the pattern:

```
%e = load %1 : SomeEnum
switch_enum %e
```

The switch_enum_addr should reuse the load's address operand instead of
creating a new location for the SSA value of %e.

While we are there also don't create a new stack location for extracting payload
in a switch_enum_addr case target basic block if that extracted payload is not
used. That is if the basic block arguments in all switch target blocks are
unused in the switch_enum version.

```
switch_enum %enum, case #Optional.some!enumelt: bb1,
                   case #Optional.none!enumelt: bb2

bb2(%payload : $Payload):
   // unused %payload value
```

rdar://156602951
2025-08-07 07:21:51 -07:00
Meghana Gupta e317a603fc Add simplification for end_cow_mutation_addr
We insert end_cow_mutation_addr for lifetime dependent values dependent on mutable addresses.
end_cow_mutation_addr can be simplified to end_cow_mutation after other optimizations like inlining, specialization etc

This PR adds an instruction simplification to transform end_cow_mutation_addr to end_cow_mutation.
This can enable array optimizations which look for end_cow_mutation.
2025-07-14 13:46:13 -07:00
Arnold Schwaighofer 521fb07523 LoadableByAddress: Fix shouldTransformYields to use substituted types
rdar://149281263
issues/80818
2025-04-21 13:46:41 -07:00
Arnold Schwaighofer 370b7e8219 LargeTypesReg2Mem: Add unchecked_bitwise_cast to the projections we need to propagate largeness from destination to source operand
rdar://148545382
2025-04-17 09:18:31 -07:00
John McCall 3fe70968cc Remove the substitution map from zeroInitializer builtin in SIL.
This is a value operation that can work just fine on lowered types,
so there's no need to carry along a formal type. Make the value/address
duality clearer, and enforce it in the verifier.
2025-03-26 00:34:15 -04:00
Arnold Schwaighofer 5feb287d19 LoadableByAddress: Add missing BranchInst in a switch
rdar://146980802
2025-03-19 09:50:02 -07:00
Arnold Schwaighofer 8120fa3989 LoadableByAddress: Must be able to materialize undef operands
rdar://145092298
2025-02-19 13:59:32 -08:00
Arnold Schwaighofer 7aef2b549c Fix static variable initializations 2025-01-09 11:42:04 -08:00
Arnold Schwaighofer abbe36a1e9 LoadableByAddress: Replace some magic constants
NFC intended. Although there is a little fix in setNumRegisters.
2025-01-06 11:57:37 -08:00
Arnold Schwaighofer be5223f0fa LargeLoadable types: propagate large type property along projections
Back propagate the isLargeType property along projections. This is
neccessary because c union types can "hide" largeness.

rdar://141775951
2024-12-20 09:09:42 -08:00
Nate Chandler 79c62dd5a0 [CoroutineAccessors] Rewrite allocs in LBA.
When replacing a `begin_apply` with an allocation, replace uses of the
old allocation with uses of the new.
2024-11-19 12:57:49 -08:00
Arnold Schwaighofer ec5c432e09 LargeTypesReg2Mem: Don't ignore BuiltinFixedArrayType
rdar://139457907
2024-11-07 15:30:17 -08:00
Arnold Schwaighofer d925eefdba IRGen: Fix a map entry use-after-free issue in LargeLoadableHeuristic
rdar://139398409
2024-11-07 09:51:16 -08:00
Arnold Schwaighofer 6755d0e104 LargeTypesReg2Mem: Ammend heuristic to include projections of block
arguments
2024-11-04 17:06:24 -08:00
Arnold Schwaighofer 787c996394 LargeTypesReg2Mem: Add a new heuristic that trys harder to keep large
values on the stack

This heuristic can be enabled by passing -Xfrontend
-enable-aggressive-reg2mem.

rdar://123916109
2024-10-31 13:22:06 -07:00
Arnold Schwaighofer e909720e4d LargeTypesReg2Mem: Map undef SIL values to an undefined address on the stack
SIL can have undefined values along some path (which dynamically is
never executed).

rdar://136600129
2024-09-24 15:43:38 -07:00
Arnold Schwaighofer eeaf9621f2 [loadable by address reg2mem] Implement missing throw instruction
rdar://134914984
2024-09-05 08:12:32 -07:00
Tim Kientzle 598e5104ef Merge pull request #74184 from tbkka/tbkka-assertions2
Add `#include "swift/Basic/Assertions.h"` to a lot of source files
2024-06-20 12:13:28 -07:00
Arnold Schwaighofer bddc50ef9e LargeTypesReg2Mem: Handle mark_dependence
This instruction is a no-op. We can just delete it.

rdar://129873110
2024-06-19 09:36:43 -07:00
Tim Kientzle 1098054291 Merge branch 'main' into tbkka-assertions2 2024-06-18 17:52:00 -07:00
Akira Hatanaka d92f181ace Create two versions (for caller and callee) of the functions that answer questions about parameter convention (#74124)
Create two versions of the following functions:

isConsumedParameter
isGuaranteedParameter
SILParameterInfo::isConsumed
SILParameterInfo::isGuaranteed
SILArgumentConvention::isOwnedConvention
SILArgumentConvention::isGuaranteedConvention

These changes will be needed when we add a new convention for
non-trivial C++ types as the functions will return different answers
depending on whether they are called for the caller or the callee. This
commit doesn't change any functionality.
2024-06-18 09:06:09 -07:00
Tim Kientzle 1d961ba22d Add #include "swift/Basic/Assertions.h" to a lot of source files
Although I don't plan to bring over new assertions wholesale
into the current qualification branch, it's entirely possible
that various minor changes in main will use the new assertions;
having this basic support in the release branch will simplify that.
(This is why I'm adding the includes as a separate pass from
rewriting the individual assertions)
2024-06-05 19:37:30 -07:00
Arnold Schwaighofer d89bf2893b IRGen: Properly adjust the closure type of a partial_apply of an objc_method
It needs to match with the (large loadable) lowered closure type in the rest of
the program: Large types in the signature need to be passed indirectly.

rdar://127367321
2024-05-22 09:39:15 -07:00
Arnold Schwaighofer b5d9ff4557 Merge pull request #73626 from aschwaighofer/large_types_reg2mem_unchecked_trivial_bitcast_fix
IRGen: Large types reg2mem - Fix visitUncheckedTrivialBitCastInst
2024-05-15 20:16:05 -07:00
Slava Pestov a356b10929 SIL: Sink local archetype substitution into remapType() 2024-05-15 17:39:00 -04:00
Arnold Schwaighofer 1258563c3b IRGen: Large types reg2mem - Fix visitUncheckedTrivialBitCastInst
An unchecked_trivial_bit_cast can go from a bigger type to a smaller
type. Therefore we must allocate stack storage for the operand type
rather than the result type. Otherwise, we can end up storing bigger
values into smaller storage -- not good.

rdar://128086028
2024-05-14 13:45:45 -07:00
Anton Korobeynikov e7e2ad19f1 Ensure we are using mapped SIL type for switch_enum case and not the original lowered one (#73385)
Fixes #73018
2024-05-09 13:41:34 -07:00
Emil Pedersen 644509ea1c [DebugInfo] Fix LoadableByAddress losing debug info
LoadableByAddress was losing debug info, including at -Onone.
When converting a load-store pair to a copy_addr, the debug info
attached to the load was not salvaged.
Additionally, the wrong scope was being attached to other debug
values.
2024-05-01 18:44:33 -07:00
Arnold Schwaighofer ff4682941a LargeTypesReg2Mem: Fix yield and begin_apply instructions
Speculative fix for #swift/issues/72911
2024-04-25 16:51:33 -07:00
Arnold Schwaighofer 51e148d153 Merge pull request #72694 from aschwaighofer/c_array_heuristic
LargeTypesReg2Mem: Add a heuristic for C character arrays
2024-03-29 12:43:20 -07:00
Arnold Schwaighofer 081a872b81 LargeTypesReg2Mem: Add a heuristic for C character arrays
They can lower to a set of i64 registers we consider small but exploded
to (manifested as SSA) registers they will cause significant code bloat.

rdar://125265576
2024-03-29 05:35:26 -07:00
Anton Korobeynikov 2ad4a60619 Ensure LoadableByAddress rewrites pointer_to_address (#72619)
if the target address is address of tuple type that should be rewritten.

Fixes #72363
2024-03-27 12:12:47 -07:00
Anton Korobeynikov a63078f279 Add missed case to AssignAddressToDef. (#72617)
Fixes #71744
2024-03-27 10:49:48 -07:00
Michael Gottesman 11f0ff6e32 [sil] Ensure that all SILValues have a parent function by making it so that SILUndef is uniqued at the function instead of module level.
For years, optimizer engineers have been hitting a common bug caused by passes
assuming all SILValues have a parent function only to be surprised by SILUndef.
Generally we see SILUndef not that often so we see this come up later in
testing. This patch eliminates that problem by making SILUndef uniqued at the
function level instead of the module level. This ensures that it makes sense for
SILUndef to have a parent function, eliminating this possibility since we can
define an API to get its parent function.

rdar://123484595
2024-02-27 13:14:47 -08:00
Arnold Schwaighofer a17d79cd72 LargeTypesReg2Mem: Fix copy_addr optimization
Make sure that the value we store is the loaded value ...
rdar://123340151
2024-02-21 13:26:07 -08:00
Arnold Schwaighofer 7f1fc90df7 LargeTypesReg2Mem: Gracefully handle the case when the Container type is deemed small but the Containee is not
rdar://123340151
2024-02-21 12:38:57 -08:00
Arnold Schwaighofer 216b3c8c9d LargeTypesReg2Mem: Remove overzealous assert
Functions with c convention can have large direct arguments that are not
re-written by the LoadableByAddress pass before they get to reg2Mem
optimization.

https://github.com/apple/swift/issues/71757
2024-02-20 16:44:30 -08:00
John McCall d5142668f4 SIL and IRGen support for @isolated(any). SILGen to come. 2024-02-13 03:04:13 -05:00
Arnold Schwaighofer ed7dead648 LargeTypesReg2Mem: Handle the optional projections with larger payloads
rdar://122598081
2024-02-09 13:09:03 -08:00
Andrew Trick da3f583492 Add mark_dependence [unresolved]
In preparation for inserting mark_dependence instructions for lifetime
dependencies early, immediately after SILGen. That will simplify the
implementation of borrowed arguments.

Marking them unresolved is needed to make OSSA verification
conservative until lifetime dependence diagnostics runs.
2024-02-08 22:53:16 -08:00
Allan Shortlidge cd122d2075 NFC: Suppress a -Wunreachable-code-loop-increment warning. 2024-02-01 18:16:05 -08:00
Arnold Schwaighofer 5e2144bdb8 Large loadable types peepholes and address assignment
Contract some load store patterns and assign addresses to (non-address)
SIL SSA values.

rdar://115571407
2024-01-29 17:10:46 -08:00
Andrew Trick 0cf47e10ec Fix LoadableByAddress assert: missing case.
Pattern:

    %f = function_ref @invocationClosure : $@convention(thin) (BigStruct) -> ()
    %s = struct $InvocationWrapper (%f : $@convention(thin) (BigStruct) -> ())

Error:

    Unhandled use of FunctionRefInst
    UNREACHABLE executed at /swift/lib/IRGen/LoadableByAddress.cpp:3057!

This fix simply adds the unrecognized case. Rewriting a function_ref
does not change its value, so I'm not sure why the assert exists in
the first place.
2024-01-14 16:31:12 -08:00