Commit Graph

232 Commits

Author SHA1 Message Date
Michael Gottesman
a70733c222 [loadable-by-address] Update debug variable type when rewriting alloc_stack
The LoadableByAddress pass rewrites alloc_stack instructions when their element
type changes (e.g., a function parameter convention changes from @guaranteed to
@in_guaranteed for large/opaque types). However, it was passing the original
SILDebugVariable info through unchanged. When the VarInfo has an explicit Type
field set this creates a mismatch between the SSA type and the debug type,
causing a SIL verifier failure.

In this patch, I just change the pass to update the alloc_stack as
appropriate. I verified that this cannot happen with debug_value since we do not
store the extra type info in it if we have an object typed thing. Since
loadable_by_address only changes objects -> addresses, it can never happen.

rdar://164268891
2026-02-20 10:46:39 -08:00
Michael Gottesman
c785ec7bac [gardening] Fix typo in a function name. 2026-02-20 10:46:39 -08:00
Dario Rexin
6af54cd164 [IRGen] Return large loadable types as @guaranteed_address
rdar://167713693

The LoadableByAddress pass recognizes large loadable types in borrow accessors and returns them by @guaranteed_address.
This prevents stack allocations and unnecessary copies.

Co-authored-by: Meghana Gupta <meghana_gupta@apple.com>
2026-01-29 13:33:47 -08:00
Arnold Schwaighofer
4f19313f14 Merge pull request #86148 from aschwaighofer/large_types_reg2mem_large_inline_array_compile_time_fix
LargeTypesReg2Mem: Avoid building an explosion schema just to count the number of registers for very large types
2025-12-22 15:11:21 -08:00
Arnold Schwaighofer
f02d91395a Add some comments to constants 2025-12-19 12:30:33 -08:00
Arnold Schwaighofer
6c930432ad LargeTypesReg2Mem: Avoid building an explosion schema just to count the number of registers for very large types
Computing the explosion schema for large types can be expensive.

E.g the following example would spend multiple seconds to compute the
explosion scheme for the array type. It is not worth adding the
complexity to have a special case explosion schema that handles types
like that since exploding a value like that is going to be detrimental.

```
struct Test {
  private var values: [0xfffffff of [UInt32]] =
  InlineArray(repeating: [])
}
```

rdar://165202495
2025-12-19 12:07:48 -08:00
Erik Eckstein
daa8850a77 LoadableByAddress: handle the unchecked_bitwise_cast instruction
Fixes a compiler crash in LoadableByAddress.
Unfortunately I don't have a test case for this.
2025-12-19 17:44:01 +01:00
Slava Pestov
819738c83e AST: Rename mapTypeIntoContext() => mapTypeIntoEnvironment(), mapTypeOutOfContext() => mapTypeOutOfEnvironment() 2025-11-12 14:48:19 -05:00
Arnold Schwaighofer
78a3393f72 Add more information to backtraces from the LoadableByAddress pass
Should not change functionality outside of displaying more information in backtraces.
2025-10-24 06:31:07 -07:00
Arnold Schwaighofer
1e8e3b82df LargeTypesReg2Mem: Peephole load followed by single user to reuse the load's address
For example the pattern:

```
%e = load %1 : SomeEnum
switch_enum %e
```

The switch_enum_addr should reuse the load's address operand instead of
creating a new location for the SSA value of %e.

While we are there also don't create a new stack location for extracting payload
in a switch_enum_addr case target basic block if that extracted payload is not
used. That is if the basic block arguments in all switch target blocks are
unused in the switch_enum version.

```
switch_enum %enum, case #Optional.some!enumelt: bb1,
                   case #Optional.none!enumelt: bb2

bb2(%payload : $Payload):
   // unused %payload value
```

rdar://156602951
2025-08-07 07:21:51 -07:00
Meghana Gupta
e317a603fc Add simplification for end_cow_mutation_addr
We insert end_cow_mutation_addr for lifetime dependent values dependent on mutable addresses.
end_cow_mutation_addr can be simplified to end_cow_mutation after other optimizations like inlining, specialization etc

This PR adds an instruction simplification to transform end_cow_mutation_addr to end_cow_mutation.
This can enable array optimizations which look for end_cow_mutation.
2025-07-14 13:46:13 -07:00
Arnold Schwaighofer
521fb07523 LoadableByAddress: Fix shouldTransformYields to use substituted types
rdar://149281263
issues/80818
2025-04-21 13:46:41 -07:00
Arnold Schwaighofer
370b7e8219 LargeTypesReg2Mem: Add unchecked_bitwise_cast to the projections we need to propagate largeness from destination to source operand
rdar://148545382
2025-04-17 09:18:31 -07:00
John McCall
3fe70968cc Remove the substitution map from zeroInitializer builtin in SIL.
This is a value operation that can work just fine on lowered types,
so there's no need to carry along a formal type. Make the value/address
duality clearer, and enforce it in the verifier.
2025-03-26 00:34:15 -04:00
Arnold Schwaighofer
5feb287d19 LoadableByAddress: Add missing BranchInst in a switch
rdar://146980802
2025-03-19 09:50:02 -07:00
Arnold Schwaighofer
8120fa3989 LoadableByAddress: Must be able to materialize undef operands
rdar://145092298
2025-02-19 13:59:32 -08:00
Arnold Schwaighofer
7aef2b549c Fix static variable initializations 2025-01-09 11:42:04 -08:00
Arnold Schwaighofer
abbe36a1e9 LoadableByAddress: Replace some magic constants
NFC intended. Although there is a little fix in setNumRegisters.
2025-01-06 11:57:37 -08:00
Arnold Schwaighofer
be5223f0fa LargeLoadable types: propagate large type property along projections
Back propagate the isLargeType property along projections. This is
neccessary because c union types can "hide" largeness.

rdar://141775951
2024-12-20 09:09:42 -08:00
Nate Chandler
79c62dd5a0 [CoroutineAccessors] Rewrite allocs in LBA.
When replacing a `begin_apply` with an allocation, replace uses of the
old allocation with uses of the new.
2024-11-19 12:57:49 -08:00
Arnold Schwaighofer
ec5c432e09 LargeTypesReg2Mem: Don't ignore BuiltinFixedArrayType
rdar://139457907
2024-11-07 15:30:17 -08:00
Arnold Schwaighofer
d925eefdba IRGen: Fix a map entry use-after-free issue in LargeLoadableHeuristic
rdar://139398409
2024-11-07 09:51:16 -08:00
Arnold Schwaighofer
6755d0e104 LargeTypesReg2Mem: Ammend heuristic to include projections of block
arguments
2024-11-04 17:06:24 -08:00
Arnold Schwaighofer
787c996394 LargeTypesReg2Mem: Add a new heuristic that trys harder to keep large
values on the stack

This heuristic can be enabled by passing -Xfrontend
-enable-aggressive-reg2mem.

rdar://123916109
2024-10-31 13:22:06 -07:00
Arnold Schwaighofer
e909720e4d LargeTypesReg2Mem: Map undef SIL values to an undefined address on the stack
SIL can have undefined values along some path (which dynamically is
never executed).

rdar://136600129
2024-09-24 15:43:38 -07:00
Arnold Schwaighofer
eeaf9621f2 [loadable by address reg2mem] Implement missing throw instruction
rdar://134914984
2024-09-05 08:12:32 -07:00
Tim Kientzle
598e5104ef Merge pull request #74184 from tbkka/tbkka-assertions2
Add `#include "swift/Basic/Assertions.h"` to a lot of source files
2024-06-20 12:13:28 -07:00
Arnold Schwaighofer
bddc50ef9e LargeTypesReg2Mem: Handle mark_dependence
This instruction is a no-op. We can just delete it.

rdar://129873110
2024-06-19 09:36:43 -07:00
Tim Kientzle
1098054291 Merge branch 'main' into tbkka-assertions2 2024-06-18 17:52:00 -07:00
Akira Hatanaka
d92f181ace Create two versions (for caller and callee) of the functions that answer questions about parameter convention (#74124)
Create two versions of the following functions:

isConsumedParameter
isGuaranteedParameter
SILParameterInfo::isConsumed
SILParameterInfo::isGuaranteed
SILArgumentConvention::isOwnedConvention
SILArgumentConvention::isGuaranteedConvention

These changes will be needed when we add a new convention for
non-trivial C++ types as the functions will return different answers
depending on whether they are called for the caller or the callee. This
commit doesn't change any functionality.
2024-06-18 09:06:09 -07:00
Tim Kientzle
1d961ba22d Add #include "swift/Basic/Assertions.h" to a lot of source files
Although I don't plan to bring over new assertions wholesale
into the current qualification branch, it's entirely possible
that various minor changes in main will use the new assertions;
having this basic support in the release branch will simplify that.
(This is why I'm adding the includes as a separate pass from
rewriting the individual assertions)
2024-06-05 19:37:30 -07:00
Arnold Schwaighofer
d89bf2893b IRGen: Properly adjust the closure type of a partial_apply of an objc_method
It needs to match with the (large loadable) lowered closure type in the rest of
the program: Large types in the signature need to be passed indirectly.

rdar://127367321
2024-05-22 09:39:15 -07:00
Arnold Schwaighofer
b5d9ff4557 Merge pull request #73626 from aschwaighofer/large_types_reg2mem_unchecked_trivial_bitcast_fix
IRGen: Large types reg2mem - Fix visitUncheckedTrivialBitCastInst
2024-05-15 20:16:05 -07:00
Slava Pestov
a356b10929 SIL: Sink local archetype substitution into remapType() 2024-05-15 17:39:00 -04:00
Arnold Schwaighofer
1258563c3b IRGen: Large types reg2mem - Fix visitUncheckedTrivialBitCastInst
An unchecked_trivial_bit_cast can go from a bigger type to a smaller
type. Therefore we must allocate stack storage for the operand type
rather than the result type. Otherwise, we can end up storing bigger
values into smaller storage -- not good.

rdar://128086028
2024-05-14 13:45:45 -07:00
Anton Korobeynikov
e7e2ad19f1 Ensure we are using mapped SIL type for switch_enum case and not the original lowered one (#73385)
Fixes #73018
2024-05-09 13:41:34 -07:00
Emil Pedersen
644509ea1c [DebugInfo] Fix LoadableByAddress losing debug info
LoadableByAddress was losing debug info, including at -Onone.
When converting a load-store pair to a copy_addr, the debug info
attached to the load was not salvaged.
Additionally, the wrong scope was being attached to other debug
values.
2024-05-01 18:44:33 -07:00
Arnold Schwaighofer
ff4682941a LargeTypesReg2Mem: Fix yield and begin_apply instructions
Speculative fix for #swift/issues/72911
2024-04-25 16:51:33 -07:00
Arnold Schwaighofer
51e148d153 Merge pull request #72694 from aschwaighofer/c_array_heuristic
LargeTypesReg2Mem: Add a heuristic for C character arrays
2024-03-29 12:43:20 -07:00
Arnold Schwaighofer
081a872b81 LargeTypesReg2Mem: Add a heuristic for C character arrays
They can lower to a set of i64 registers we consider small but exploded
to (manifested as SSA) registers they will cause significant code bloat.

rdar://125265576
2024-03-29 05:35:26 -07:00
Anton Korobeynikov
2ad4a60619 Ensure LoadableByAddress rewrites pointer_to_address (#72619)
if the target address is address of tuple type that should be rewritten.

Fixes #72363
2024-03-27 12:12:47 -07:00
Anton Korobeynikov
a63078f279 Add missed case to AssignAddressToDef. (#72617)
Fixes #71744
2024-03-27 10:49:48 -07:00
Michael Gottesman
11f0ff6e32 [sil] Ensure that all SILValues have a parent function by making it so that SILUndef is uniqued at the function instead of module level.
For years, optimizer engineers have been hitting a common bug caused by passes
assuming all SILValues have a parent function only to be surprised by SILUndef.
Generally we see SILUndef not that often so we see this come up later in
testing. This patch eliminates that problem by making SILUndef uniqued at the
function level instead of the module level. This ensures that it makes sense for
SILUndef to have a parent function, eliminating this possibility since we can
define an API to get its parent function.

rdar://123484595
2024-02-27 13:14:47 -08:00
Arnold Schwaighofer
a17d79cd72 LargeTypesReg2Mem: Fix copy_addr optimization
Make sure that the value we store is the loaded value ...
rdar://123340151
2024-02-21 13:26:07 -08:00
Arnold Schwaighofer
7f1fc90df7 LargeTypesReg2Mem: Gracefully handle the case when the Container type is deemed small but the Containee is not
rdar://123340151
2024-02-21 12:38:57 -08:00
Arnold Schwaighofer
216b3c8c9d LargeTypesReg2Mem: Remove overzealous assert
Functions with c convention can have large direct arguments that are not
re-written by the LoadableByAddress pass before they get to reg2Mem
optimization.

https://github.com/apple/swift/issues/71757
2024-02-20 16:44:30 -08:00
John McCall
d5142668f4 SIL and IRGen support for @isolated(any). SILGen to come. 2024-02-13 03:04:13 -05:00
Arnold Schwaighofer
ed7dead648 LargeTypesReg2Mem: Handle the optional projections with larger payloads
rdar://122598081
2024-02-09 13:09:03 -08:00
Andrew Trick
da3f583492 Add mark_dependence [unresolved]
In preparation for inserting mark_dependence instructions for lifetime
dependencies early, immediately after SILGen. That will simplify the
implementation of borrowed arguments.

Marking them unresolved is needed to make OSSA verification
conservative until lifetime dependence diagnostics runs.
2024-02-08 22:53:16 -08:00
Allan Shortlidge
cd122d2075 NFC: Suppress a -Wunreachable-code-loop-increment warning. 2024-02-01 18:16:05 -08:00