Commit Graph

224 Commits

Author SHA1 Message Date
Arnold Schwaighofer
78a3393f72 Add more information to backtraces from the LoadableByAddress pass
Should not change functionality outside of displaying more information in backtraces.
2025-10-24 06:31:07 -07:00
Arnold Schwaighofer
1e8e3b82df LargeTypesReg2Mem: Peephole load followed by single user to reuse the load's address
For example the pattern:

```
%e = load %1 : SomeEnum
switch_enum %e
```

The switch_enum_addr should reuse the load's address operand instead of
creating a new location for the SSA value of %e.

While we are there also don't create a new stack location for extracting payload
in a switch_enum_addr case target basic block if that extracted payload is not
used. That is if the basic block arguments in all switch target blocks are
unused in the switch_enum version.

```
switch_enum %enum, case #Optional.some!enumelt: bb1,
                   case #Optional.none!enumelt: bb2

bb2(%payload : $Payload):
   // unused %payload value
```

rdar://156602951
2025-08-07 07:21:51 -07:00
Meghana Gupta
e317a603fc Add simplification for end_cow_mutation_addr
We insert end_cow_mutation_addr for lifetime dependent values dependent on mutable addresses.
end_cow_mutation_addr can be simplified to end_cow_mutation after other optimizations like inlining, specialization etc

This PR adds an instruction simplification to transform end_cow_mutation_addr to end_cow_mutation.
This can enable array optimizations which look for end_cow_mutation.
2025-07-14 13:46:13 -07:00
Arnold Schwaighofer
521fb07523 LoadableByAddress: Fix shouldTransformYields to use substituted types
rdar://149281263
issues/80818
2025-04-21 13:46:41 -07:00
Arnold Schwaighofer
370b7e8219 LargeTypesReg2Mem: Add unchecked_bitwise_cast to the projections we need to propagate largeness from destination to source operand
rdar://148545382
2025-04-17 09:18:31 -07:00
John McCall
3fe70968cc Remove the substitution map from zeroInitializer builtin in SIL.
This is a value operation that can work just fine on lowered types,
so there's no need to carry along a formal type. Make the value/address
duality clearer, and enforce it in the verifier.
2025-03-26 00:34:15 -04:00
Arnold Schwaighofer
5feb287d19 LoadableByAddress: Add missing BranchInst in a switch
rdar://146980802
2025-03-19 09:50:02 -07:00
Arnold Schwaighofer
8120fa3989 LoadableByAddress: Must be able to materialize undef operands
rdar://145092298
2025-02-19 13:59:32 -08:00
Arnold Schwaighofer
7aef2b549c Fix static variable initializations 2025-01-09 11:42:04 -08:00
Arnold Schwaighofer
abbe36a1e9 LoadableByAddress: Replace some magic constants
NFC intended. Although there is a little fix in setNumRegisters.
2025-01-06 11:57:37 -08:00
Arnold Schwaighofer
be5223f0fa LargeLoadable types: propagate large type property along projections
Back propagate the isLargeType property along projections. This is
neccessary because c union types can "hide" largeness.

rdar://141775951
2024-12-20 09:09:42 -08:00
Nate Chandler
79c62dd5a0 [CoroutineAccessors] Rewrite allocs in LBA.
When replacing a `begin_apply` with an allocation, replace uses of the
old allocation with uses of the new.
2024-11-19 12:57:49 -08:00
Arnold Schwaighofer
ec5c432e09 LargeTypesReg2Mem: Don't ignore BuiltinFixedArrayType
rdar://139457907
2024-11-07 15:30:17 -08:00
Arnold Schwaighofer
d925eefdba IRGen: Fix a map entry use-after-free issue in LargeLoadableHeuristic
rdar://139398409
2024-11-07 09:51:16 -08:00
Arnold Schwaighofer
6755d0e104 LargeTypesReg2Mem: Ammend heuristic to include projections of block
arguments
2024-11-04 17:06:24 -08:00
Arnold Schwaighofer
787c996394 LargeTypesReg2Mem: Add a new heuristic that trys harder to keep large
values on the stack

This heuristic can be enabled by passing -Xfrontend
-enable-aggressive-reg2mem.

rdar://123916109
2024-10-31 13:22:06 -07:00
Arnold Schwaighofer
e909720e4d LargeTypesReg2Mem: Map undef SIL values to an undefined address on the stack
SIL can have undefined values along some path (which dynamically is
never executed).

rdar://136600129
2024-09-24 15:43:38 -07:00
Arnold Schwaighofer
eeaf9621f2 [loadable by address reg2mem] Implement missing throw instruction
rdar://134914984
2024-09-05 08:12:32 -07:00
Tim Kientzle
598e5104ef Merge pull request #74184 from tbkka/tbkka-assertions2
Add `#include "swift/Basic/Assertions.h"` to a lot of source files
2024-06-20 12:13:28 -07:00
Arnold Schwaighofer
bddc50ef9e LargeTypesReg2Mem: Handle mark_dependence
This instruction is a no-op. We can just delete it.

rdar://129873110
2024-06-19 09:36:43 -07:00
Tim Kientzle
1098054291 Merge branch 'main' into tbkka-assertions2 2024-06-18 17:52:00 -07:00
Akira Hatanaka
d92f181ace Create two versions (for caller and callee) of the functions that answer questions about parameter convention (#74124)
Create two versions of the following functions:

isConsumedParameter
isGuaranteedParameter
SILParameterInfo::isConsumed
SILParameterInfo::isGuaranteed
SILArgumentConvention::isOwnedConvention
SILArgumentConvention::isGuaranteedConvention

These changes will be needed when we add a new convention for
non-trivial C++ types as the functions will return different answers
depending on whether they are called for the caller or the callee. This
commit doesn't change any functionality.
2024-06-18 09:06:09 -07:00
Tim Kientzle
1d961ba22d Add #include "swift/Basic/Assertions.h" to a lot of source files
Although I don't plan to bring over new assertions wholesale
into the current qualification branch, it's entirely possible
that various minor changes in main will use the new assertions;
having this basic support in the release branch will simplify that.
(This is why I'm adding the includes as a separate pass from
rewriting the individual assertions)
2024-06-05 19:37:30 -07:00
Arnold Schwaighofer
d89bf2893b IRGen: Properly adjust the closure type of a partial_apply of an objc_method
It needs to match with the (large loadable) lowered closure type in the rest of
the program: Large types in the signature need to be passed indirectly.

rdar://127367321
2024-05-22 09:39:15 -07:00
Arnold Schwaighofer
b5d9ff4557 Merge pull request #73626 from aschwaighofer/large_types_reg2mem_unchecked_trivial_bitcast_fix
IRGen: Large types reg2mem - Fix visitUncheckedTrivialBitCastInst
2024-05-15 20:16:05 -07:00
Slava Pestov
a356b10929 SIL: Sink local archetype substitution into remapType() 2024-05-15 17:39:00 -04:00
Arnold Schwaighofer
1258563c3b IRGen: Large types reg2mem - Fix visitUncheckedTrivialBitCastInst
An unchecked_trivial_bit_cast can go from a bigger type to a smaller
type. Therefore we must allocate stack storage for the operand type
rather than the result type. Otherwise, we can end up storing bigger
values into smaller storage -- not good.

rdar://128086028
2024-05-14 13:45:45 -07:00
Anton Korobeynikov
e7e2ad19f1 Ensure we are using mapped SIL type for switch_enum case and not the original lowered one (#73385)
Fixes #73018
2024-05-09 13:41:34 -07:00
Emil Pedersen
644509ea1c [DebugInfo] Fix LoadableByAddress losing debug info
LoadableByAddress was losing debug info, including at -Onone.
When converting a load-store pair to a copy_addr, the debug info
attached to the load was not salvaged.
Additionally, the wrong scope was being attached to other debug
values.
2024-05-01 18:44:33 -07:00
Arnold Schwaighofer
ff4682941a LargeTypesReg2Mem: Fix yield and begin_apply instructions
Speculative fix for #swift/issues/72911
2024-04-25 16:51:33 -07:00
Arnold Schwaighofer
51e148d153 Merge pull request #72694 from aschwaighofer/c_array_heuristic
LargeTypesReg2Mem: Add a heuristic for C character arrays
2024-03-29 12:43:20 -07:00
Arnold Schwaighofer
081a872b81 LargeTypesReg2Mem: Add a heuristic for C character arrays
They can lower to a set of i64 registers we consider small but exploded
to (manifested as SSA) registers they will cause significant code bloat.

rdar://125265576
2024-03-29 05:35:26 -07:00
Anton Korobeynikov
2ad4a60619 Ensure LoadableByAddress rewrites pointer_to_address (#72619)
if the target address is address of tuple type that should be rewritten.

Fixes #72363
2024-03-27 12:12:47 -07:00
Anton Korobeynikov
a63078f279 Add missed case to AssignAddressToDef. (#72617)
Fixes #71744
2024-03-27 10:49:48 -07:00
Michael Gottesman
11f0ff6e32 [sil] Ensure that all SILValues have a parent function by making it so that SILUndef is uniqued at the function instead of module level.
For years, optimizer engineers have been hitting a common bug caused by passes
assuming all SILValues have a parent function only to be surprised by SILUndef.
Generally we see SILUndef not that often so we see this come up later in
testing. This patch eliminates that problem by making SILUndef uniqued at the
function level instead of the module level. This ensures that it makes sense for
SILUndef to have a parent function, eliminating this possibility since we can
define an API to get its parent function.

rdar://123484595
2024-02-27 13:14:47 -08:00
Arnold Schwaighofer
a17d79cd72 LargeTypesReg2Mem: Fix copy_addr optimization
Make sure that the value we store is the loaded value ...
rdar://123340151
2024-02-21 13:26:07 -08:00
Arnold Schwaighofer
7f1fc90df7 LargeTypesReg2Mem: Gracefully handle the case when the Container type is deemed small but the Containee is not
rdar://123340151
2024-02-21 12:38:57 -08:00
Arnold Schwaighofer
216b3c8c9d LargeTypesReg2Mem: Remove overzealous assert
Functions with c convention can have large direct arguments that are not
re-written by the LoadableByAddress pass before they get to reg2Mem
optimization.

https://github.com/apple/swift/issues/71757
2024-02-20 16:44:30 -08:00
John McCall
d5142668f4 SIL and IRGen support for @isolated(any). SILGen to come. 2024-02-13 03:04:13 -05:00
Arnold Schwaighofer
ed7dead648 LargeTypesReg2Mem: Handle the optional projections with larger payloads
rdar://122598081
2024-02-09 13:09:03 -08:00
Andrew Trick
da3f583492 Add mark_dependence [unresolved]
In preparation for inserting mark_dependence instructions for lifetime
dependencies early, immediately after SILGen. That will simplify the
implementation of borrowed arguments.

Marking them unresolved is needed to make OSSA verification
conservative until lifetime dependence diagnostics runs.
2024-02-08 22:53:16 -08:00
Allan Shortlidge
cd122d2075 NFC: Suppress a -Wunreachable-code-loop-increment warning. 2024-02-01 18:16:05 -08:00
Arnold Schwaighofer
5e2144bdb8 Large loadable types peepholes and address assignment
Contract some load store patterns and assign addresses to (non-address)
SIL SSA values.

rdar://115571407
2024-01-29 17:10:46 -08:00
Andrew Trick
0cf47e10ec Fix LoadableByAddress assert: missing case.
Pattern:

    %f = function_ref @invocationClosure : $@convention(thin) (BigStruct) -> ()
    %s = struct $InvocationWrapper (%f : $@convention(thin) (BigStruct) -> ())

Error:

    Unhandled use of FunctionRefInst
    UNREACHABLE executed at /swift/lib/IRGen/LoadableByAddress.cpp:3057!

This fix simply adds the unrecognized case. Rewriting a function_ref
does not change its value, so I'm not sure why the assert exists in
the first place.
2024-01-14 16:31:12 -08:00
Erik Eckstein
bc99986cf9 SIL: add a dependency token operand to global_addr
Optionally, the dependency to the initialization of the global can be specified with a dependency token `depends_on <token>`.
This is usually a `builtin "once"` which calls the initializer for the global variable.
2024-01-10 09:33:58 +01:00
Andrew Trick
264cbaea42 Add mark_dependence [nonescaping] flag.
The dependent 'value' may be marked 'nonescaping', which guarantees that the
lifetime dependence is statically enforceable. In this case, the compiler
must be able to follow all values forwarded from the dependent 'value', and
recognize all final (non-forwarded, non-escaping) use points. This implies
that `findPointerEscape` is false. A diagnostic pass checks that the
incoming SIL to verify that these use points are all initially within the
'base' lifetime. Regular 'mark_dependence' semantics ensure that
optimizations cannot violate the lifetime dependence after diagnostics.
2024-01-04 14:47:35 -08:00
Michael Gottesman
1764bd2479 [sil] Change SILParameterInfo/SILResultInfo's differentiability parameter to use an OptionSet so we can add other options.
I am doing this in preparation for adding options to SILParameterInfo/
SILResultInfo that state that a parameter/result is transferring. Even though I
could have just introduced a new bit here, I instead streamlined the interface
of SILParameterInfo/SILResultInfo to use an OptionSet instead of individual bits
to make it easier to add new flags here. The reason why it is easier is that
along API (e.x.: function argument) boundaries one does not have to marshal each
field or pass each field. Instead one can just pass the whole OptionSet as an
opaque thing. Using this I was able to change serialization/deserialization of
SILParameterInfo/SILResultInfo so that one does not need to update them if one
adds new fields!

The reason why I am doing this for both SILParameterInfo/SILResultInfo in the
same commit is because they share code in the demangler that I did not want to
have to duplicate in an intervening commit. By changing them both at the same
type, I didn't have to change anything without an actual need to.

I am doing this in a separate commit from adding transferring support so I can
validate correctness using the tests for the options already supported
(currently only differentiability).
2024-01-02 15:03:05 -08:00
Andrew Trick
61c12e4be0 Fix LoadableByAddress to use @in_guaranteed for unowned values
LoadableByAddress was accidentally changing ownership of
direct_unowned values to @in (owned). This generates unsupported SIL
for on-stack partial applies, which now breaks SIL verification.

This also resulted in extra copies of values inside of closure
contexts. Before calling the original function, the value would need to
be copied onto the stack and the context would be destroyed. Now, we
simply pass a pointer directly from the closure context.  See
IRGen/indirect_argument.sil+huge_partial_application.

I'm pretty sure fixing this has no effect on the mangling of public symbols.
2023-08-12 21:21:18 -07:00
Erik Eckstein
625619ee17 SIL: add a bare attribute to global_value
The `bare` attribute indicates that the object header is not used throughout the lifetime of the value.
This means, no reference counting operations are performed on the object and its metadata is not used.
The header of bare objects doesn't need to be initialized.
2023-06-29 06:57:05 +02:00
Meghana Gupta
16c300c2af Remove OwnershipForwardingConversionInst, ConversionInst.
Add new ConversionOperation abstraction, use this in place of ConversionInst
2023-06-15 10:53:28 -07:00