Commit Graph

216 Commits

Author SHA1 Message Date
Arnold Schwaighofer
7aef2b549c Fix static variable initializations 2025-01-09 11:42:04 -08:00
Arnold Schwaighofer
abbe36a1e9 LoadableByAddress: Replace some magic constants
NFC intended. Although there is a little fix in setNumRegisters.
2025-01-06 11:57:37 -08:00
Arnold Schwaighofer
be5223f0fa LargeLoadable types: propagate large type property along projections
Back propagate the isLargeType property along projections. This is
neccessary because c union types can "hide" largeness.

rdar://141775951
2024-12-20 09:09:42 -08:00
Nate Chandler
79c62dd5a0 [CoroutineAccessors] Rewrite allocs in LBA.
When replacing a `begin_apply` with an allocation, replace uses of the
old allocation with uses of the new.
2024-11-19 12:57:49 -08:00
Arnold Schwaighofer
ec5c432e09 LargeTypesReg2Mem: Don't ignore BuiltinFixedArrayType
rdar://139457907
2024-11-07 15:30:17 -08:00
Arnold Schwaighofer
d925eefdba IRGen: Fix a map entry use-after-free issue in LargeLoadableHeuristic
rdar://139398409
2024-11-07 09:51:16 -08:00
Arnold Schwaighofer
6755d0e104 LargeTypesReg2Mem: Ammend heuristic to include projections of block
arguments
2024-11-04 17:06:24 -08:00
Arnold Schwaighofer
787c996394 LargeTypesReg2Mem: Add a new heuristic that trys harder to keep large
values on the stack

This heuristic can be enabled by passing -Xfrontend
-enable-aggressive-reg2mem.

rdar://123916109
2024-10-31 13:22:06 -07:00
Arnold Schwaighofer
e909720e4d LargeTypesReg2Mem: Map undef SIL values to an undefined address on the stack
SIL can have undefined values along some path (which dynamically is
never executed).

rdar://136600129
2024-09-24 15:43:38 -07:00
Arnold Schwaighofer
eeaf9621f2 [loadable by address reg2mem] Implement missing throw instruction
rdar://134914984
2024-09-05 08:12:32 -07:00
Tim Kientzle
598e5104ef Merge pull request #74184 from tbkka/tbkka-assertions2
Add `#include "swift/Basic/Assertions.h"` to a lot of source files
2024-06-20 12:13:28 -07:00
Arnold Schwaighofer
bddc50ef9e LargeTypesReg2Mem: Handle mark_dependence
This instruction is a no-op. We can just delete it.

rdar://129873110
2024-06-19 09:36:43 -07:00
Tim Kientzle
1098054291 Merge branch 'main' into tbkka-assertions2 2024-06-18 17:52:00 -07:00
Akira Hatanaka
d92f181ace Create two versions (for caller and callee) of the functions that answer questions about parameter convention (#74124)
Create two versions of the following functions:

isConsumedParameter
isGuaranteedParameter
SILParameterInfo::isConsumed
SILParameterInfo::isGuaranteed
SILArgumentConvention::isOwnedConvention
SILArgumentConvention::isGuaranteedConvention

These changes will be needed when we add a new convention for
non-trivial C++ types as the functions will return different answers
depending on whether they are called for the caller or the callee. This
commit doesn't change any functionality.
2024-06-18 09:06:09 -07:00
Tim Kientzle
1d961ba22d Add #include "swift/Basic/Assertions.h" to a lot of source files
Although I don't plan to bring over new assertions wholesale
into the current qualification branch, it's entirely possible
that various minor changes in main will use the new assertions;
having this basic support in the release branch will simplify that.
(This is why I'm adding the includes as a separate pass from
rewriting the individual assertions)
2024-06-05 19:37:30 -07:00
Arnold Schwaighofer
d89bf2893b IRGen: Properly adjust the closure type of a partial_apply of an objc_method
It needs to match with the (large loadable) lowered closure type in the rest of
the program: Large types in the signature need to be passed indirectly.

rdar://127367321
2024-05-22 09:39:15 -07:00
Arnold Schwaighofer
b5d9ff4557 Merge pull request #73626 from aschwaighofer/large_types_reg2mem_unchecked_trivial_bitcast_fix
IRGen: Large types reg2mem - Fix visitUncheckedTrivialBitCastInst
2024-05-15 20:16:05 -07:00
Slava Pestov
a356b10929 SIL: Sink local archetype substitution into remapType() 2024-05-15 17:39:00 -04:00
Arnold Schwaighofer
1258563c3b IRGen: Large types reg2mem - Fix visitUncheckedTrivialBitCastInst
An unchecked_trivial_bit_cast can go from a bigger type to a smaller
type. Therefore we must allocate stack storage for the operand type
rather than the result type. Otherwise, we can end up storing bigger
values into smaller storage -- not good.

rdar://128086028
2024-05-14 13:45:45 -07:00
Anton Korobeynikov
e7e2ad19f1 Ensure we are using mapped SIL type for switch_enum case and not the original lowered one (#73385)
Fixes #73018
2024-05-09 13:41:34 -07:00
Emil Pedersen
644509ea1c [DebugInfo] Fix LoadableByAddress losing debug info
LoadableByAddress was losing debug info, including at -Onone.
When converting a load-store pair to a copy_addr, the debug info
attached to the load was not salvaged.
Additionally, the wrong scope was being attached to other debug
values.
2024-05-01 18:44:33 -07:00
Arnold Schwaighofer
ff4682941a LargeTypesReg2Mem: Fix yield and begin_apply instructions
Speculative fix for #swift/issues/72911
2024-04-25 16:51:33 -07:00
Arnold Schwaighofer
51e148d153 Merge pull request #72694 from aschwaighofer/c_array_heuristic
LargeTypesReg2Mem: Add a heuristic for C character arrays
2024-03-29 12:43:20 -07:00
Arnold Schwaighofer
081a872b81 LargeTypesReg2Mem: Add a heuristic for C character arrays
They can lower to a set of i64 registers we consider small but exploded
to (manifested as SSA) registers they will cause significant code bloat.

rdar://125265576
2024-03-29 05:35:26 -07:00
Anton Korobeynikov
2ad4a60619 Ensure LoadableByAddress rewrites pointer_to_address (#72619)
if the target address is address of tuple type that should be rewritten.

Fixes #72363
2024-03-27 12:12:47 -07:00
Anton Korobeynikov
a63078f279 Add missed case to AssignAddressToDef. (#72617)
Fixes #71744
2024-03-27 10:49:48 -07:00
Michael Gottesman
11f0ff6e32 [sil] Ensure that all SILValues have a parent function by making it so that SILUndef is uniqued at the function instead of module level.
For years, optimizer engineers have been hitting a common bug caused by passes
assuming all SILValues have a parent function only to be surprised by SILUndef.
Generally we see SILUndef not that often so we see this come up later in
testing. This patch eliminates that problem by making SILUndef uniqued at the
function level instead of the module level. This ensures that it makes sense for
SILUndef to have a parent function, eliminating this possibility since we can
define an API to get its parent function.

rdar://123484595
2024-02-27 13:14:47 -08:00
Arnold Schwaighofer
a17d79cd72 LargeTypesReg2Mem: Fix copy_addr optimization
Make sure that the value we store is the loaded value ...
rdar://123340151
2024-02-21 13:26:07 -08:00
Arnold Schwaighofer
7f1fc90df7 LargeTypesReg2Mem: Gracefully handle the case when the Container type is deemed small but the Containee is not
rdar://123340151
2024-02-21 12:38:57 -08:00
Arnold Schwaighofer
216b3c8c9d LargeTypesReg2Mem: Remove overzealous assert
Functions with c convention can have large direct arguments that are not
re-written by the LoadableByAddress pass before they get to reg2Mem
optimization.

https://github.com/apple/swift/issues/71757
2024-02-20 16:44:30 -08:00
John McCall
d5142668f4 SIL and IRGen support for @isolated(any). SILGen to come. 2024-02-13 03:04:13 -05:00
Arnold Schwaighofer
ed7dead648 LargeTypesReg2Mem: Handle the optional projections with larger payloads
rdar://122598081
2024-02-09 13:09:03 -08:00
Andrew Trick
da3f583492 Add mark_dependence [unresolved]
In preparation for inserting mark_dependence instructions for lifetime
dependencies early, immediately after SILGen. That will simplify the
implementation of borrowed arguments.

Marking them unresolved is needed to make OSSA verification
conservative until lifetime dependence diagnostics runs.
2024-02-08 22:53:16 -08:00
Allan Shortlidge
cd122d2075 NFC: Suppress a -Wunreachable-code-loop-increment warning. 2024-02-01 18:16:05 -08:00
Arnold Schwaighofer
5e2144bdb8 Large loadable types peepholes and address assignment
Contract some load store patterns and assign addresses to (non-address)
SIL SSA values.

rdar://115571407
2024-01-29 17:10:46 -08:00
Andrew Trick
0cf47e10ec Fix LoadableByAddress assert: missing case.
Pattern:

    %f = function_ref @invocationClosure : $@convention(thin) (BigStruct) -> ()
    %s = struct $InvocationWrapper (%f : $@convention(thin) (BigStruct) -> ())

Error:

    Unhandled use of FunctionRefInst
    UNREACHABLE executed at /swift/lib/IRGen/LoadableByAddress.cpp:3057!

This fix simply adds the unrecognized case. Rewriting a function_ref
does not change its value, so I'm not sure why the assert exists in
the first place.
2024-01-14 16:31:12 -08:00
Erik Eckstein
bc99986cf9 SIL: add a dependency token operand to global_addr
Optionally, the dependency to the initialization of the global can be specified with a dependency token `depends_on <token>`.
This is usually a `builtin "once"` which calls the initializer for the global variable.
2024-01-10 09:33:58 +01:00
Andrew Trick
264cbaea42 Add mark_dependence [nonescaping] flag.
The dependent 'value' may be marked 'nonescaping', which guarantees that the
lifetime dependence is statically enforceable. In this case, the compiler
must be able to follow all values forwarded from the dependent 'value', and
recognize all final (non-forwarded, non-escaping) use points. This implies
that `findPointerEscape` is false. A diagnostic pass checks that the
incoming SIL to verify that these use points are all initially within the
'base' lifetime. Regular 'mark_dependence' semantics ensure that
optimizations cannot violate the lifetime dependence after diagnostics.
2024-01-04 14:47:35 -08:00
Michael Gottesman
1764bd2479 [sil] Change SILParameterInfo/SILResultInfo's differentiability parameter to use an OptionSet so we can add other options.
I am doing this in preparation for adding options to SILParameterInfo/
SILResultInfo that state that a parameter/result is transferring. Even though I
could have just introduced a new bit here, I instead streamlined the interface
of SILParameterInfo/SILResultInfo to use an OptionSet instead of individual bits
to make it easier to add new flags here. The reason why it is easier is that
along API (e.x.: function argument) boundaries one does not have to marshal each
field or pass each field. Instead one can just pass the whole OptionSet as an
opaque thing. Using this I was able to change serialization/deserialization of
SILParameterInfo/SILResultInfo so that one does not need to update them if one
adds new fields!

The reason why I am doing this for both SILParameterInfo/SILResultInfo in the
same commit is because they share code in the demangler that I did not want to
have to duplicate in an intervening commit. By changing them both at the same
type, I didn't have to change anything without an actual need to.

I am doing this in a separate commit from adding transferring support so I can
validate correctness using the tests for the options already supported
(currently only differentiability).
2024-01-02 15:03:05 -08:00
Andrew Trick
61c12e4be0 Fix LoadableByAddress to use @in_guaranteed for unowned values
LoadableByAddress was accidentally changing ownership of
direct_unowned values to @in (owned). This generates unsupported SIL
for on-stack partial applies, which now breaks SIL verification.

This also resulted in extra copies of values inside of closure
contexts. Before calling the original function, the value would need to
be copied onto the stack and the context would be destroyed. Now, we
simply pass a pointer directly from the closure context.  See
IRGen/indirect_argument.sil+huge_partial_application.

I'm pretty sure fixing this has no effect on the mangling of public symbols.
2023-08-12 21:21:18 -07:00
Erik Eckstein
625619ee17 SIL: add a bare attribute to global_value
The `bare` attribute indicates that the object header is not used throughout the lifetime of the value.
This means, no reference counting operations are performed on the object and its metadata is not used.
The header of bare objects doesn't need to be initialized.
2023-06-29 06:57:05 +02:00
Meghana Gupta
16c300c2af Remove OwnershipForwardingConversionInst, ConversionInst.
Add new ConversionOperation abstraction, use this in place of ConversionInst
2023-06-15 10:53:28 -07:00
Erik Eckstein
40a7527b67 LoadableByAddress: convert types of functions which are contained in structs inside global static initializers
We missed converting such types inside static initializers of global variables.
This results in ptrauth crashes when ptrauth is enabled.

rdar://108165425
2023-04-28 08:46:08 +02:00
Adrian Prantl
158772c2ab Rebase SILScope generation on top of ASTScope.
This patch replaces the stateful generation of SILScope information in
SILGenFunction with data derived from the ASTScope hierarchy, which should be
100% in sync with the scopes needed for local variables. The goal is to
eliminate the surprising effects that the stack of cleanup operations can have
on the current state of SILBuilder leading to a fully deterministic (in the
sense of: predictible by a human) association of SILDebugScopes with
SILInstructions. The patch also eliminates the need to many workarounds. There
are still some accomodations for several Sema transformation passes such as
ResultBuilders, which don't correctly update the source locations when moving
around nodes. If these were implemented as macros, this problem would disappear.

This necessary rewrite of the macro scope handling included in this patch also
adds proper support nested macro expansions.

This fixes

rdar://88274783

and either fixes or at least partially addresses the following:

rdar://89252827
rdar://105186946
rdar://105757810
rdar://105997826
rdar://105102288
2023-04-04 15:20:11 -07:00
Nate Chandler
8d8577e5b0 [SIL] Removed Indirect_In_Constant convention.
It is no different from @in.

Continue parse @in_constant in textual and serialized SIL, but just as
an alias for @in.
2022-12-09 21:54:00 -08:00
Alex Lorenz
4c7015ad68 [interop][SwiftToCxx] ensure LoadableByAddress lowering is done on the SILFunction for while we're lowering the signature for
This correctly now passes self parameters for direct values that are too large to be direct
2022-09-07 17:42:50 -07:00
Arnold Schwaighofer
58420a164b LoadableByAddress: Updating global variables' types
Because of pointer authentication the type of global variables needs to
be updated.

rdar://93688980
2022-08-09 09:03:03 -07:00
Hamish Knight
9da53193da [AST] Remove ParameterTypeFlags from ParenType and TupleType
The last clients that relied on stashing parameter
type flags on these types are now gone.
2022-08-02 13:56:32 +01:00
Joe Groff
47201cb4f2 Merge pull request #58713 from jckarter/sil-combiner-apply-result-conversions
SILCombine: Handle result conversions for apply (convert_function) peephole
2022-05-10 13:05:56 -07:00
Joe Groff
2a668b9500 LoadableByAddress: Fix up tuple projections by changing insn type
instead of bitcasting the original type. Once the original tuple value is
rewritten, the projected element will be the type we want, so the
original instruction would be invalid anyway, and we can make
a new instruction with the correct type directly.
2022-05-07 10:52:41 -07:00