If an enum case has a payload but the unsubstituted payload type is
zero-sized, we would convert the case into a no-payload case.
This was valid when the only invariant that had to be preserved
is that an enum's layout is the same between all substitutions
of a generic type.
However this is now wrong if the payload type is resiliently-sized,
because other resilience domains may not have knowledge that it is
zero-sized.
The new utility methods will also be used in class layout.
If an enum is public and @_fixed_layout, we can only use fixed-size
optimizations if the payload types are fixed-size in all resilience
domains.
However, if an enum is resilient or internal to a module, other
resilience domains cannot have knowledge of its layout, so we can
still use fixed-size optimizations if the payload types are known
to be fixed-size in the current resilience domain only.
If an enum is fixed-layout in our resilience domain but not
universally fixed-layout, other resilience domains will use
runtime functions to project and inject payloads.
These expect to find the payload size in the metadata, so
emit it if the enum is not universally fixed-layout.
Note that we do know the payload size, so it is constant
for us; there's no runtime call required to initialize
the metadata.
If the payload has insufficient extra inhabitants and there's more than one empty case that requires spilling extra tag bits, we need to check the payload part of the value to discriminate them.
Decrease the size of nominal type descriptors and make them true-const by relative-addressing the other metadata they need to reference, which should all be included in the same image as the descriptor itself. Relative-referencing string constants exposes a bug in the Apple linker, which crashes when resolving relative relocations to coalesceable symbols (rdar://problem/22674524); work around this for now by revoking the `unnamed_addr`-ness of string constants that we take relative references to. (I haven't tested whether GNU ld or gold also have this problem on Linux; it may be possible to conditionalize the workaround to only apply to Darwin targets for now.)
We were checking if the tag was <= numPayloadCases, not <. Fix this
and swap the order of the basic blocks so that the no-tag case is
the fall through for consistency with getEnumTag().
This value witness function takes an address of an enum value where the
payload has already been initialized, together with a case index, and
forms the enum value.
The formal behavior can be thought of as satisfying an identity in
relation to the existing two enum value witnesses. For any enum
value, the following is to leave the value unchanged:
tag = getEnumTag(value)
destructiveProjectEnumData(value)
destructiveInjectEnumData(value, tag)
This is the last missing piece for the inject_enum_addr SIL instruction
to handle resilient enums, allowing the implementation of an enum to be
decoupled from its uses. Also, it should be useful for dynamically
constructing enum cases with write reflection, once we get around to
doing such a thing.
The body of the value witness is emitted by a new emitStoreTag() method
on EnumImplStrategy. This is similar to the existing storeTag(), except
the case index is a value instead of a contant.
This is implemented as follows for the different enum strategies:
1) For enums consisting of a single case, this is trivial.
2) For enums where all cases are empty, stores the case index into the
payload area.
3) For enums with a single payload case, emits a call to a runtime
function. Note that for non-generic single payload enums, this could
be open-coded more efficiently, but the function still has the
correct behavior since it supports extra inhabitants and so on.
A follow-up patch will make this more efficient.
4) For multi-payload enums, there are two cases:
a) If one of the payloads is generic or resilient, the enum is
dynamically-sized, and a call to a runtime function is emitted.
b) If the entire enum is fixed-size, the value witness checks if
the case is empty or not.
If the case has a payload, the case index is swizzled into
spare bits of the payload, if any, with remaining bits going
into the extra tag area.
If the case is empty, the case index is swizzled into the
spare bits of the payload, the remaining bits of the payload,
and the extra tag area.
The implementations of emitStoreTag() duplicate existing logic in the
enum strategies, in particular case 4)b) is rather complicated.
Code cleanups are welcome here!
This is a bit of a hodge-podge of related changes that I decided
weren't quite worth teasing apart:
First, rename the weak{Retain,Release} entrypoints to
unowned{Retain,Release} to better reflect their actual use
from generated code.
Second, standardize the names of the rest of the entrypoints around
unowned{operation}.
Third, standardize IRGen's internal naming scheme and API for
reference-counting so that (1) there are generic functions for
emitting operations using a given reference-counting style and
(2) all operations explicitly call out the kind and style of
reference counting.
Finally, implement a number of new entrypoints for unknown unowned
reference-counting. These entrypoints use a completely different
and incompatible scheme for working with ObjC references. The
primary difference is that the new scheme abandons the flawed idea
(which I take responsibility for) that we can simulate an unowned
reference count for ObjC references, and instead moves towards an
address-only scheme when the reference might store an ObjC reference.
(The current implementation is still trivially takable, but that is
not something we should be relying on.) These will be tested in a
follow-up commit. For now, we still rely on the bad assumption of
reference-countability.
Resilient enums are manipulated as opaque values.
Clients are still allowed to assume physical case indices and case
payload types for now -- we might add a level of indirection here,
which would require designing a new case dispatch mechanism.
Resilient enums are never constructed directly, only by calling
case constructor functions. Case constructors already get emitted,
however they're [transparent] -- this will change in a subsequent
patch.
We could save on code size by emitting an InjectEnumTag value
witness function that can construct any case given a physical case
number, rather than emitting constructors for each case, but for
now going through case constructor functions will suffice.
The previous patch added an assert, but it turns out we could still
end up in this code path if the payload was fixed-size *and*
address-only. This is not a very common case, so don't try to
optimize it.
In the indirect initializeWithCopy value witness, we had a
terrible hack where the *source* of the initialization was
modified, stripping and emplacing the tag bits before and
after the copy of the payload, respectively.
Since address-only types should not have spare bits at all,
replace the hack with an assertion.
For example, if a @_fixed_layout struct A contains a resilient struct B
from the same module M, then inside M, A can have a fixed size, but
outside, A has a dynamic size because B is opaque. In this case, A is
not "universally fixed-size". This impacts multi-payload enums, because
if A is placed inside a multi-payload enum E which is lowered inside X,
we would get a fixed layout with spare bits, but lowering E outside of
X would yield a dynamic layout. This is incorrect.
Fix this by plumbing through a new predicate IsAlwaysFixedSize, which
is similar to IsPOD and IsBitwiseTakable, where a compound type inherits
the property if all leaf types exhibit it, and only use spare bits if
the original and substituted types have this property.
We were calling hasTypeParameter() on the interface type of the enum
element. Since enum elements are case constructors now, the interface
type was a GenericFunctionType, and since conceptually these cannot
contain free type variables, this would always return to false.
The right fix here is to pass down the unsubstituted type info and
look at the spare bits of that when doing multi-payload enum layout.
Now that this works, we can remove a FIXME that was added to patch
around this.
John and I discussed this and agreed that we only need two cases here,
not four. In the future this may be merged with ResilienceExpansion,
and become a struct with additional availability information, but
we're definitely sure we don't need four levels here.
Replace isSingle{Unknown,Swift}ReferenceCountedObject() with a single
entry point that also returns the reference counting style. Use this
in GenEnum to emit more specific entry points than native and unknown.
This will give a slight performance boost on Darwin, and enable use of
the blocks runtime on Linux.
Progress on <rdar://problem/23315750>.
Fixes two issues with the original patch:
- If numCaseBits was >= 32, the high bits of the tag were shifted
by an incorrect amount. In this case, the high bits were always
zero anyway, but the invalid shift triggered an assertion in
ARM64 codegen. Fixed to not generate the offending instructions
at all, since just loading the payload value is enough.
- When the payload is very small, the number of no-payload cases
could be greater than 2**payloadBits, requiring multiple payload
tags to represent them all. There was an arithmetic error in
this case.
This re-applies r31328.
Swift SVN r31365
The payload tag discriminates between payload cases, but empty cases
are stored in the common spare bits of the payload types, so the logic
here would report all empty cases as having the first empty case selected.
And when writing tests, I didn't cover enums with multiple empty *and*
non-empty cases. Oops...
Now we emit a little bit more code to assemble the correct case index
from both the payload tag and payload value, then select between the
two values depending on the payload tag.
Fixes <rdar://problem/22192074>.
Swift SVN r31328
We can still produce an undefined address for the empty value, even though we considered them as no-payload cases for layout purposes. Fixes rdar://problem/21515490.
Swift SVN r30490
Full type metadata isn't necessary to calculate the runtime layout of a dependent struct or enum; we only need the non-function data from the value witness table (size, alignment, extra inhabitant count, and POD/BT/etc. flags). This can be generated more efficiently than the type metadata for many types--if we know a specific instantiation is fixed-layout, we can regenerate the layout information, or if we know the type has the same layout as another well-known type, we can get the layout from a common value witness table. This breaks a deadlock in most (but not all) cases where a value type is recursive using classes or fixed-layout indirected structs like UnsafePointer. rdar://problem/19898165
This time, factor out the ObjC-dependent parts of the tests so they only run with ObjC interop.
Swift SVN r30266
Full type metadata isn't necessary to calculate the runtime layout of a dependent struct or enum; we only need the non-function data from the value witness table (size, alignment, extra inhabitant count, and POD/BT/etc. flags). This can be generated more efficiently than the type metadata for many types--if we know a specific instantiation is fixed-layout, we can regenerate the layout information, or if we know the type has the same layout as another well-known type, we can get the layout from a common value witness table. This breaks a deadlock in most (but not all) cases where a value type is recursive using classes or fixed-layout indirected structs like UnsafePointer. rdar://problem/19898165
Swift SVN r30243
In the case where there is a single tag bit and extra inhabitants, we
generate a conditional branch, but we weren't emitting both destination
blocks.
Fixes rdar://problem/21742335.
Swift SVN r30228
These will be used for reflection, and eventually to speed up generic
operations on single payload enums as well.
Progress on <rdar://problem/21739870>.
Swift SVN r30214
The isDependentType() query is woefully misunderstood. Some places
seem to want it to mean "a generic type parameter of dependent member
type", which corresponds to what is effectively a type parameter in
the language, while others want it to mean "contains a type parameter
anywhere in the type". Tease out these two meanings in
isTypeParameter() and hasTypeParameter(), respectively, and sort out
the callers.
Swift SVN r29945
We were crashing grabbing the extra inhabitant mask, which does not
always exist. Instead, test to see if there are extra inhabitants and
only use that mask in the comparison if there are in fact extra
inhabitants in the enum layout.
Fixes rdar://problem/21514065.
Swift SVN r29662
If an enum has only a single tag bit, emit a cond_br instead of a switch, and if it has only one no-payload case, jump directly to the destination for that case instead of wasting time testing the bit pattern and branching to unreachable. Fixes rdar://problem/21093111.
Reapplying now that Adrian fixed the debug info issue this exposed.
Swift SVN r29202
This may be the cause of a crash and burn on the iOS builders:
Assertion failed: (Offset <= PieceOffset && "overlapping or duplicate pieces"), function finalize
Swift SVN r29027
If an enum has only a single tag bit, emit a cond_br instead of a switch, and if it has only one no-payload case, jump directly to the destination for that case instead of wasting time testing the bit pattern and branching to unreachable. Fixes rdar://problem/21093111.
Swift SVN r29023
The Linux assert() is apparently doing a positive test on the value,
whereas the Darwin assert() is negating it.
APInt has an operator!(), but APInt cannot be implicitly converted to a
bool, so directly testing the value caused issues only on the Linux
build.
JoeG says the asserts were testing pointer nullness before his recent
representation changes, and aren't meaningful with the new
representation, so they should just be removed.
Swift SVN r29011
Using LLVM large integers to represent enum payloads has been causing compiler performance and code size problems with large types, and has also exposed a long tail of backend bugs. Replace them with an "EnumPayload" abstraction that manages breaking a large opaque binary value into chunks, along with masking, testing, and extracting typed data from the binary blob. For now, use a word-sized chunking schema always, though the architecture here is set up to eventually allow the use of an arbitrary explosion schema, which would benefit single-payload enums by allowing the payload to follow the explosion schema of the contained value.
This time, adjust the assertion in emitCompare not to perform a check before we've established that the payload is empty, since APInt doesn't have a 0-bit state and the default-constructed form is nondeterminisitic. (We should probably use a more-tailored representation for enum payload bit patterns than APInt or ClusteredBitVector.)
Swift SVN r28985
Using LLVM large integers to represent enum payloads has been causing compiler performance and code size problems with large types, and has also exposed a long tail of backend bugs. Replace them with an "EnumPayload" abstraction that manages breaking a large opaque binary value into chunks, along with masking, testing, and extracting typed data from the binary blob. For now, use a word-sized chunking schema always, though the architecture here is set up to eventually allow the use of an arbitrary explosion schema, which would benefit single-payload enums by allowing the payload to follow the explosion schema of the contained value.
Swift SVN r28982
Don't project every value witness from the metadata every time we need one; this wastes code size in a way LLVM can't really optimize since it doesn't know the metadata is immutable. The code size wins on the standard library are disappointingly small (stdlib only shrinks by 4KB), but this makes generic IR a lot more compact and easier to read.
Swift SVN r28095
