Implements SE-0055: https://github.com/apple/swift-evolution/blob/master/proposals/0055-optional-unsafe-pointers.md
- Add NULL as an extra inhabitant of Builtin.RawPointer (currently
hardcoded to 0 rather than being target-dependent).
- Import non-object pointers as Optional/IUO when nullable/null_unspecified
(like everything else).
- Change the type checker's *-to-pointer conversions to handle a layer of
optional.
- Use 'AutoreleasingUnsafeMutablePointer<NSError?>?' as the type of error
parameters exported to Objective-C.
- Drop NilLiteralConvertible conformance for all pointer types.
- Update the standard library and then all the tests.
I've decided to leave this commit only updating existing tests; any new
tests will come in the following commits. (That may mean some additional
implementation work to follow.)
The other major piece that's missing here is migration. I'm hoping we get
a lot of that with Swift 1.1's work for optional object references, but
I still need to investigate.
initialization in-place on demand. Initialize parent metadata
references correctly on struct and enum metadata.
Also includes several minor improvements related to relative
pointers that I was using before deciding to simply switch the
parent reference to an absolute reference to get better access
patterns.
Includes a fix since the earlier commit to make enum metadata
writable if they have an unfilled payload size. This didn't show
up on Darwin because "constant" is currently unenforced there in
global data containing relocations.
This patch requires an associated LLDB change which is being
submitted in parallel.
initialization in-place on demand. Initialize parent metadata
references correctly on struct and enum metadata.
Also includes several minor improvements related to relative
pointers that I was using before deciding to simply switch the
parent reference to an absolute reference to get better access
patterns.
This is more amenable to cross-platform remote reflection tests.
Also add a new callback to the memory reader: getSymbolAddress,
which will be used for getting the addresses of nominal type
descriptors for concrete and fixed generic types.
- Read the Parent pointer out of Class/Value metadata and create
typerefs for them.
- Add Parent fields to NominalTypeRef and BoundGenericTypeRef.
- Add TypeRef::getSubstMap(), which creates a new generic argument
map after substitution has taken place on it. This is used to
continue to burrow into nested value types, where generic type
parameters may have a different index.
- Use a DenseMap as that generic argument map.
- Unconditionally key the generic argument map with (Depth, Index)
- Clean up ordering and presentation of Index and Depth. In the rest
of the compiler, Depth comes before Index.
The size of a protocol's metadata was not a multiple of 8 bytes, so
on 64-bit platforms, the runtime would copy default witnesses from
the wrong address, because IRGen metadata does not add alignment padding,
whereas the in-memory structure does.
Fix this by adding a 32-bit padding field at the end of the protocol
descriptor. Technically this is not necessary on 32-bit, but this keeps
things simpler for now.
The test case for this is a library evolution test exercising resilient
default protocol requirements, but it is not quite ready to go in yet.
Adds a rough sketch of what will be a test harness, currently only supported
on OS X:
- Launch a child process: an executable written in Swift
- Receive the child process's Mach port
- Receive reflection section addresses and the address of a heap instance
of interest
- Perform field type lookup on the instance remotely (TODO)
- Add RuntimeTarget template This will allow for converting between
metadata structures for native host and remote target architectures.
- Create InProcess and External templates for stored pointers
Add a few more types to abstract pointer access in the runtime
structures but keep native in-process pointer access the same as that
with a plain old pointer type.
There is now a notion of a "stored pointer", which is just the raw value
of the pointer, and the actual pointer type, which is used for loads.
Decoupling these allows us to fork the behavior when looking at metadata
in an external process, but keep things the same for the in-process
case.
There are two basic "runtime targets" that you can use to work with
metadata:
InProcess: Defines the pointer to be trivially a T* and stored as a
uintptr_t. A Metadata * is exactly as it was before, but defined via
AbstractMetadata<InProcess>.
External: A template that requires a target to specify its pointer size.
ExternalPointer: An opaque pointer in another address space that can't
(and shouldn't) be indirected with operator* or operator->. The memory
reader will fetch the data explicitly.
"minimal" is defined as the set of requirements that would be
passed to a function with the type's generic signature that
takes the thick metadata of the parent type as its only argument.
This makes sure that runtime functions use proper calling conventions, get the required visibility, etc.
We annotate the most popular runtime functions in terms of how often they are invoked from Swift code.
- Almost all variants of retain/release functions are annotated to use the new calling convention.
- Some popular non-reference counting functions like swift_getGenericMetadata or swift_dynamicCast are annotated as well.
The set of runtime functions annotated to use the new calling convention should exactly match the definitions in RuntimeFunctions.def!
...and explicitly mark symbols we export, either for use by executables or for runtime-stdlib interaction. Until the stdlib supports resilience we have to allow programs to link to these SPI symbols.
- Nearly done: TypeRefs and the mangled name decoder.
- Add the swift-reflection-test tool.
The field reflection pipeline is roughly:
- Decode type references
- Substitute generic parameters
- Calculate sizes and offsets
There is currently only one action in the tool, which will test the
*Decode* part of the pipeline: `dump-reflection-section`. This reads
the *swift3_reflect section from an object file and dumps the decoded
type references for all of the stored properties and enum cases in the
file.
- TODO: Write tests with various type arrangements to exercise the
decoder - there are likely some holes in the decoder still since the
AST mangler is quite rich in its kinds.
TODO: The next test mode, `dump-field-types`, will do the following:
1. Launch a swift executable with a canned stopping point
2. Get the address of a heap object instance of interest
3. Dump the fully substituted typerefs of all of the stored properties
or enum case payloads.
That test mode will be more involved since it will attach to another
process and need to read from its address space but will test the
entire out-of-process reflection pipeline in a controlled environment.
We can maybe take this test a step further, with an option or a new
test mode, that prints the entire heap reference graph rooted at that
object of interest, in order to test the ability to detect reference
cycles, for example.
This is the first patch in a series that will allow new protocol
requirements to be added resiliently, with the runtime filling in
default implementations in witness tables.
First, this adds a new flag to the protocol descriptor indicating
that the protocol is resilient. In this case, there are two
additional fields, MinimumWitnessTableSizeInWords and
DefaultWitnessTableSizeInWords, followed by tail-allocated
default witnesses.
The swift_getGenericWitnessTable() entry point now fills in the
default witnesses from the protocol if the given witness table
template is smaller than the expected witness table size.
This also changes the layout of instantiated witness tables to move
the address point to the end of private data. Previously the private
data came after the requirements, but this meant that adding new
requirements would require sliding the private data at runtime and
accessing it indirectly. It is much simpler to access it from
negative offsets instead.
I updated IRGen to emit the new metadata, but currently all protocols
are flagged as not resilient, and default witnesses are not emitted;
this will come in a subsequent patch once some more plumbing is
in place.
To avoid generating GOT entries for references to protocols defined
in the current module, I had to add some hacks to the existing hack
for this. I'll hopefully clean this up in a principled manner later.
- Implement emission of type references for nominal type field
reflection, using a small custom encoder resulting in packed
structs, not strings. This will let us embed 7-bit encoded
32-bit relative offsets directly in the structure (not yet
hooked in).
- Use the AST Mangler for encoding type references
Archetypes and internal references were complicating this before, so we
can take the opportunity to reuse this machinery and avoid unique code
and new ABI.
Next up: Tests for reading the reflection sections and converting the
demangle tree into a tree of type references.
Todo: For concrete types, serialize the types for associated types of
their conformances to bootstrap the typeref substitution process.
rdar://problem/15617914
This comes with a fix for a null pointer dereference in _typeByName()
that would pop with foreign classes that do not have a
NominalTypeDescriptor.
Also, I decided to back out part of the change for now, where the
NominalTypeDescriptor references an accessor function instead of a
pattern, since this broke LLDB, which reaches into the pattern to
get the generic cache.
Soon we will split off the generic cache from the pattern, and at
that time we can change the NominalTypeDescriptor to point at the
cache. But for now, let's avoid needless churn in LLDB by keeping
that part of the setup unchanged.
Change conformance records to reference NominalTypeDescriptors instead of
metadata patterns for resilient or generic types.
For a resilient type, we don't know if the metadata is constant or not,
so we can't directly reference either constant metadata or the metadata
template.
Also, whereas previously NominalTypeDescriptors would point to the
metadata pattern, they now point to the metadata accessor function.
This allows the recently-added logic for instantiating concrete types
by name to continue working.
In turn, swift_initClassMetadata_UniversalStrategy() would reach into
the NominalTypeDescriptor to get the pattern out, so that its bump
allocator could be used to allocate ivar tables. Since the pattern is
no longer available this way, we have to pass it in as a parameter.
In the future, we will split off the read-write metadata cache entry
from the pattern; then swift_initClassMetadata_UniversalStrategy() can
just take a pointer to that, since it doesn't actually need anything
else from the pattern.
Since Clang doesn't guarantee alignment for function pointers, I had
to kill the cute trick that packed the NominalTypeKind into the low
bits of the relative pointer to the pattern; instead the kind is now
stored out of line. We could fix this by packing it with some other
field, or keep it this way in case we add new flags later.
Now that generic metadata is instantiated by calling accessor functions,
this change removes the last remaining place that metadata patterns were
referenced from outside the module they were defined in. Now, the layout
of the metadata pattern and the behavior of swift_getGenericMetadata()
is purely an implementation detail of generic metadata accessors.
This patch allows two previously-XFAIL'd tests to pass.
Instead of directly emitting calls to swift_getGenericMetadata*() and
referencing metadata templates, call a metadata accessor function
corresponding to the UnboundGenericType of the NominalTypeDecl.
The body of this accessor forwards arguments to a runtime metadata
instantiation function, together with the template.
Also, move some code around, so that metadata accesses which are
only done as part of the body of a metadata accessor function are
handled separately in emitTypeMetadataAccessFunction().
Apart from protocol conformances, this means metadata templates are
no longer referenced from outside the module where they were defined.
Recent changes added support for resiliently-sized enums, and
enums resilient to changes in implementation strategy.
This patch adds resilient case numbering, fixing the problem
where adding new payload cases would break existing code by
changing the numbering of no-payload cases.
The problem is that internally, enum cases are numbered with payload
cases coming first, followed by no-payload cases. While each list
is itself in declaration order, with new additions coming at the
end, we need to partition it to give us a fast runtime test for
"is this a payload or no-payload case index."
The resilient numbering strategy used here is that the getEnumTag
and destructiveInjectEnumTag value witness functions now take a
tag index in the range [-ElementsWithPayload..ElementsWithNoPayload-1].
Payload elements are numbered in *reverse* declaration order, so
adding new payload cases yields decreasing tag indices, and adding
new no-payload cases yields increasing tag indices, allowing use
sites to be resilient.
This adds the adjustment between 'fragile' and 'resilient' tag
indices in a somewhat unsatisfying manner, because the calculation
could be pushed down further into EnumImplStrategy, simplifying
both the IRGen code and the generated IR. I'll clean this up later.
In the meantime, clean up some other stuff in GenEnum.cpp, mostly
abstracting code that walks cases.
An individual field record for a nominal type consists of:
- 32-bit general purpose flags,
- 32-bit relative offset to the encoded type reference string, or
32-bit relative offset to the mangled name of the type defined
in another image, and
- 32-bit relative offset to the field name string.
Decrease the size of nominal type descriptors and make them true-const by relative-addressing the other metadata they need to reference, which should all be included in the same image as the descriptor itself. Relative-referencing string constants exposes a bug in the Apple linker, which crashes when resolving relative relocations to coalesceable symbols (rdar://problem/22674524); work around this for now by revoking the `unnamed_addr`-ness of string constants that we take relative references to. (I haven't tested whether GNU ld or gold also have this problem on Linux; it may be possible to conditionalize the workaround to only apply to Darwin targets for now.)
