This library uses GenericMetadataBuilder with a ReaderWriter that can read data and resolve pointers from MachO files, and emit a JSON representation of a dylib containing the built metadata.
We use LLVM's binary file readers to parse the MachO files and resolve fixups so we can follow pointers. This code is somewhat MachO specific, but could be generalized to other formats that LLVM supports.
rdar://116592577
Create a version of the metadata specialization code which is abstracted so that it can work in different contexts, such as building specialized metadata from dylibs on disk rather than from inside a running process.
The GenericMetadataBuilder class is templatized on a ReaderWriter. The ReaderWriter abstracts out everything that's different between in-process and external construction of this data. Instead of reading and writing pointers directly, the builder calls the ReaderWriter to resolve and write pointers. The ReaderWriter also handles symbol lookups and looking up other Swift types by name.
This is accompanied by a simple implementation of the ReaderWriter which works in-process. The abstracted calls to resolve and write pointers are implemented using standard pointer dereferencing.
A new SWIFT_DEBUG_VALIDATE_EXTERNAL_GENERIC_METADATA_BUILDER environment variable uses the in-process ReaderWriter to validate the builder by running it in parallel with the existing metadata builder code in the runtime. When enabled, the GenericMetadataBuilder is used to build a second copy of metadata built by the runtime, and the two are compared to ensure that they match. When this environment variable is not set, the new builder code is inactive.
The builder is incomplete, and this initial version only works on structs. Any unsupported type produces an error, and skips the validation.
rdar://116592420
When witness tables for enums are instantiated at runtime via
swift::swift_initEnumMetadataMultiPayload
the witnesses
getEnumTagSinglePayload
storeEnumTagSinglePayload
are filled with swift_getMultiPayloadEnumTagSinglePayload (previously
getMultiPayloadEnumTagSinglePayload) and
swift_storeMultiPayloadEnumTagSinglePayload (previously
storeMultiPayloadEnumTagSinglePayload). Concretely, that occurs when
instantiating the value witness table for a generic enum which has more
than one case with a payload, like Result<T>. To enable the compiler to
do the same work, those functions need to be visible to it.
Here, those functions are made visible to the compiler. Doing so
requires changing the way they are declared and adding them to
RuntimeFunctions.def which in turn requires the definition of some
functions to describe the availability of those functions.
This is essentially a long-belated follow-up to Arnold's #12606.
The key observation here is that the enum-tag-single-payload witnesses
are strictly more powerful than the XI witnesses: you can simulate
the XI witnesses by using an extra case count that's <= the XI count.
Of course the result is less efficient than the XI witnesses, but
that's less important than overall code size, and we can work on
fast-paths for that.
The extra inhabitant count is stored in a 32-bit field (always present)
following the ValueWitnessFlags, which now occupy a fixed 32 bits.
This inflates non-XI VWTs on 32-bit targets by a word, but the net effect
on XI VWTs is to shrink them by two words, which is likely to be the
more important change. Also, being able to access the XI count directly
should be a nice win.
We've been running doxygen with the autobrief option for a couple of
years now. This makes the \brief markers into our comments
redundant. Since they are a visual distraction and we don't want to
encourage more \brief markers in new code either, this patch removes
them all.
Patch produced by
for i in $(git grep -l '\\brief'); do perl -pi -e 's/\\brief //g' $i & done
From what I see the only fields are DATA_VALUE_WITNESS which
all have type size_t. I converted them to use the target-dependent
`StoredSize`. While I was around I fixed also isValueInline()
to do the right thing (it was using ValueBuffer instead of
TargetValueBuffer) and all the getters for the data value witnesses.
<rdar://problem/41546568>
* Remove RegisterPreservingCC. It was unused.
* Remove DefaultCC from the runtime. The distinction between C_CC and DefaultCC
was unused and inconsistently applied. Separate C_CC and DefaultCC are
still present in the compiler.
* Remove function pointer indirection from runtime functions except those
that are used by Instruments. The remaining Instruments interface is
expected to change later due to function pointer liability.
* Remove swift_rt_ wrappers. Function pointers are an ABI liability that we
don't want, and there are better ways to get nonlazy binding if we need it.
The fully custom wrappers were only needed for RegisterPreservingCC and
for optimizing the Instruments function pointers.
- Create the value witness table as a separate global object instead
of concatenating it to the metadata pattern.
- Always pass the metadata to the runtime and let the runtime handle
instantiating or modifying the value witness table.
- Pass the right layout algorithm version to the runtime; currently
this is always "Swift 5".
- Create a runtime function to instantiate single-case enums.
Among other things, this makes the copying of the VWT, and any
modifications of it, explicit and in the runtime, which is more
future-proof.
- Add RuntimeTarget template This will allow for converting between
metadata structures for native host and remote target architectures.
- Create InProcess and External templates for stored pointers
Add a few more types to abstract pointer access in the runtime
structures but keep native in-process pointer access the same as that
with a plain old pointer type.
There is now a notion of a "stored pointer", which is just the raw value
of the pointer, and the actual pointer type, which is used for loads.
Decoupling these allows us to fork the behavior when looking at metadata
in an external process, but keep things the same for the in-process
case.
There are two basic "runtime targets" that you can use to work with
metadata:
InProcess: Defines the pointer to be trivially a T* and stored as a
uintptr_t. A Metadata * is exactly as it was before, but defined via
AbstractMetadata<InProcess>.
External: A template that requires a target to specify its pointer size.
ExternalPointer: An opaque pointer in another address space that can't
(and shouldn't) be indirected with operator* or operator->. The memory
reader will fetch the data explicitly.
This makes sure that runtime functions use proper calling conventions, get the required visibility, etc.
We annotate the most popular runtime functions in terms of how often they are invoked from Swift code.
- Almost all variants of retain/release functions are annotated to use the new calling convention.
- Some popular non-reference counting functions like swift_getGenericMetadata or swift_dynamicCast are annotated as well.
The set of runtime functions annotated to use the new calling convention should exactly match the definitions in RuntimeFunctions.def!
...and explicitly mark symbols we export, either for use by executables or for runtime-stdlib interaction. Until the stdlib supports resilience we have to allow programs to link to these SPI symbols.
This reverts commit r30215.
Fixes a bunch of problems on the ASAN bot.
Before:
Swift :: 1_stdlib/ErrorType.swift
Swift :: 1_stdlib/Runtime.swift
Swift :: Constraints/bridging.swift
Swift :: Constraints/diagnostics.swift
Swift :: Constraints/lvalues.swift
Swift :: DebugInfo/variables-repl.swift
Swift :: Interpreter/enum_runtime_alignment.swift
Swift :: Interpreter/nil_error_value.swift
Swift :: Interpreter/return_from_main.swift
Swift :: Misc/misc_diagnostics.swift
Swift :: Prototypes/Result.swift
Swift :: expr/expressions.swift
Swift-Unit :: runtime/SwiftRuntimeTests/MetadataTest.installCommonValueWitnesses_pod_indirect
After:
Swift :: Constraints/bridging.swift
Swift :: Constraints/diagnostics.swift
Swift :: Constraints/lvalues.swift
Swift :: Misc/misc_diagnostics.swift
Swift :: expr/expressions.swift
Swift-Unit :: runtime/SwiftRuntimeTests/MetadataTest.installCommonValueWitnesses_pod_indirect
Swift SVN r30396
Full type metadata isn't necessary to calculate the runtime layout of a dependent struct or enum; we only need the non-function data from the value witness table (size, alignment, extra inhabitant count, and POD/BT/etc. flags). This can be generated more efficiently than the type metadata for many types--if we know a specific instantiation is fixed-layout, we can regenerate the layout information, or if we know the type has the same layout as another well-known type, we can get the layout from a common value witness table. This breaks a deadlock in most (but not all) cases where a value type is recursive using classes or fixed-layout indirected structs like UnsafePointer. rdar://problem/19898165
This time, factor out the ObjC-dependent parts of the tests so they only run with ObjC interop.
Swift SVN r30266
Full type metadata isn't necessary to calculate the runtime layout of a dependent struct or enum; we only need the non-function data from the value witness table (size, alignment, extra inhabitant count, and POD/BT/etc. flags). This can be generated more efficiently than the type metadata for many types--if we know a specific instantiation is fixed-layout, we can regenerate the layout information, or if we know the type has the same layout as another well-known type, we can get the layout from a common value witness table. This breaks a deadlock in most (but not all) cases where a value type is recursive using classes or fixed-layout indirected structs like UnsafePointer. rdar://problem/19898165
Swift SVN r30243
This came up for multi-payload enums without generic parameters, eg
enum MyError {
case BusError
case TrainError(Int)
case DataLoss(String)
}
Fixes <rdar://problem/21739870>.
Swift SVN r30215
These will be used for reflection, and eventually to speed up generic
operations on single payload enums as well.
Progress on <rdar://problem/21739870>.
Swift SVN r30214
This change attempts to introduce the functionality without being too
disruptive. After we branch, I want to consolidate some of the runtime
functions and implement this functionality for multi-payload enums
as well, which requires adding new runtime metadata.
Example:
(swift) enum Color { case Red, Green, Blue(Int) }
(swift) print(Color.Red)
REPL.Color.Red
(swift) print(Color.Blue(5))
REPL.Color.Blue(5)
Implements <rdar://problem/18334936>.
Swift SVN r28430
