swift-mirror/docs/proposals/C Pointer Interop.rst

:orphan:

Summary
=======

Pointer arguments are a fact of life in C and Cocoa, and there's no way we're
going to realistically annotate or wrap every API safely. However, there are
plenty of well-behaved APIs that use pointer arguments in well-behaved ways
that map naturally to Swift argument conventions, and we should interact with
those APIs in a natural, Swift-ish way. To do so, I propose adding language
and library facilities that enable the following uses of pointer
arguments:

- Const pointer arguments ``const int *``, including const pointers to ObjC
  classes ``NSFoo * const *``, can be used as "in" array arguments, 
  as ``inout`` scalar arguments, or as ``UnsafePointer`` arguments.
- Non-const pointer arguments to C types, ``int *``, can be used as ``inout``
  array or scalar arguments, or as ``UnsafePointer`` arguments.
- Non-const pointer arguments to ObjC class types, ``NSFoo **``, can be used as
  ``inout`` scalar arguments or passed ``nil``. (They cannot be used as
  array arguments or as ``UnsafePointer`` arguments.)
- ``const void *`` and ``void *`` pointers can be used in the same ways as
  pointers to any C type (but not ObjC types).

This model intentionally doesn't try to provide a mapping to every possible
use case for pointers in C.  It also intentionally avoids addressing special
cases we could potentially provide higher-level support for. Some particular
issues this proposal specifically does not address:

- Pointer return values
- Special handling of ``char*`` and/or ``const char*`` arguments
- Special handling of Core Foundation types
- Special handling of ``NSError**`` arguments
- Precise lifetime of values (beyond the minimal lifetime extension to the
  duration of a call)
- Overriding of ObjC methods that take pointer arguments with subclass methods
  that take non-pointer arguments

Design Considerations
=====================

Const Pointer Arguments
-----------------------

Arrays
~~~~~~

Const pointer arguments are frequently used in both C and Objective-C to take
an array of arguments effectively by value. To support this use case, we should
support passing a Swift ``Array`` value to a const pointer argument. An
example from Core Foundation is ``CGColorCreate``, which takes a
``CGFloat`` array of color space-specific components::

  let rgb = CGColorSpaceCreateCalibratedRGB()
  let cyan = CGColorCreate(rgb, [0, 1, 1])

We are willing to assume that the API is well-behaved and does not mutate the
pointed-to array, so we can safely pass an interior pointer to the array storage
without worrying about mutation.

"In" Arguments
~~~~~~~~~~~~~~

Const pointer arguments are also used in many cases where a value is unmodified,
but its identity is important. Somewhat more rarely, const pointer arguments
are used as pure "in" value arguments with no regard for identity; this is
particularly prevalent on platforms like Win32 where there has historically
been no standard ABI for passing structs by value, but pure "in" pointer
parameters are rare on our platforms.  The potential consequences of
disregarding value identity with C APIs are too high to allow scalar arguments
to implicitly be used as pointer arguments, so although const pointers should
never be used for actual mutation, we propose that only ``inout`` scalar
arguments be accepted for const pointer parameters.  Although our semantics
normally do not guarantee value identity, ``inout`` parameters that refer to
stored variables or stored properties of C-derived types are in practice never
subjected to implicit writebacks except in limited circumstances such as
capture of ``inout`` references in closures that could be diagnosed. Requiring
``inout`` also prevents the use of rvalues or ``let`` bindings that never have
well-defined addresses as pointer arguments. If using an rvalue as a pointer
argument is desired, it can easily be wrapped in an array.

Non-Const Pointer Arguments
---------------------------

C Types
~~~~~~~

Non-const arguments of C type can be used as "out" or "inout" parameters,
either of scalars or of arrays, and so should accept ``inout`` parameters of
array or scalar type. Although a C API may expect a pure "out" parameter and
not require initialization of its arguments, it is safer to assume the argument
is ``inout`` and always require initialization. For array parameters, the
exact point of mutation inside the callee cannot be known, so a copy-on-write
array buffer must be eagerly uniqued prior to the address of the array being
taken.

ObjC Types
~~~~~~~~~~

ARC semantics treat an ``NSFoo**`` type as a pointer to an ``__autoreleasing``
``NSFoo*``, so the use of such an argument is limited to it being a scalar
``inout`` object reference or ``nil``.

Pointer Return Values
---------------------

This proposal does not address the handling of return values, which should still
be imported into Swift as ``UnsafePointer`` values.


Library Features
================

The necessary conversions can be represented entirely in the standard library
with the help of some new language features--inout address conversion, inout
writeback conversion, and interior pointer conversion--which are described
below. There are five categories of argument behavior needed, and thus five
new standard library types.  These types should have no user-accessible
operations of their own other than their implicit conversions. The necessary
types are as follows:

- ``CConstPointer<T>`` is the imported representation of a ``const T *``
  argument. It is implicitly convertible from ``inout T`` by inout address
  conversion and from ``Array<T>`` by immutable interior pointer
  conversion. It is also implicitly convertible to and from ``UnsafePointer<T>``
  by normal conversion.
- ``CMutablePointer<T>`` is the imported representation of a ``T *``
  argument for a POD C type ``T``. It is implicitly convertible from
  ``inout T`` by inout address conversion and from ``inout Array<T>`` by mutating
  interior pointer conversion. It is also implicitly convertible to and from
  ``UnsafePointer<T>`` by normal conversion.
- ``CConstVoidPointer`` and ``CMutableVoidPointer`` are the imported
  representations of ``const void *`` and ``void *`` respectively.
  They are implicitly convertible by normal conversion from, respectively,
  any ``CConstPointer<T>`` or any ``CMutablePointer<T>``, and thus are
  transitively convertible from the corresponding ``inout`` and array types.
- ``ObjCInOut<T>`` is the imported representation of a ``T **``
  argument for an ObjC class type ``T``. It is implicitly convertible from
  ``inout T`` by inout writeback conversion and is implicitly convertible
  from ``nil``. It cannot be converted from an array or to ``UnsafePointer``.

New Language Features
=====================

To support the necessary semantics for argument passing, some new conversion
forms need to be supported by the language with special-cased lifetime behavior.

Interior Pointer Conversions
----------------------------

To be able to pass a pointer to array data as an argument, we need to be able
to guarantee the lifetime of the array buffer for the duration of the call.
If mutation can potentially occur through the pointer, then copy-on-write
buffers must also be uniqued prior to taking the address. A new form of
conversion, ``@unsafe_interior_pointer_conversion``, can be applied to an
instance method of a type, to allow that type to return both a converted
pointer and an owning reference that guarantees the validity of the pointer.
Such methods can be either ``mutating`` or non-mutating; only non-mutating
conversions are considered for non- ``inout`` parameters, and only ``mutating``
conversions are considered for ``inout`` parameters::

  extension Array {
    @unsafe_interior_pointer_conversion
    func convertToConstPointer()
    -> (CConstPointer<T>, ArrayBuffer<T>) {
      return (CConstPointer(self.base), self.owner)
    }

    @unsafe_interior_pointer_conversion
    mutating func convertToMutablePointer()
    -> (CMutablePointer<T>, ArrayBuffer<T>) {
      // Make the backing buffer unique before handing out a mutable pointer.
      self.makeUnique()
      return (CMutablePointer(self.base), self.owner)
    }
  }

``@unsafe_interior_pointer_conversion`` conversions are only considered in
argument contexts. If such a conversion is found, the first element of the
return tuple is used as the argument, and a strong reference to the second
element is held for the duration of the callee that receives the converted
argument.

Inout Address Conversion
------------------------

To pass an ``inout`` as a pointer argument, we need to be able to lock an
address for the ``inout`` for the duration of the call, which is not normally
possible. This functionality only needs to be available to the standard library,
so can be expressed in terms of builtins. A type can conform to the
``BuiltinInOutAddressConvertible`` protocol to be convertible from an
inout reference. The protocol is defined as follows::

  protocol BuiltinInOutAddressConvertible {
    /// The type from which inout conversions are allowed to the conforming
    /// type.
    typealias InOutType

    /// Create a value of the conforming type using the address of an inout
    /// argument.
    class func _convertFromInOutAddress(p: Builtin.RawPointer) -> Self
  }

An example of a conformance for ``CMutablePointer``::

  struct CMutablePointer<T>: BuiltinInOutAddressConvertible {
    let ptr: Builtin.RawPointer

    typealias InOutType = T

    @transparent
    static func _convertFromInOutAddress(p: Builtin.RawPointer)
    -> CMutablePointer {
      return CMutablePointer(p)
    }
  }

  func foo(p: CMutablePointer<Int>) { }

  var i = 0
  foo(&i)

The lifetime of the variable, stored property owning object, or writeback
buffer backing the inout is guaranteed for the lifetime of the callee that
receives the converted parameter, as if the callee had received the inout
parameter directly.

Inout Writeback Conversion
--------------------------

Inout address conversion alone is not enough for ``ObjCInOut`` to work as
intended, because the change to the ``__autoreleasing`` convention for the
pointed-to object reference requires a writeback temporary. The
``BuiltinInOutWritebackConvertible`` protocol allows for an additional
writeback to be introduced before and after the address of the ``inout`` is
taken::

  protocol BuiltinInOutWritebackConvertible {
    /// The original type from which inout conversions are allowed to the
    /// conforming type.
    typealias InOutType

    /// The type of the temporary writeback whose address is used to construct
    /// the converted value.
    typealias WritebackType

    /// Get the initial value the writeback temporary should have on entry to
    /// the call.
    class func _createWriteback(inout InOutType) -> WritebackType

    /// Create a value of the conforming type using the address of the writeback
    /// temporary.
    class func _convertFromWritebackAddress(p: Builtin.RawPointer) -> Self

    /// Write the writeback temporary back to the original value.
    class func _commitWriteback(inout InOutType, WritebackType)
  }

An example of a conformance for ``ObjCInOut``::

  struct ObjCInOut<T: class>: BuiltinInOutWritebackConvertible {
    let ptr: Builtin.RawPointer

    typealias InOutType = @unchecked T?
    typealias WritebackType = Builtin.RawPointer

    @transparent
    static func _createWriteback(inout ref: @unchecked T?)
    -> Builtin.RawPointer {
      // The initial object reference is passed into the callee effectively
      // __unsafe_unretained, so pass it as a RawPointer.
      return reinterpretCast(ref)
    }

    @transparent
    static func _commitWriteback(inout ref: @unchecked T?,
                                 value: Builtin.RawPointer) {
      // The reference is autoreleased on return from the caller, so retain it
      // by loading it back as a T?.
      ref = reinterpretCast(value)
    }

    @transparent
    static func _convertFromWritebackAddress(value: Builtin.RawPointer) {
      return ObjCInOut(value)
    }
  }

The lifetime of the writeback is guaranteed for the lifetime of the callee that
receives the converted parameter, as if the callee had received the writeback
temporary as a mutable logical property of the original inout parameter.