Introduce a second level of standard substitutions to the mangling,
all of the form `Sc<character>`, and use it to provide standard
substitutions for most of the _Concurrency types.
This is a precursor to rdar://78269642 and a good mangling-size
optimization in its own right.
Repurpose mangling operator `Y` as an umbrella operator that covers new attributes on function types. Free up operators `J`, `j`, and `k`.
```
async ::= 'Ya' // 'async' annotation on function types
sendable ::= 'Yb' // @Sendable on function types
throws ::= 'K' // 'throws' annotation on function types
differentiable ::= 'Yjf' // @differentiable(_forward) on function type
differentiable ::= 'Yjr' // @differentiable(reverse) on function type
differentiable ::= 'Yjd' // @differentiable on function type
differentiable ::= 'Yjl' // @differentiable(_linear) on function type
```
Resolves rdar://76299796.
`@noDerivative` was not mangled in function types, and was resolved incorrectly when there's an ownership specifier. It is fixed by this patch with the following changes:
* Add `NoDerivative` demangle node represented by a `k` operator.
```
list-type ::= type identifier? 'k'? 'z'? 'h'? 'n'? 'd'? // type with optional label, '@noDerivative', inout convention, shared convention, owned convention, and variadic specifier
```
* Fix `NoDerivative`'s overflown offset in `ParameterTypeFlags` (`7` -> `6`).
* In type decoder and type resolver where attributed type nodes are processed, add support for nested attributed nodes, e.g. `inout @noDerivative T`.
* Add `TypeResolverContext::InoutFunctionInput` so that when we resolve an `inout @noDerivative T` parameter, the `@noDerivative T` checking logic won't get a `TypeResolverContext::None` set by the caller.
Resolves rdar://75916833.
* Move differentiability kinds from target function type metadata to trailing objects so that we don't exhaust all remaining bits of function type metadata.
* Differentiability kind is now stored in a tail-allocated word when function type flags say it's differentiable, located immediately after the normal function type metadata's contents (with proper alignment in between).
* Add new runtime function `swift_getFunctionTypeMetadataDifferentiable` which handles differentiable function types.
* Fix mangling of different differentiability kinds in function types. Mangle it like `ConcurrentFunctionType` so that we can drop special cases for escaping functions.
```
function-signature ::= params-type params-type async? sendable? throws? differentiable? // results and parameters
...
differentiable ::= 'jf' // @differentiable(_forward) on function type
differentiable ::= 'jr' // @differentiable(reverse) on function type
differentiable ::= 'jd' // @differentiable on function type
differentiable ::= 'jl' // @differentiable(_linear) on function type
```
Resolves rdar://75240064.
Add the following new mangling rules.
```
global ::= from-type to-type 'TJO' AUTODIFF-FUNCTION-KIND // autodiff self-reordering reabstraction thunk
global ::= from-type 'TJS' AUTODIFF-FUNCTION-KIND INDEX-SUBSET 'p' INDEX-SUBSET 'r' INDEX-SUBSET 'P' // autodiff linear map subset parameters thunk
global ::= global to-type 'TJS' AUTODIFF-FUNCTION-KIND INDEX-SUBSET 'p' INDEX-SUBSET 'r' INDEX-SUBSET 'P' // autodiff derivative function subset parameters thunk
```
Example:
```console
$s13TangentVector16_Differentiation14DifferentiablePQzAaDQy_SdAFIegnnnr_TJSdSSSpSrSUSP ---> autodiff subset parameters thunk for differential from @escaping @callee_guaranteed (@in_guaranteed A._Differentiation.Differentiable.TangentVector, @in_guaranteed B._Differentiation.Differentiable.TangentVector, @in_guaranteed Swift.Double) -> (@out B._Differentiation.Differentiable.TangentVector) with respect to parameters {0, 1, 2} and results {0} to parameters {0, 2}
$sS2f8mangling3FooV13TangentVectorVIegydd_SfAESfIegydd_TJOp ---> autodiff self-reordering reabstraction thunk for pullback from @escaping @callee_guaranteed (@unowned Swift.Float) -> (@unowned Swift.Float, @unowned mangling.Foo.TangentVector) to @escaping @callee_guaranteed (@unowned Swift.Float) -> (@unowned mangling.Foo.TangentVector, @unowned Swift.Float)
```
Resolves rdar://72666310 / SR-13508.
Also fix a bug in `AutoDiffFunction` mangling where the original may be a global that contains more than 1 node (rdar://74151229 / SR-14106).
Compiler:
- Add `Forward` and `Reverse` to `DifferentiabilityKind`.
- Expand `DifferentiabilityMask` in `ExtInfo` to 3 bits so that it now holds all 4 cases of `DifferentiabilityKind`.
- Parse `@differentiable(reverse)` and `@differentiable(_forward)` declaration attributes and type attributes.
- Emit a warning for `@differentiable` without `reverse`.
- Emit an error for `@differentiable(_forward)`.
- Rename `@differentiable(linear)` to `@differentiable(_linear)`.
- Make `@differentiable(reverse)` type lowering go through today's `@differentiable` code path. We will specialize it to reverse-mode in a follow-up patch.
ABI:
- Add `Forward` and `Reverse` to `FunctionMetadataDifferentiabilityKind`.
- Extend `TargetFunctionTypeFlags` by 1 bit to store the highest bit of differentiability kind (linear). Note that there is a 2-bit gap in `DifferentiabilityMask` which is reserved for `AsyncMask` and `ConcurrentMask`; `AsyncMask` is ABI-stable so we cannot change that.
_Differentiation module:
- Replace all occurrences of `@differentiable` with `@differentiable(reverse)`.
- Delete `_transpose(of:)`.
Resolves rdar://69980056.
- Add `DispatchThunkDerivative` and `MethodDescriptorDerivative` as link entities. The derivative functions of initializers, subscripts, properties, and methods are **all methods**, so we don't need other link entities for this purpose.
- Mangle dispatch thunks and method descriptors. Make `AutoDiffFunction` a context node since it can be nested.
Resolves SR-13866 (rdar://71318828) and SR-13125 (rdar://65240599).
Introduce `@concurrent` attribute on function types, including:
* Parsing as a type attribute
* (De-/re-/)mangling for concurrent function types
* Implicit conversion from @concurrent to non-@concurrent
- (De-)serialization for concurrent function types
- AST printing and dumping support
- `Mangle::ASTMangler::mangleAutoDiffDerivativeFunction()` and `Mangle::ASTMangler::mangleAutoDiffLinearMap()` accept original function declarations and return a mangled name for a derivative function or linear map. This is called during SILGen and TBDGen.
- `Mangle::DifferentiationMangler` handles differentiation function mangling in the differentiation transform. This part is necessary because we need to perform demangling on the original function and remangle it as part of a differentiation function mangling tree in order to get the correct substitutions in the mangled derivative generic signature.
A mangled differentiation function name includes:
- The original function.
- The differentiation function kind.
- The parameter indices for differentiation.
- The result indices for differentiation.
- The derivative generic signature.
Previously, the suffix "AD" was used to mangle AsyncFunctionPointers.
That was incorrect because it was already used in the mangling scheme.
Here, that error is fixed by using 'u' under the thunk or specialization
operator 'T' to mangle AsyncFunctionPointers. Additionally, printing
and demangling support is added.
rdar://problem/72336407
"TB" is used instead of "Tg" in case the specialized function has a resilient argument type and this argument is re-abstracted (from indirect to direct passing).
It can be re-abstracted in case the specialization is compiled in the type's resilience domain (i.e. in it's module).
We need a separate mangling for this to distinguish from specializations - with the same type - but in different resilience domains.
Note that this change does not affect the ABI: it's only used for generated module-internal specializations.
If there are multiple retroactive conformances in the mangling tree, they are put under a TypeList node.
This case was not handled by the re-mangler.
The crash shows up in an assert-build of the compiler, because the re-mangler is used for mangling verification.
rdar://problem/68467435
This is a roll-forward of https://github.com/apple/swift/pull/32950, with explicit c++17 version removed from tests. This is not needed since C++17 is the default anyway.
--
In this PR we teach `ClangImporter` to import typedef statements with template instantiation as its underlying type.
```c++
template<class T>
struct MagicWrapper {
T t;
};
struct MagicNumber {};
typedef MagicWrapper<MagicNumber> WrappedMagicNumber;
```
will be made available in Swift as if `WrappedMagicNumber` is a regular struct.
In C++, multiple distinct typedeffed instantiations resolve to the same canonical type. We implement this by creating a hidden intermediate struct that typedef aliasses.
The struct is named as `__CxxTemplateInst` plus Itanium mangled type of the instantiation. For the example above the name of the hidden struct is `__CxxTemplateInst12MagicWrapperI11MagicNumberE`. Double underscore (denoting a reserved C++ identifier) is used to discourage direct usage. We chose Itanium mangling scheme because it produces valid Swift identifiers and covers all C++ edge cases.
Imported module interface of the example above:
```swift
struct __CxxTemplateInst12MagicWrapperI11MagicNumberE {
var t: MagicNumber
}
struct MagicNumber {}
typealias WrappedMagicNumber = __CxxTemplateInst12MagicWrapperI11MagicNumberE
```
We modified the `SwiftLookupTable` logic to show hidden structs in `swift_ide_test` for convenience.
Co-authored-by: Rosica Dejanovska <rosica@google.com>
Co-authored-by: Dmitri Gribenko <gribozavr@gmail.com>
Co-authored-by: Robert Widmann <devteam.codafi@gmail.com>
* [Mangling] Add a new mangling to represent opaque return type for ObjC runtime name
* [Docs] Add the new 'Qu' mangling to 'Mangling.rst' document
* [Test] Update test invocation arguments
Add `async` to the type system. `async` can be written as part of a
function type or function declaration, following the parameter list, e.g.,
func doSomeWork() async { ... }
`async` functions are distinct from non-`async` functions and there
are no conversions amongst them. At present, `async` functions do not
*do* anything, but this commit fully supports them as a distinct kind
of function throughout:
* Parsing of `async`
* AST representation of `async` in declarations and types
* Syntactic type representation of `async`
* (De-/re-)mangling of function types involving 'async'
* Runtime type representation and reconstruction of function types
involving `async`.
* Dynamic casting restrictions for `async` function types
* (De-)serialization of `async` function types
* Disabling overriding, witness matching, and conversions with
differing `async`
* Fix NULL deref for invalid mangled input
The `Qo` operator expects to consume a type name and a list (terminated with a `y` empty list marker) from the stack. After popping the list, it doesn't check whether the stack is empty, so `$syQo` crashes (it pops down to the `y` then tries to pop again).
This PR just adds the obvious check to guard against this.
Resolves rdar://63128307
* Audit Punycode implementation against RFC3492
Fuzz tests have revealed some weaknesses in the error handling of our Punycode implementation used to mangle Unicode identifiers. A more detailed comparison of the implementation against the algorithm detailed in RFC3492 showed that most of the arithmetic overflow checks were omitted and the ones that were present were handled as success instead of failure.
A typical example:
RFC3492 algorithm:
```
let w = w * (base - t), fail on overflow
```
Original implementation:
```
w = w * (base - t);
```
Corrected implementation:
```
if (w > std::numeric_limits<int>::max() / (base - t))
return false;
w = w * (base - t);
```
Resolves rdar://63392615
The TypeDecoder logic had a bug that caused crashes when it saw a tuple type with a variadic marker. Since variadic tuples aren't supported, this changes the logic to cleanly reject a tuple with a variadic marker.
`DifferentiableFunctionInst` now stores result indices.
`SILAutoDiffIndices` now stores result indices instead of a source index.
`@differentiable` SIL function types may now have multiple differentiability
result indices and `@noDerivative` resutls.
`@differentiable` AST function types do not have `@noDerivative` results (yet),
so this functionality is not exposed to users.
Resolves TF-689 and TF-1256.
Infrastructural support for TF-983: supporting differentiation of `apply`
instructions with multiple active semantic results.
The `Qo` operator expects to consume a type name and a list (terminated with a `y` empty list marker) from the stack. After popping the list, it doesn't check whether the stack is empty, so `$syQo` crashes (it pops down to the `y` then tries to pop again).
This PR just adds the obvious check to guard against this.
Resolves rdar://63128307
Mangle `@noDerivative` parameters to fix type reconstruction errors.
Resolves SR-12650. The new mangling is non-breaking.
When differentiation supports multiple result indices and `@noDerivative`
results are added, we can reuse some of this mangling support.
Add mangling scheme for `@differentiable` and `@differentiable(linear)` function
types. Mangling support is important for debug information, among other things.
Update docs and add tests.
Resolves TF-948.
Teach SILGen to emit a separate SIL function to capture the
initialization of the backing storage type for a wrapped property
based on the wrapped value. This eliminates manual code expansion at
every use site.
When mangling a dependent protocol conformance ref, the mangler currently uses `0_` to mean an unknown index and `N_` to mean the index `N - 1`. Unfortunately, this is somewhat confused: `0_` is actually the mangling for index 1, and index 0 is supposed to be mangled as just `_`, so true indexes are actually offset by 2. So the first thing to do here is to clarify what's going on throughout the mangler, demangler, and ABI documentation.
Also, the demangler attempts to produce a `DependentProtocolConformance*` node with the appropriate child nodes and an optional index payload. Unfortunately, demangle nodes cannot have both children and a value payload, so whenever it creates a node with an index payload, the demangler will assert. It does this whenever the mangled index is not 0; since (per above) the mangler always produces a non-zero mangled index in this production, the demangler will always assert when processing these. So clearly this is well-tested code, since +asserts builds will always trigger the demangler when mangling a name in the first place. To fix this, we need to make the index a child of the mangling node instead of its payload; at the same time, we can make it store the semantically correct index value and just introduce a new `UnknownIndex` node to handle the `0_` case. This is easy because all current clients ignore this information.
Finally, due to an apparent copy-and-paste error, the demangler attempts to produce a `DependentProtocolConformanceRoot` node for associated protocol conformances; this is easily resolved.
This fixes the crash in SR-10926 (rdar://51710424). The obscurity of this crash --- which originally made us think it might be related to Error self-conformance --- is because it is only triggered when a function signature takes advantage of a concrete-but-dependent retroactive conformance, which (to be both concrete and dependent) must furthermore be conditional. Testing the other cases besides a root conformance requires an even more obscure testcase.
The type checker calls these types Builtin.FPIEEE<size>; the demangler
should too.
This is just cosmetic at the moment, but it was causing problems when
I added support for builtin types to the TypeDecoder.
New(er) grammar:
// same module as conforming type, or non-unique
protocol-conformance-ref ::= protocol 'HP'
// same module as protocol
protocol-conformance-ref ::= protocol 'Hp'
// retroactive
protocol-conformance-ref ::= protocol module
We don't make use of this distinction anywhere yet, but we could in
the future.
Due to some unfortunate refactoring, protocol-conformance-ref is a
nonterminal in the mangling grammar that doesn't have its own
operator:
```
protocol-conformance-ref ::= protocol module?
```
Both "module" and "protocol" can be an "identifier", which introduces
a mangling collision. Address the mangling collision by using the
operator "HP".
Fixes rdar://problem/46735592.
Start emitting associated conformance requirement descriptors for
inherited protocols, so we have a symbol to reference from resilient
witness tables and mangled names in the future.
Change the retroactive conformance mangling to use the new
any-protocol-conformance mangling, which maintains more information about
concrete conformances. Specifically, it maintains conformance information
for conditional requirements. It also uses the protocol-conformance-ref
production that will eventually allow symbolic references to protocol
conformance descriptors.
While here, extend the “is retroactive” check during mangling to look for
retroactive conformances in the conditional requirements of a conformance.
The immediate conformance might not be retroactive, but its specialization
might depend on a retroactive conformance. Mangle these as “retroactive”, so
we can correctly reconstruct the exact type.
