A few things:
1. Internally except for in the parser and the clang importer, we only represent
'sending'. This means that it will be easy to remove 'transferring' once enough
time has passed.
2. I included a warning that suggested to the user to change 'transferring' ->
'sending'.
3. I duplicated the parsing diagnostics for 'sending' so both will still get
different sets of diagnostics for parsing issues... but anywhere below parsing,
I have just changed 'transferring' to 'sending' since transferring isn't
represented at those lower levels.
4. Since SendingArgsAndResults is always enabled when TransferringArgsAndResults
is enabled (NOTE not vis-a-versa), we know that we can always parse sending. So
we import "transferring" as "sending". This means that even if one marks a
function with "transferring", the compiler will guard it behind a
SendingArgsAndResults -D flag and in the imported header print out sending.
rdar://128216574
We still only parse transferring... but this sets us up for adding the new
'sending' syntax by first validating that this internal change does not mess up
the current transferring impl since we want both to keep working for now.
rdar://128216574
Add a new demangler option which excludes a closure's type signature.
This will be used in lldb.
Closures are not subject to overloading, and so the signature will never be used to
disambiguate. A demangled closure is uniquely identifiable by its index(s) and parent.
Where opaque types are involved, the concrete type signature can be quite complex. This
demangling option allows callers to avoid printing the underlying complex nested
concrete types.
Example:
before: `closure #1 (Swift.Int) -> () in closure #1 (Swift.Int) -> () in main`
after: `closure #1 in closure #1 in main`
Replace use of `snprintf()` with some custom code (this is around 10x
faster on my machine).
Move a few of the `Node` member functions to the header so they're
inlined.
Optimize the `deepEquals()` function by adding an `isSimilarTo()`
method on `Node`; the checks that were happening in the `deepEquals()`
function could be implemented more efficiently by making use of
details of the internal representation of `Node`.
rdar://125739630
This PR implements first set of changes required to support autodiff for coroutines. It mostly targeted to `_modify` accessors in standard library (and beyond), but overall implementation is quite generic.
There are some specifics of implementation and known limitations:
- Only `@yield_once` coroutines are naturally supported
- VJP is a coroutine itself: it yields the results *and* returns a pullback closure as a normal return. This allows us to capture values produced in resume part of a coroutine (this is required for defers and other cleanups / commits)
- Pullback is a coroutine, we assume that coroutine cannot abort and therefore we execute the original coroutine in reverse from return via yield and then back to the entry
- It seems there is no semantically sane way to support `_read` coroutines (as we will need to "accept" adjoints via yields), therefore only coroutines with inout yields are supported (`_modify` accessors). Pullbacks of such coroutines take adjoint buffer as input argument, yield this buffer (to accumulate adjoint values in the caller) and finally return the adjoints indirectly.
- Coroutines (as opposed to normal functions) are not first-class values: there is no AST type for them, one cannot e.g. store them into tuples, etc. So, everywhere where AST type is required, we have to hack around.
- As there is no AST type for coroutines, there is no way one could register custom derivative for coroutines. So far only compiler-produced derivatives are supported
- There are lots of common things wrt normal function apply's, but still there are subtle but important differences. I tried to organize the code to enable code reuse, still it was not always possible, so some code duplication could be seen
- The order of how pullback closures are produced in VJP is a bit different: for normal apply's VJP produces both value and pullback closure via a single nested VJP apply. This is not so anymore with coroutine VJP's: yielded values are produced at `begin_apply` site and pullback closure is available only from `end_apply`, so we need to track the order in which pullbacks are produced (and arrange consumption of the values accordingly – effectively delay them)
- On the way some complementary changes were required in e.g. mangler / demangler
This patch covers the generation of derivatives up to SIL level, however, it is not enough as codegen of `partial_apply` of a coroutine is completely broken. The fix for this will be submitted separately as it is not directly autodiff-related.
---------
Co-authored-by: Andrew Savonichev <andrew.savonichev@gmail.com>
Co-authored-by: Richard Wei <rxwei@apple.com>
The names of the private witness table accessor thunks we generate for
an opaque return type mangle the concrete conformance of the underlying
type.
If a conformance requirement of the opaque return type was witnessed by
a conditional conformance of a variadic generic type, we would crash
because of an unimplemented case in the mangler.
Fixes rdar://problem/125668798.
The deepHash() function gets called repeatedly as we descend the
node tree, which results in O(n^2) behaviour because we're traversing
entire node subtree from each node we try substitution in, in order
to calculate the hash.
Fix by adding a hash table for hashes, so that we can look up hashes
we've already computed.
This appears to yield a 26.8% saving in local tests.
rdar://125739630
Invertible protocols are currently always mangled with `Ri`, followed by
a single letter for each invertible protocol (e.g., `c` and `e` for
`Copyable` and `Escapable`, respectively), followed by the generic
parameter index. However, this requires that we extend the mangling
for any future invertible protocols, which mean they won't be
backward compatible.
Replace this mangling with one that mangles the bit # for the
invertible protocol, e.g., `Ri_` (followed by the generic parameter
index) is bit 0, which is `Copyable`. `Ri0_` (then generic parameter
index) is bit 1, which is `Escapable`. This allows us to round-trip
through mangled names for any invertible protocol, without any
knowledge of what the invertible protocol is, providing forward
compatibility. The same forward compatibility is present in all
metadata and the runtime, allowing us to add more invertible
protocols in the future without updating any of them, and also
allowing backward compatibility.
Only the demangling to human-readable strings maps the bit numbers
back to their names, and there's a fallback printing with just the bit
number when appropriate.
Also generalize the mangling a bit to allow for mangling of invertible
requirements on associated types, e.g., `S.Sequence: ~Copyable`. This
is currently unsupported by the compiler or runtime, but that may
change, and it was easy enough to finish off the mangling work for it.
It's illegal to call `node->addChild()` with a `NULL` child argument;
it's possible to construct unexpected `Node` trees by passing invalid
manglings, and in this case that was causing `popTypeAndGetChild()` to
fail (because the top node was not a `Type` node), which then meant
that the call to `addChild` had a `NULL` child argument.
The simplest fix is to use `createWithChildren()` to do the node
construction, because that function checks its arguments for `NULL`s.
rdar://125350219
LLVM is presumably moving towards `std::string_view` -
`StringRef::startswith` is deprecated on tip. `SmallString::startswith`
was just renamed there (maybe with some small deprecation inbetween, but
if so, we've missed it).
The `SmallString::startswith` references were moved to
`.str().starts_with()`, rather than adding the `starts_with` on
`stable/20230725` as we only had a few of them. Open to switching that
over if anyone feels strongly though.
This includes runtime support for instantiating transferring param/result in
function types. This is especially important since that is how we instantiate
function types like: typealias Fn = (transferring X) -> ().
rdar://123118061
This library uses GenericMetadataBuilder with a ReaderWriter that can read data and resolve pointers from MachO files, and emit a JSON representation of a dylib containing the built metadata.
We use LLVM's binary file readers to parse the MachO files and resolve fixups so we can follow pointers. This code is somewhat MachO specific, but could be generalized to other formats that LLVM supports.
rdar://116592577
ASL is deprecated in macOS 10.12. It may be time to transition to os_log now
that deployment targets have been raised to 10.12, but until that project
starts these warnings are just pollution.
Filed rdar://121066531 to track adoption of `os_log()` if appropriate.
rdar://119329771
This layout allows adding pre-specializations for trivial types that have a different size, but the same stride. This is especially useful for collections, where the stride is the important factor.
Function body macros allow one to introduce a function body for a
particular function, either providing a body for a function that
doesn't have one, or wholesale replacing the body of a function that
was written with a new one.
Yet more preprocessor metaprogramming to eliminate per-macro-role boilerplate
in the compiler. This time, focused on mangling, demangling, and remangling
of the accessor macro roles.
Using symbolic references instead of a text based mangling avoids the
expensive type descriptor scan when objective c protocols are requested.
rdar://111536582
There are certainly more such issues in this code, but this is
one that was recently reported.
While here, re-enable some disabled test cases that currently pass.
Resolves rdar://104671103
Reformatting everything now that we have `llvm` namespaces. I've
separated this from the main commit to help manage merge-conflicts and
for making it a bit easier to read the mega-patch.
This is phase-1 of switching from llvm::Optional to std::optional in the
next rebranch. llvm::Optional was removed from upstream LLVM, so we need
to migrate off rather soon. On Darwin, std::optional, and llvm::Optional
have the same layout, so we don't need to be as concerned about ABI
beyond the name mangling. `llvm::Optional` is only returned from one
function in
```
getStandardTypeSubst(StringRef TypeName,
bool allowConcurrencyManglings);
```
It's the return value, so it should not impact the mangling of the
function, and the layout is the same as `std::optional`, so it should be
mostly okay. This function doesn't appear to have users, and the ABI was
already broken 2 years ago for concurrency and no one seemed to notice
so this should be "okay".
I'm doing the migration incrementally so that folks working on main can
cherry-pick back to the release/5.9 branch. Once 5.9 is done and locked
away, then we can go through and finish the replacement. Since `None`
and `Optional` show up in contexts where they are not `llvm::None` and
`llvm::Optional`, I'm preparing the work now by going through and
removing the namespace unwrapping and making the `llvm` namespace
explicit. This should make it fairly mechanical to go through and
replace llvm::Optional with std::optional, and llvm::None with
std::nullopt. It's also a change that can be brought onto the
release/5.9 with minimal impact. This should be an NFC change.
The demangler already has an error mechanism to report if demangling
failed. Add null pointer checks before every access in
Demangle::getUnspecialized, and return an error if the child doesn't
exist.
rdar://110141007
