Replace use of `snprintf()` with some custom code (this is around 10x
faster on my machine).
Move a few of the `Node` member functions to the header so they're
inlined.
Optimize the `deepEquals()` function by adding an `isSimilarTo()`
method on `Node`; the checks that were happening in the `deepEquals()`
function could be implemented more efficiently by making use of
details of the internal representation of `Node`.
rdar://125739630
This PR implements first set of changes required to support autodiff for coroutines. It mostly targeted to `_modify` accessors in standard library (and beyond), but overall implementation is quite generic.
There are some specifics of implementation and known limitations:
- Only `@yield_once` coroutines are naturally supported
- VJP is a coroutine itself: it yields the results *and* returns a pullback closure as a normal return. This allows us to capture values produced in resume part of a coroutine (this is required for defers and other cleanups / commits)
- Pullback is a coroutine, we assume that coroutine cannot abort and therefore we execute the original coroutine in reverse from return via yield and then back to the entry
- It seems there is no semantically sane way to support `_read` coroutines (as we will need to "accept" adjoints via yields), therefore only coroutines with inout yields are supported (`_modify` accessors). Pullbacks of such coroutines take adjoint buffer as input argument, yield this buffer (to accumulate adjoint values in the caller) and finally return the adjoints indirectly.
- Coroutines (as opposed to normal functions) are not first-class values: there is no AST type for them, one cannot e.g. store them into tuples, etc. So, everywhere where AST type is required, we have to hack around.
- As there is no AST type for coroutines, there is no way one could register custom derivative for coroutines. So far only compiler-produced derivatives are supported
- There are lots of common things wrt normal function apply's, but still there are subtle but important differences. I tried to organize the code to enable code reuse, still it was not always possible, so some code duplication could be seen
- The order of how pullback closures are produced in VJP is a bit different: for normal apply's VJP produces both value and pullback closure via a single nested VJP apply. This is not so anymore with coroutine VJP's: yielded values are produced at `begin_apply` site and pullback closure is available only from `end_apply`, so we need to track the order in which pullbacks are produced (and arrange consumption of the values accordingly – effectively delay them)
- On the way some complementary changes were required in e.g. mangler / demangler
This patch covers the generation of derivatives up to SIL level, however, it is not enough as codegen of `partial_apply` of a coroutine is completely broken. The fix for this will be submitted separately as it is not directly autodiff-related.
---------
Co-authored-by: Andrew Savonichev <andrew.savonichev@gmail.com>
Co-authored-by: Richard Wei <rxwei@apple.com>
This includes runtime support for instantiating transferring param/result in
function types. This is especially important since that is how we instantiate
function types like: typealias Fn = (transferring X) -> ().
rdar://123118061
This library uses GenericMetadataBuilder with a ReaderWriter that can read data and resolve pointers from MachO files, and emit a JSON representation of a dylib containing the built metadata.
We use LLVM's binary file readers to parse the MachO files and resolve fixups so we can follow pointers. This code is somewhat MachO specific, but could be generalized to other formats that LLVM supports.
rdar://116592577
This executable is intended to be installed in the toolchain and act as
an executable compiler plugin just like other 'macro' plugins.
This plugin server has an optional method 'loadPluginLibrary' that
dynamically loads dylib plugins.
The compiler has a newly added option '-external-plugin-path'. This
option receives a pair of the plugin library search path (just like
'-plugin-path') and the corresponding "plugin server" path, separated
by '#'. i.e.
-external-plugin-path
<plugin library search path>#<plugin server executable path>
For exmaple, when there's a macro decl:
@freestanding(expression)
macro stringify<T>(T) -> (T, String) =
#externalMacro(module: "BasicMacro", type: "StringifyMacro")
The compiler look for 'libBasicMacro.dylib' in '-plugin-path' paths,
if not found, it falls back to '-external-plugin-path' and tries to find
'libBasicMacro.dylib' in them. If it's found, the "plugin server" path
is launched just like an executable plugin, then 'loadPluginLibrary'
method is invoked via IPC, which 'dlopen' the library path in the plugin
server. At the actual macro expansion, the mangled name for
'BasicMacro.StringifyMacro' is used to resolve the macro just like
dylib plugins in the compiler.
This is useful for
* Isolating the plugin process, so the plugin crashes doesn't result
the compiler crash
* Being able to use library plugins linked with other `swift-syntax`
versions
rdar://105104850
Remove the forward declaration for `llvm::raw_ostream` as this creates
ambiguity for `__swift::__runtime::llvm::StringRef` and
`llvm::StringRef` as now `llvm` is made visible to the namespace lookup
rules.
* initial
* it works
demangling mostly works
fix dots
printing works
add tests
add conformance to AnyKeyPath
implement SPI
subscripts fully work
comments
use cross platform image inspection
remove unnecessary comment
fix
fix issues
add conditional conformance
add types
try to fix the api-digester test
cr feedback: move impls behind flag, remove addChain(), switch statement, fallthrough instead of if-elses, move import
cr feedback: refactor switch statement
fix #ifdef
reindent, cr feedback: removes manual memory management
fix missing whitespace
fix typo
fix indentation issues
switch to regexes
checks should test in on all platforms
print types in subscripts
add test for empty subscript
Update test/api-digester/stability-stdlib-abi-without-asserts.test
Co-authored-by: Xiaodi Wu <13952+xwu@users.noreply.github.com>
add commas
fix failing test
fix stdlib annotation
cr feedback: remove global, refactor ifdef
cr feedback: switch back to manual memory management
switch to 5.8 macro
add new weakly linked functions to the allowlist
fix one more failing test
more cr feedback
more cr feedback
* fix invisible unicode
The demangling library can't use the error handling from the main runtime
because it isn't always linked with it. However, it's useful to have
some error handling, and in particular to be able to get data into the
crash logs.
This is complicated because of the way the demangling library gets used,
the upshot of which is that I've had to add a second object library just
for libswiftCore's use, so that the demangler will use the runtime's
error handling functions when present, and fall back on its own when
they aren't.
rdar://89139049
Change `SimplifiedUIDemangleOptions` to remove "partial function" prefixes when demangling async coroutine symbols.
This removes the prefixes "await resume partial function" and "suspend resume partial function" from demangled names, in doing so hides the effect of async/coroutine function splitting from stack traces and other symbolication. This output will produce the source level function name.
For example, a symbol that previously would have demangled to:
```
(1) await resume partial function for static Main.main()
```
will, with this change, demangle to:
```
static Main.main()
```
See https://github.com/apple/swift/pull/36978 where `ShowAsyncResumePartial` was introduced for lldb.
rdar://90455541
* [Distributed] Implement func metadata and executeDistributedTarget
dont expose new entrypoints
able to get all the way to calling _execute
* [Distributed] reimplement distributed get type info impls
* [Distributed] comment out distributed_actor_remoteCall for now
* [Distributed] disable test on linux for now
The internal compiler headers should not include swift shim headers. Removing this dependency allows libSwift to import Swift compiler headers (otherwise, we get name conflics, because we import SwiftShims headers twice: from the source includes and build includes).
Because DEMANGLER_ASSERT() might cause the remanglers to return a ManglingError
with the code ManglingError::AssertionFailed, it's useful to have a line number
in the ManglingError as well as the other information. This is also potentially
helpful for other cases where the code is used multiple times in the remanglers.
rdar://79725187
First pass at adding error handling to the actual Remangler. There are
still some assert() calls at this point that I'm thinking about.
rdar://79725187
Mangling can fail, usually because the Node structure has been built
incorrectly or because something isn't supported with the old remangler.
We shouldn't just terminate the program when that happens, particularly
if it happens because someone has passed bad data to the demangler.
rdar://79725187
Compiler:
- Add `Forward` and `Reverse` to `DifferentiabilityKind`.
- Expand `DifferentiabilityMask` in `ExtInfo` to 3 bits so that it now holds all 4 cases of `DifferentiabilityKind`.
- Parse `@differentiable(reverse)` and `@differentiable(_forward)` declaration attributes and type attributes.
- Emit a warning for `@differentiable` without `reverse`.
- Emit an error for `@differentiable(_forward)`.
- Rename `@differentiable(linear)` to `@differentiable(_linear)`.
- Make `@differentiable(reverse)` type lowering go through today's `@differentiable` code path. We will specialize it to reverse-mode in a follow-up patch.
ABI:
- Add `Forward` and `Reverse` to `FunctionMetadataDifferentiabilityKind`.
- Extend `TargetFunctionTypeFlags` by 1 bit to store the highest bit of differentiability kind (linear). Note that there is a 2-bit gap in `DifferentiabilityMask` which is reserved for `AsyncMask` and `ConcurrentMask`; `AsyncMask` is ABI-stable so we cannot change that.
_Differentiation module:
- Replace all occurrences of `@differentiable` with `@differentiable(reverse)`.
- Delete `_transpose(of:)`.
Resolves rdar://69980056.
- `Mangle::ASTMangler::mangleAutoDiffDerivativeFunction()` and `Mangle::ASTMangler::mangleAutoDiffLinearMap()` accept original function declarations and return a mangled name for a derivative function or linear map. This is called during SILGen and TBDGen.
- `Mangle::DifferentiationMangler` handles differentiation function mangling in the differentiation transform. This part is necessary because we need to perform demangling on the original function and remangle it as part of a differentiation function mangling tree in order to get the correct substitutions in the mangled derivative generic signature.
A mangled differentiation function name includes:
- The original function.
- The differentiation function kind.
- The parameter indices for differentiation.
- The result indices for differentiation.
- The derivative generic signature.
Since libDemangling is included in the Swift standard library,
ODR violations can occur on platforms that allow statically
linking stdlib if Swift code is linked with other compiler
libraries that also transitively pull in libDemangling, and if
the stdlib version and compiler version do not match exactly
(even down to commit drift between releases). This lets the
runtime conditionally segregate its copies of the libDemangling
symbols from those in the compiler using an inline namespace
without affecting usage throughout source.
This part of a series of patches to bring ASTPrinter and Swift Demangler to
feature parity, which is needed by LLDB, which depends on using the strings
produced by either interchangibly.
rdar://problem/64222171
This is analogous to ASTPrinter's FullyQualifiedTypesIfAmbiguous option.
This part of a series of patches to bring ASTPrinter and Swift Demangler to
feature parity, which is needed by LLDB, which depends on using the strings
produced by either interchangibly.
rdar://problem/63700540
This part of a series of patches to bring ASTPrinter and Swift Demangler to
feature parity, which is needed by LLDB, which depends on using the strings
produced by either interchangibly.
rdar://problem/63700540
This part of a series of patches to bring ASTPrinter and Swift Demangler to
feature parity, which is needed by LLDB, which depends on using the strings
produced by either interchangibly.
<rdar://problem/63700540>
The demangler tolerates arbitrary suffixes on mangled names, and parses them as a Suffix node. When looking up a class by an ObjC mangled name, we don't want such demanglings to succeed, because this will result in false positives. It's expected that NSClassFromString(someClassName + "some suffix") will fail, unless something has actually created a class with that suffix.
rdar://problem/60012296
Extract common code from the old and new remangler into a common base class.
This lets the old remangler benefit from the changes I did recently in the new remangler.
LLDB would like to substitute the original Archetype names from the
source code when demangling symbols instead of the confusing generic
'A', 'B', ...
<rdar://problem/48259889>
The demangler can be initialized with a preallocated memory on the stack. Only in case of an overflow, the bump pointer allocator mallocs new memory.
Also, support that a new instance of a demangler can "borrow" the free memory from an existing demangler. This is useful because in the runtime the demangler is invoked recursively. With this feature, all the nested demanglers can share a single stack allocated space.
This is done by disallowing nodes with children to also have index or text payloads.
In some cases those payloads were not needed anyway, because the information can be derived later.
In other cases the fix was to insert an additional child node with the index/text payload.
Also, implement single or double children as "inline" children, which avoids needing a separate node vector for children.
All this reduces the needed size for node trees by over 2x.
We've been running doxygen with the autobrief option for a couple of
years now. This makes the \brief markers into our comments
redundant. Since they are a visual distraction and we don't want to
encourage more \brief markers in new code either, this patch removes
them all.
Patch produced by
for i in $(git grep -l '\\brief'); do perl -pi -e 's/\\brief //g' $i & done
