This is a first version of cross module optimization (CMO).
The basic idea for CMO is to use the existing library evolution compiler features, but in an automated way. A new SIL module pass "annotates" functions and types with @inlinable and @usableFromInline. This results in functions being serialized into the swiftmodule file and thus available for optimizations in client modules.
The annotation is done with a worklist-algorithm, starting from public functions and continuing with entities which are used from already selected functions. A heuristic performs a preselection on which functions to consider - currently just generic functions are selected.
The serializer then writes annotated functions (including function bodies) into the swiftmodule file of the compiled module. Client modules are able to de-serialize such functions from their imported modules and use them for optimiations, like generic specialization.
The optimization is gated by a new compiler option -cross-module-optimization (also available in the swift driver).
By default this option is off. Without turning the option on, this change is (almost) a NFC.
rdar://problem/22591518
Note: The change in ASTBuilder::createFunctionType is functionally minor,
but we need the FunctionType::Params computed _before_ the ExtInfo, so we
need to shuffle a bunch of code around.
Frontend outputs source-as-compiled, and source-ranges file with function body ranges and ranges that were unparsed in secondaries.
Driver computes diffs for each source file. If diffs are in function bodies, only recompiles that one file. Else if diffs are in what another file did not parse, then the other file need not be rebuilt.
The new option `-sanitize-recover=` takes a list of sanitizers that
recovery instrumentation should be enabled for. Currently we only
support it for Address Sanitizer.
If the option is not specified then the generated instrumentation does
not allow error recovery.
This option mirrors the `-fsanitize-recover=` option of Clang.
We don't enable recoverable instrumentation by default because it may
lead to code size blow up (control flow has to be resumable).
The motivation behind this change is that today, setting
`ASAN_OPTIONS=halt_on_error=0` at runtime doesn't always work. If you
compile without the `-sanitize-recover=address` option (equivalent to
the current behavior of the swift compiler) then the generated
instrumentation doesn't allow for error recovery. What this means is
that if you set `ASAN_OPTIONS=halt_on_error=0` at runtime and if an ASan
issue is caught via instrumentation then the process will always halt
regardless of how `halt_on_error` is set. However, if ASan catches an
issue via one of its interceptors (e.g. memcpy) then `the halt_on_error`
runtime option is respected.
With `-sanitize-recover=address` the generated instrumentation allows
for error recovery which means that the `halt_on_error` runtime option
is also respected when the ASan issue is caught by instrumentation.
ASan's default for `halt_on_error` is true which means this issue only
effects people who choose to not use the default behavior.
rdar://problem/56346688
Educational notes are small pieces of documentation which explain a concept
relevant to some diagnostic message. If -enable-descriptive-diagnostics is
passed, they will be printed after a diagnostic message if available.
Educational notes can be found at /usr/share/doc/diagnostics in a
toolchain, and are associated with specific compiler diagnostics in
EducationalNotes.def.
Diagnose ephemeral conversions that are passed to @_nonEphemeral
parameters. Currently, this defaults to a warning with a frontend flag
to upgrade to an error. Hopefully this will become an error by default
in a future language version.
This option is useful to debug the compiler itself, to simulate debug info as it
would be generated when producing optimized code, but without the unpredictable
output of an optimizing debugger.
This flag will feature-gate work on producing more descriptive diagnostic messages.
It will remain a hidden frontend option until these improvements are ready to ship.
We generate .swiftsourceinfo for stdlib in the build directory because ABI checker
could issue diagnostics to the stdlib source. However, this may also change other
diagnostic tests. Both Brent and Jordan have raised concern about this. After
adding this flag, other diagnostic tests could ignore .swiftsourceinfo files even
though when they are present so our tests will reflect what most users experience
when sources for stdlib are unavailable.
This flag will enable all experimental differentiable programming features.
The default will be `true` on tensorflow branch but `false` on master branch.
Features will first be updated on tensorflow branch to use this flag, before
being upstreamed to master. The goal is to achieve a minimal code diff between
the two branches.
The [TF-820](https://bugs.swift.org/browse/TF-820) master issue tracks upstreaming differentiable programming.
---
Rationale: we chose to add a frontend flag rather than a `build-script`/CMake flag for easier testing. Differentiable programming `lit` tests can be run by specifying this additional flag without recompiling the compiler and standard library.
[Forum discussion on upstreaming differentiable programming.](https://forums.swift.org/t/upstreaming-differentiable-attribute-and-differentiable-protocol/26821)
This flag, currently staged in as `-experimental-skip-non-inlinable-function-bodies`, will cause the typechecker to skip typechecking bodies of functions that will not be serialized in the resulting `.swiftmodule`. This patch also includes a SIL verifier that ensures that we don’t accidentally include a body that we should have skipped.
There is still some work left to make sure the emitted .swiftmodule is exactly the same as what’s emitted without the flag, which is what’s causing the benchmark noise above. I’ll be committing follow-up patches to address those, but for now I’m going to land the implementation behind a flag.
Previously, we would unconditionally set the `installapi` flag, which
hard errors when trying to merge with a TAPI-generated TBD file that
was generated during the install phase.
Fixes rdar://42406088
Add `-no-toolchain-stdlib-rpath` flag: the negative version of
`-toolchain-stdlib-rpath`.
Make `-no-toolchain-stdlib-rpath` be the default: use `/usr/lib/swift` as
default RPATH on Darwin platforms instead of toolchain standard library.
Adapted from https://github.com/apple/swift/pull/27206.
tensorflow branch requires the opposite default (use toolchain standard
library as RPATH) because some stdlib modules like TensorFlow do not exist in
`/usr/lib/swift`.
When we transform each expression or statement in a function builder,
introduce a one-way constraint so that type information does not flow
backwards from the context into that statement or expression. This
more closely mimics the behavior of normal code, where type inference
is per-statement, flowing from top to bottom.
This also allows us to isolate different expressions and statements
within a closure that's passed into a function builder parameter,
reducing the search space and (hopefully) improving compile times for
large function builder closures.
For now, put this functionality behind the compiler flag
`-enable-function-builder-one-way-constraints` for testing purposes;
we still have both optimization and correctness work to do to turn
this on by default.
This flag adds diagnostic names to the end of their messages, e.g. 'error: cannot convert value of type '[Any]' to specified type '[Int]' [cannot_convert_initializer_value]'. It's intended to be used for debugging purposes when working on the compiler.
