This invalidation kind is used when a compute-effects pass changes function effects.
Also, let optimization passes which don't change effects only invalidate the `FunctionBody` and not `Everything`.
This is a dedicated instruction for incrementing a
profiler counter, which lowers to the
`llvm.instrprof.increment` intrinsic. This
replaces the builtin instruction that was
previously used, and ensures that its arguments
are statically known. This ensures that SIL
optimization passes do not invalidate the
instruction, fixing some code coverage cases in
`-O`.
rdar://39146527
This is exactly like copy_addr except that it is not viewed from the verifiers
perspective as an "invalid" copy of a move only value. It is intended to be used
in two contexts:
1. When the move checker emits a diagnostic since it could not eliminate a copy,
we still need to produce valid SIL without copy_addr on move only types since we
will hit canonical SIL eventually even if we don't actually codegen the SIL. The
pass can just convert said copy_addr to explicit_copy_addr and everyone is
happy.
2. To implement the explicit copy function for address only types.
These instructions have the following attributes:
1. copyably_to_moveonlywrapper takes in a 'T' and maps it to a '@moveOnly
T'. This is semantically used when initializing a new moveOnly binding from a
copyable value. It semantically destroys its input @owned value and returns a
brand new independent @owned @moveOnly value. It also is used to convert a
trivial copyable value with type 'Trivial' into an owned non-trivial value of
type '@moveOnly Trivial'. If one thinks of '@moveOnly' as a monad, this is how
one injects a copyable value into the move only space.
2. moveonlywrapper_to_copyable takes in a '@moveOnly T' and produces a new 'T'
value. This is a 'forwarding' instruction where at parse time, we only allow for
one to choose it to be [owned] or [guaranteed].
* moveonlywrapper_to_copyable [owned] is used to signal the end of lifetime of
the '@moveOnly' wrapper. SILGen inserts these when ever a move only value has
its ownership passed to a situation where a copyable value is needed. Since it
is consuming, we know that the no implicit copy checker will ensure that if we
need a copy for it, the program will emit a diagnostic.
* moveonlywrapper_to_copyable [guaranteed] is used to pass a @moveOnly T value
as a copyable guaranteed parameter with type 'T' to a function. In the case of
using no-implicit-copy checking this is always fine since no-implicit-copy is a
local pattern. This would be an error when performing no escape
checking. Importantly, this instruction also is where in the case of an
@moveOnly trivial type, we convert from the non-trivial representation to the
trivial representation.
Some important notes:
1. In a forthcoming commit, I am going to rebase the no implicit copy checker on
top of these instructions. By using '@moveOnly' in the type system, we can
ensure that later in the SIL pipeline, we can have optimizations easily ignore
the code.
2. Be aware of is that due to SILGen only emitting '@moveOnly T' along immediate
accesses to the variable and always converts to a copyable representation when
calling other code, we can simply eliminate from the IR all moveonly-ness from
the IR using a lowering pass (that I am going to upstream). In the evil scheme
we are accomplishing here, we perform lowering of trivial values right after
ownership lowering and before diagnostics to simplify the pipeline.
On another note, I also fixed a few things in SILParsing around getASTType() vs
getRawASTType().
This is an instruction that I am going to use to drive some of the ownership
based dataflow optimizations that I am writing now. The instruction contains a
kind that allows one to know what type of checking is required and allows the
need to add a bunch of independent instructions for independent checkers. Each
checker is responsible for removing all of its own mark instructions. NOTE:
MarkMustCheckInst is only allowed in Raw SIL since once we are in Canonical SIL
we want to ensure that all such checking has already occurred.
Introduce a new instruction `dealloc_stack_ref ` and remove the `stack` flag from `dealloc_ref`.
The `dealloc_ref [stack]` was confusing, because all it does is to mark the deallocation of the stack space for a stack promoted object.
* rename the CrossModuleSerializationSetup pass to simply CrossModuleOptimization
* remove the CMO specific serializer pass. Instead run the CrossModuleSerializationSetup pass directly before the standard serializer pass.
* correctly handle shared functions (e.g. specializations)
* refactoring
This instruction is similar to a copy_addr except that it marks a move of an
address that has to be checked. In order to keep the memory lifetime verifier
happy, the semantics before the checker runs are the mark_unresolved_move_addr is
equivalent to copy_addr [init] (not copy_addr [take][init]).
The use of this instruction is that Mandatory Inlining converts builtin "move"
to a mark_unresolved_move_addr when inlining the function "_move" (the only
place said builtin is invoked).
This is then run through a special checker (that is later in this PR) that
either proves that the mark_unresolved_move_addr can actually be a move in which
case it converts it to copy_addr [take][init] or if it can not be a move, emit
an error and convert the instruction to a copy_addr [init]. After this is done
for all instructions, we loop back through again and emit an error on any
mark_unresolved_move_addr that were not processed earlier allowing for us to
know that we have completeness.
NOTE: The move kills checker for addresses is going to run after Mandatory
Inlining, but before predictable memory opts and friends.
Required for UnsafeRawPointer.withMemoryReboud(to:).
%out_token = rebind_memory %0 : $Builtin.RawPointer to %in_token
%0 must be of $Builtin.RawPointer type
%in_token represents a cached set of bound types from a prior memory state.
%out_token is an opaque $Builtin.Word representing the previously bound
types for this memory region.
This instruction's semantics are identical to ``bind_memory``, except
that the types to which memory will be bound, and the extent of the
memory region is unknown at compile time. Instead, the bound-types are
represented by a token that was produced by a prior memory binding
operation. ``%in_token`` must be the result of bind_memory or
The key thing is that the move checker will not consider the explicit copy value
to be a copy_value that can be rewritten, ensuring that any uses of the result
of the explicit copy_value (consuming or other wise) are not checked.
Similar to the _move operator I recently introduced, this is a transparent
function so we can perform one level of specialization and thus at least be
generic over all concrete types.
This is a new instruction that can be used by SILGen to perform a semantic move
in between two entities that are considered separate variables at the AST
level. I am going to use it to implement an experimental borrow checker.
This PR contains the following:
1. I define move_value, setup parsing, printing, serializing, deserializing,
cloning, and filled in all of the visitors as appropriate.
2. I added createMoveValue and emitMoveValueOperation SILBuilder
APIs. createMoveValue always creates a move and asserts is passed a trivial
type. emitMoveValueOperation in contrast, will short circuit if passed a
trivial value and just return the trivial value.
3. I added IRGen tests to show that we can push this through the entire system.
This is all just scaffolding for the instruction to live in SIL land and as of
this PR doesn't actually do anything.
Through various means, it is possible for a synchronous actor-isolated
function to escape to another concurrency domain and be called from
outside the actor. The problem existed previously, but has become far
easier to trigger now that `@escaping` closures and local functions
can be actor-isolated.
Introduce runtime detection of such data races, where a synchronous
actor-isolated function ends up being called from the wrong executor.
Do this by emitting an executor check in actor-isolated synchronous
functions, where we query the executor in thread-local storage and
ensure that it is what we expect. If it isn't, the runtime complains.
The runtime's complaints can be controlled with the environment
variable `SWIFT_UNEXPECTED_EXECUTOR_LOG_LEVEL`:
0 - disable checking
1 - warn when a data race is detected
2 - error and abort when a data race is detected
At an implementation level, this introduces a new concurrency runtime
entry point `_checkExpectedExecutor` that checks the given executor
(on which the function should always have been called) against the
executor on which is called (which is in thread-local storage). There
is a special carve-out here for `@MainActor` code, where we check
against the OS's notion of "main thread" as well, so that `@MainActor`
code can be called via (e.g.) the Dispatch library's
`DispatchQueue.main.async`.
The new SIL instruction `extract_executor` performs the lowering of an
actor down to its executor, which is implicit in the `hop_to_executor`
instruction. Extend the LowerHopToExecutor pass to perform said
lowering.
While the comment is correct to state that this won't enable any
new optimizations with -Onone, it does enable IRGen's lazy
function emission, which is important for 'reasync' functions,
which we don't want to emit at all even at -Onone.
This fixes debug stdlib builds with the new reasync versions
of the &&, || and ?? operators.
This instructions ensures that all instructions, which need to run on the specified executor actually run on that executor.
For details see the description in SIL.rst.
`get_async_continuation[_addr]` begins a suspend operation by accessing the continuation value that can resume
the task, which can then be used in a callback or event handler before executing `await_async_continuation` to
suspend the task.
Today unchecked_bitwise_cast returns a value with ObjCUnowned ownership. This is
important to do since the instruction can truncate memory meaning we want to
treat it as a new object that must be copied before use.
This means that in OSSA we do not have a purely ossa forwarding unchecked
layout-compatible assuming cast. This role is filled by unchecked_value_cast.
The ``base_addr_for_offset`` instruction creates a base address for offset calculations.
The result can be used by address projections, like ``struct_element_addr``, which themselves return the offset of the projected fields.
IR generation simply creates a null pointer for ``base_addr_for_offset``.
We were not using the primary benefits of an intrusive list, namely the
ability to insert or remove from the middle of the list, so let's switch
to a plain vector. This also avoids linked-list pointer chasing.
* a new [immutable] attribute on ref_element_addr and ref_tail_addr
* new instructions: begin_cow_mutation and end_cow_mutation
These new instructions are intended to be used for the stdlib's COW containers, e.g. Array.
They allow more aggressive optimizations, especially for Array.
Add `linear_function` and `linear_function_extract` instructions.
`linear_function` creates a `@differentiable(linear)` function-typed value from
an original function operand and a transpose function operand (optional).
`linear_function_extract` extracts either the original or transpose function
value from a `@differentiable(linear)` function.
Resolves TF-1142 and TF-1143.
Add `differentiable_function` and `differentiable_function_extract`
instructions.
`differentiable_function` creates a `@differentiable` function-typed
value from an original function operand and derivative function operands
(optional).
`differentiable_function_extract` extracts either the original or
derivative function value from a `@differentiable` function.
The differentiation transform canonicalizes `differentiable_function`
instructions, filling in derivative function operands if missing.
Resolves TF-1139 and TF-1140.
The `differentiability_witness_function` instruction looks up a
differentiability witness function (JVP, VJP, or transpose) for a referenced
function via SIL differentiability witnesses.
Add round-trip parsing/serialization and IRGen tests.
Notes:
- Differentiability witnesses for linear functions require more support.
`differentiability_witness_function [transpose]` instructions do not yet
have IRGen.
- Nothing currently generates `differentiability_witness_function` instructions.
The differentiation transform does, but it hasn't been upstreamed yet.
Resolves TF-1141.
Otherwise it can happen that e.g. specialization runs between CrossModuleSerializationSetup and serialization, resulting that an inlinable function references a shared function (which doesn't have a public linkage).
The solution is to move serialization right after CrossModuleSerializationSetup. But only do that if cross-module-optimization is enabled (it would be a disruptive change to move serialization in general).
NOTE: This is not in the mandatory passes (which run before this). This will
enable me to strip out ownership after we serialize without touching frontend
code. It also makes Onone and O use the same code paths for serialization
instead of one happening in the driver (Onone today) and the other in a SIL pass
(-O, -Osize).
The reason that I updated the sil-func-extractor test is that I found a bug in
how we emit sib files, namely if you try to emit a sib file to stdout, the
llvm-bcanalyzer flags it as malformed. If I output the .sib into a file rather
than trying to use stdout, everything works.
- Clear the 'serialized' flag on witness tables and vtables
after serialization, not just functions. This fixes SIL
verifier failures if post-serialization SIL is printed
out and parsed back in.
- Clear the 'serialized' flag when deserializing functions,
witness tables and vtables in a module that has already
been serialized. This fixes SIL verifier failures if
we deserialize more declarations after serializing SIL.
We were seeing SIL verifier failures on bots that run the
tests with the stdlib built with non-standard flags.
Unfortunately I don't have a reduced test case that would
fail in PR testing without these fixes.
Fixes <rdar://problem/36682929>.
