This is a verification routine that is only invoked in PassManager
destructors. I am going to use this to ensure that the
PassManagerVerifierAnalysis only runs at such points (even when sil-verify-all
is enabled) since it is too expensive to run otherwise.
NOTE: The default implementation of verifyFull in this commit is a no-op. I
wanted to have verify() be the default implementation of verifyFull(), but I do
not have more time to invest in this and it seems to catch /real/ bugs, albeit
bugs unrelated to pass manager notification verification. Instead I am going to
file an SR for someone to look at it since I need to move on from this work back
to semantic SIL. At least we will not have notification failure issues anymore
and thus a large correctness issue in the compiler has been fixed. Forward
progress!
rdar://42301529
This enables us to have state independent of the liveness of the SILFunction's
that we are tracking.
I also changed the verifier to implement only verifyFull instead of verify to
ensure that when we run with sil-verify-all this only runs at the end of pass
manager pipelines.
rdar://42301529
With this change and some other changes that I am committing in parallel, the
stdlib and all of the overlays send all proper pass manager notifications.
rdar://42301529
I also thought I was fixing a performance issue by changing
invalidateFunction not to /insert/ new entries into the map, but
I guess those entries were present in practice. So this is just
a cleanup to make ownership easier.
To do so this commit does a few different things:
1. I changed SILOptFunctionBuilder to notify the pass manager's logging
functionality when new functions are added to the module and to notify analyses
as well. NOTE: This on purpose does not put the new function on the pass manager
worklist since we do not want to by mistake introduce a large amount of
re-optimizations. Such a thing should be explicit.
2. I eliminated SILModuleTransform::notifyAddFunction. This just performed the
operations from 1. Now that SILOptFunctionBuilder performs this operation for
us, it is not needed.
3. I changed SILFunctionTransform::notifyAddFunction to just add the function to
the passmanager worklist. It does not need to notify the pass manager's logging
or analyses that a new function was added to the module since
SILOptFunctionBuilder now performs that operation. Given its reduced
functionality, I changed the name to addFunctionToPassManagerWorklist(...). The
name is a little long/verbose, but this is a feature since one should think
before getting the pass manager to rerun transforms on a function. Also, giving
it a longer name calls out the operation in the code visually, giving this
operation more prominance when reading code. NOTE: I did the rename using
Xcode's refactoring functionality!
rdar://42301529
This works around a potential circular dependence issue where TypeSubstCloner
needs access to SILOptFunctionBuilder but is in libswiftSIL.
rdar://42301529
I am going to add the code in a bit that does the notifications. I tried to pass
down the builder instead of the pass manager. I also tried not to change the
formatting.
rdar://42301529
This patch adds SIL-level debug info support for variables whose
static type is rewritten by an optimizer transformation. When a
function is (generic-)specialized or inlined, the static types of
inlined variables my change as they are remapped into the generic
environment of the inlined call site. With this patch all inlined
SILDebugScopes that point to functions with a generic signature are
recursively rewritten to point to clones of the original function with
new unique mangled names. The new mangled names consist of the old
mangled names plus the new substituions, similar (or exactly,
respectively) to how generic specialization is handled.
On libSwiftCore.dylib (x86_64), this yields a 17% increase in unique
source vars and a ~24% increase in variables with a debug location.
rdar://problem/28859432
rdar://problem/34526036
The invariant is that this analysis should be able to stay in sync with the list
of functions stored in SILModule's function list. If any functions are
added/deleted then analyses can get out of sync with the state of the underlying
SILModule.
Some notes:
1. This is currently disabled by default since there are a bunch of
violations of this in the compiler. I am in the process of fixing violations.
Some examples: the linker and global opt.
2. This is a no-op in non-assert builds.
3. The full verification will only happen when -sil-verify-all is enabled.
Otherwise, we only check that when we delete a function, we had state for the
function.
rdar://42301529
This name makes it clear that the function has not yet been deleted and also
contrasts with the past tense used in the API notifyAddedOrModifiedFunction to
show that said function has already added/modified the function.
The name notifyAddFunction is actively harmful since the pass manager uses this
entrypoint to notify analyses of added *OR* modified functions. It is up to the
caller analysis to distinguish in between these cases.
I am not vouching for the design, just trying to make names match the
current behavior.
I believe that this was just a typo from a long time ago. Calling this parameter
a SILAnalysisTy is actively misleading since as a result it seems to a naive
reading that one should be writing a recursive template:
```
class MyAnalysis : public FunctionAnalysisBase<MyAnalysis> { ... }
```
Instead of passing in the function info of the derived analysis, i.e.:
```
class MyAnalysisFunctionInfo { ... }
class MyAnalysis : public FunctionAnalysisBase<MyAnalysisFunctionInfo> { ... }
```
I also added some documentation to that affect onto FunctionAnalysisBase.
Generally in the SIL/SILOptimizer libraries we have been putting kinds in the
swift namespace, not a nested scope in a type in swift (see ValueKind as an
example of this).
Some parts were using the more modern style that we are using in the optimizer
that involves having ivars and local variables be camelCase instead of
CamelCase.
The current dumping format consists of 1 row of information per function. This
will become unweildy to write patterns for when I add additional state to
FunctionInfo.
Instead, this commit converts the dumping format of the caller analysis into a
multi line yaml format. This yaml format looks as follows:
---
calleeName: closure1
hasCaller: false
minPartialAppliedArgs: 1
partialAppliers:
- partial_apply_one_arg
- partial_apply_two_args1
fullAppliers:
...
This can easily expand over time as we expand the queries that caller analysis
can answer.
As an additional advantage, there are definitely yaml parsers that can handle
multiple yaml documents in sequence in a stream. This means that by running via
sil-opt the caller-analysis-printer pass, one now will get a yaml description of
the caller analysis state, perfect and ready for analysis.
This converts a DenseMap to a SmallMapVector and a SetVector to a
SmallSetVector. Both of these create large malloced data structures by
default. This really makes no sense when there are many functions that don't use
a partial apply or many applies.
Additionally, by changing the DenseMap to a MapVector container, this commit is
eliminating a potential source of non-determinism in the compiler since often
times we are iterating over the DenseMap to produce the results. Today all of
the usages of the DenseMap in this way are safe, but to defensively future proof
this analysis, it makes sense to use a MapVector here.
This patch adds SIL-level debug info support for variables whose
static type is rewritten by an optimizer transformation. When a
function is (generic-)specialized or inlined, the static types of
inlined variables my change as they are remapped into the generic
environment of the inlined call site. With this patch all inlined
SILDebugScopes that point to functions with a generic signature are
recursively rewritten to point to clones of the original function with
new unique mangled names. The new mangled names consist of the old
mangled names plus the new substituions, similar (or exactly,
respectively) to how generic specialization is handled.
On libSwiftCore.dylib (x86_64), this yields a 17% increase in unique
source vars and a ~24% increase in variables with a debug location.
rdar://problem/28859432
rdar://problem/34526036
Fixes <rdar://40555427> [SR-7773]:
SILCombiner::propagateConcreteTypeOfInitExistential fails to full propagate type
substitutions.
Fixes <rdar://problem/40923849>
SILCombiner::propagateConcreteTypeOfInitExistential crashes on protocol
compositions.
This rewrite fixes several fundamental bugs in the SILCombiner optimization that
propagates concrete types. In particular, the pass needs to handle:
- Arguments of callee Self type in non-self position.
- Indirect and direct return values of Self type.
- Types that indirectly depend on Self within callee function signature.
- Protocol composition existentials.
- All of the above need to work for protocol extensions as well as witness methods.
- For protocol extensions, conformance lookup should be based on the existential's conformance list.
Additionally, the optimization should not depend on a SILFunction's DeclContext,
which is not serialized. (In fact, we should prevent SIL passes from using
DeclContext). Furthermore, the code needs to be expressed in a way that one can
reason about correctness and invariants.
The root cause of these bugs is that SIL passes are written based on untested
assumptions of Swift type system. A SIL pass needs to handle all verifiable SIL
input because passes need to be composable. Bail-out logic can be added to
simplify the design; however, _the bail-out logic itself cannot make any
assumptions about the language or type system_ that aren't clearly and
explicitly enforced in the SIL verifier. This is a common mistake and major
source of bugs.
I created as many unit tests as I reasonably could to prevent this code from
regressing. Creating enough unit tests to cover all corner cases that were
broken in the original code would be intractable. But the code has been
simplified such that many corner cases disappear.
This opens up some oportunity for generalizing the optimization and eliminating
special cases. However, I want this PR to be limited to fixing correctness
issues only. In the long term, it would be preferable to replace this
optimization entirely with a much more powerful general type propagation pass.
I am tuning a new argument explosion heuristic to reduce code-size. One part of
the heuristic I am playing with is the part of the algorithm that attempts to
figure out if we could eliminate additonal arguments after performing
owned->guaranteed an additional release when we run FSO a second time. Today we
do this unconditionally. I am trying to do it in a more conservative way where
we only do it if we know that we aren't going to increase the number of
arguments too much.
rdar://41146023
