Proof-of-concept for this sort of recovery. In the real world, it's
more likely that this will happen due to differences between Swift 3
and Swift 4, rather than changes in what macros are defined, but the
latter can still happen when debugging.
There's a lot to do here to consider this production-ready. There are
no generics involved and no potential circular references, and the
/rest/ of the compiler isn't prepared for this either. But it's cool
to see it working!
Actually recovering is hidden behind the new
-enable-experimental-deserialization-recovery option; without it the
compiler will continue to eagerly abort.
All of this information is recoverable from the more-general,
more-sane signature conformances, so stop
recording/serializing/deserializing all of this extra stuff.
Back in December DougG added code to delay the formation of generic
environments until all declarations from a particular module had been
deserialized, to avoid circular dependencies caused by too-eager
deserialization of protocol members. This worked great for fully-built
modules, but still had some problems with module merging, the phase of
multi-file compilation where the "partial" swiftmodules that
correspond to each source file in a target are loaded and remitted as
a single swiftmodule. Fix this by picking one of the partial
swiftmodules as the representative one for delayed actions, and wait
until deserialization is complete for /all/ of the serialized ASTs in
the same target to form the generic environments.
rdar://problem/30984417
ExtensionDecls for nested generic types have multiple generic parameter
lists, one for each level of nested generic context.
We only serialized the outermost list, though. This didn't cause any
problems as far as I can see because most of the time we seem to use
the GenericSignature instead, which has the correct generic parameters.
However since we still have usages of getGenericParamsOfContext() on
deserialized DeclContexts, better safe than sorry.
I added a test; the test used to pass on master, but with the new
assertion I added, it would fail without the other changes in this
patch.
A lot of files transitively include Expr.h, because it was
included from SILInstruction.h, SILLocation.h and SILDeclRef.h.
However in reality most of these files don't do anything
with Exprs, especially not anything in IRGen or the SILOptimizer.
Now we're down to 171 files in the frontend which depend on
Expr.h, which is still a lot but much better than before.
ASTContext::getSpecializedConformance() already copies the
substitutions, so remove some AllocateCopy() calls.
Also, add a new overload taking a SubstitutionMap instead.
This allows removing some gatherAllSubstitutions() calls,
which have an allocation inside them.
Finally, remove the now-unused ModuleDecl parameter from
ProtocolConformance::subst() and make it public.
This was a remnant of the old generics implementation, where
all nested types were expanded into an AllArchetypes list.
For quite some time, this method no longer returned *all*
dependent types, only those with generic requirements on
them, and all if its remaining uses were a bit convoluted.
- In the generic specialization code, we used this to mangle
substitutions for generic parameters that are not subject
to a concrete same-type constraint.
A new GenericSignature::getSubstitutableParams()
function handles this use-case instead. It is similar
to getGenericParams(), but only returns generic parameters
which require substitution.
In the future, SubstitutionLists will only store replacement
types for these generic parameters, instead of the list of
types that we used to produce from getAllDependentTypes().
- In specialization mangling and speculative devirtualization,
we relied on SubstitutionLists having the same size and
order as getAllDependentTypes(). It's better to turn the
SubstitutionList into a SubstitutionMap instead, and do lookups
into the map.
- In the SIL parser, we were making a pass over the generic
requirements before looking at getAllDependentTypes();
enumeratePairedRequirements() gives the correct information
upfront.
- In SIL box serialization, we don't serialize the size of the
substitution list, since it's available from the generic
signature. Add a GenericSignature::getSubstitutionListSize()
method, but that will go away soon once SubstitionList
serialization only serializes replacement types for generic
parameters.
- A few remaining uses now call enumeratePairedRequirements()
directly.
The protocol conformance checker verifies that all of the requirements
in the protocol's requirement signature are fulfilled. Save the
conformances from that check into the NormalProtocolConformance,
because this is the record of how that concrete type satisfies the
protocol requirements.
Compute, deserialize, and verify this information, but don't use it
for anything just yet. We'll use this to eliminate the "inherited
protocol map" and possibility some redundant type-witness
information.
Replace an existing flag for cross-references to member types (that
wasn't getting much use) with one consistent with how we lookup
values. This fixes the case where someone actually has a useful type
as a member of a protocol extension, and that type gets referenced in
another module; Dispatch does exactly this.
Because you can currently only define typealiases in protocol
extensions, not new types, there's always a workaround for someone
hitting this issue: just use the underlying type.
https://bugs.swift.org/browse/SR-4076
Previously some decls (TypeAliasDecl and ExtensionDecl) had bits
explicitly marking whether they've been validated, while other decls
just deduced this from hasInterfaceType. The doing the latter doesn't
work when the interface type can be computed before doing full
validation (such as protocols and associatedtypes, which have trivial
interface types), and so an explicit bit is adopted for all decls.
The list of directly inherited protocols of a ProtocolDecl is already
encoded in the requirement signature, as conformance constraints where
the subject is Self. Gather the list from there rather than separately
computing/storing the list of "inherited protocols".
When we deserialize a protocol declaration, we had a bunch of
intervening code before it would deserialize and wire up its generic
parameter (Self), including configuring the generic environment. If
any of that code referenced an AssociatedTypeDecl of that protocol,
the deserialization of the AssociatedTypeDecl could end up computing
an erroneous type.
This might be the source of the nondeterministic deserialization
failure that seems to afflict AssociatedTypeDecls in
rdar://problem/30382791.
Previously looking up an extension would result in all extensions for
types with the same name (nested or not) being deserialized; this
could even bring in base types that had not been deserialized yet. Add
in a string to distinguish an extension's base type; in the top-level
case this is just a module name, but for nested types it's a full
mangled name.
This is a little heavier than I'd like it to be, since it means we
mangle names and then throw them away, and since it means there's a
whole bunch of extra string data in the module just for uniquely
identifying a declaration. But it's correct, and does less work than
before, and fixes a circularity issue with a nested type A.B.A that
apparently used to work.
https://bugs.swift.org/browse/SR-3915
Rather than serializing the complete structure of all archetypes
(which is completely redundant), serialize a reference to their owning
generic environment as well as their interface type. The archetype
itself will be reconsituted by mapping the interface type into that
generic environment.
Another spurious failure we're seeing occasionally in CI: a
deserialization issue involving the interface types of parameter
declarations. If this happens---either when serializing or
deserializing---dump some more debugging information.
This biggest change is:
- LayoutConstraintInfo is now a FoldingSetNode, which allows for proper canonicalization of LayoutConstraints. This is important for the correctness of type comparisons if types contain layout constraints.
No functionality changes from the client's point of view.
There's a speculative path note warning people about accidentally
using swiftmodules whose dependencies might have changed, but this
doesn't make sense when the module with the problem is also the
module being built (which happens during the module-merging phase
of multi-file compilation). In that case, pay an extra branch to
avoid showing the note.
No intended functionality change.
Introduce an algorithm to canonicalize and minimize same-type
constraints. The algorithm itself computes the equivalence classes
that would exist if all explicitly-provided same-type constraints are
ignored, and then forms a minimal, canonical set of explicit same-type
constraints to reform the actual equivalence class known to the type
checker. This should eliminate a number of problems we've seen with
inconsistently-chosen same-type constraints affecting
canonicalization.
Every other declaration kind gets this for free in its interface type,
but properties don't. Just add a bit, it's simple enough.
rdar://problem/30289803
Storing this separately is unnecessary since we already
serialize the enum element's interface type. Also, this
eliminates one of the few remaining cases where we serialize
archetypes during AST serialization.
There's a class of errors in Serialization called "circularity
issues", where declaration A in file A.swift depends on declaration B
in file B.swift, and B also depends on A. In some cases we can manage
to type-check each of these files individually due to the laziness of
'validateDecl', but then fail to merge the "partial modules" generated
from A.swift and B.swift to form a single swiftmodule for the library
(because deserialization is a little less lazy for some things). A
common case of this is when at least one of the declarations is
nested, in which case a lookup to find that declaration needs to load
all the members of the parent type. This gets even worse when the
nested type is defined in an extension.
This commit sidesteps that issue specifically for nested types by
creating a top-level, per-file table of nested types in the "partial
modules". When a type is in the same module, we can then look it up
/without/ importing all other members of the parent type.
The long-term solution is to allow accessing any members of a type
without having to load them all, something we should support not just
for module-merging while building a single target but when reading
from imported modules as well. This should improve both compile time
and memory usage, though I'm not sure to what extent. (Unfortunately,
too many things still depend on the whole members list being loaded.)
Because this is a new code path, I put in a switch to turn it off:
frontend flag -disable-serialization-nested-type-lookup-table
https://bugs.swift.org/browse/SR-3707 (and possibly others)
