Deserialization may fail if a decl in a dependency changed type between the
time a swiftmodule was built and when it was imported. This can happen because
of changes to the SDK or use of C preprocessor macros. To help understand these
problems, note the specific types causing the mismatch when it leads to a
deserialization failure.
```
.../LibWithXRef.swiftmodule:1:1: error: reference to top-level
declaration 'foo' broken by a context change; the declaration kind of
'foo' from 'A' changed since building 'LibWithXRef'
1 │ A.foo
│ │ ├─ ...
│ ├─ note: a candidate was filtered out because of a type mismatch;
expected: '() -> ()', found: '(Int) -> Float'
```
Recently, unexpected binary module sharing between compilers that had
inconsistent views of the `PlatformKind` enumeration caused me to spend several
days debugging undefined behavior in the compiler. We should validate that
enumeration values decoded during deserialization are valid, and fail fast when
they are not to make debugging this kind of issue less time consuming. This
change just validates the `PlatformKind` value decoded for an `AvailableAttr`,
but more of deserialization could be updated to do similar validation.
Resolves rdar://123770273
Serialization depended on a longstanding bug in NeedsNewVTableEntryRequest: For a member of a non-class, it always returned `true`, not `false`. It turns out this was because serialization was conflating the concepts of vtable entries and witness table entries, so it needed NeedsNewVTableEntryRequest to return true for members of protocols. Untangle this logic so that NeedsNewVTableEntryRequest can be given the logical behavior.
The Swift compiler expects the context to remain stable between when a
module is built and loaded by a client. Usually the build system would
rebuild a module if a dependency changes, or the compiler would rebuilt
the module from a swiftinterface on a context change. However, such
changes are not always detected and in that case the compiler may crash
on an inconsistency in the context. We often see this when a clang
module is poorly modularized, the headers are modified in the SDK, or
some clang define change its API.
These are project issues that used to make the compiler crash, it
provided a poor experience and doesn't encourage the developer to fix
them by themselves. Instead, let's keep track of modularization issues
encountered during deserialization and report them as proper errors when
they trigger a fatal failure preventing compilation.
Intro a new class ErrorWithUnderlyingReason to refactor duplicated logic from
three classes. It can be used as super class to any error with an underlying
error.
The crash on `SILFunction type mismatch` provides little information and
tends to be difficult to reproduce. Let's print some of the available
information and distinguish the two failure sites.
I'm not confident all required information is written down so we may
need to improve this further in the future. This version still crashes
the compiler, we may want a proper type-check to prevent this failure
with a clean diagnostic for the example used here.
rdar://53821031
Currently, ModuleFileSharedCore::fatal() calls abort(), which may be reasonable
in a swift-frontend invocation, but has dire consequences when the Swift
frontend is embedded into another process, for example, LLDB where the abort()
kills the entire debugging session.
This patch introduces a few alternatives to the ModuleFile::fatal() familiy of
functions that instead push a fatal diagnostic to the ASTContext's
DiagnosticsEngine and return an llvm::Error so the error can be roperly
communicated and the ASTContext can be wound down without killing the parent
process.
The transition is not complete, this patch does not yet handle
fatalIfUnexpected(), for example.
This patch is NFC for the Swift compiler: When DebuggerSupport in off
ModuleFile::diagnoseFatal() will still call abort(), but if it is on, the error
will be passed up, together with a pretty stack trace.
rdar://64511878
There are a bunch of places where we expect to find a record with a particular record ID/kind in the decls_block and have to bail out if we see something we don’t expect. Add an `InvalidRecordKindError` we can use in this situation to produce a useful error message and adopt it in various places.
This change also makes deserialization errors print the path to the invalid file.
Printing the name will help in the case of xrefs for an extension member
and more. This will create errors messages like this in the case of a
module with an extension not being loaded:
```
x. *** DESERIALIZATION FAILURE ***
module 'SomeModule' with full misc version ...
module with extension is not loaded (MyExtensionModule)
Cross-reference to module 'TypesOriginalModule'
... TypeBeingExtended
```
rdar://91316948
Rather than outputting diagnostics and to stderr, output all the extra
information added when deserialization fatally fails to the pretty stack
trace instead. Since the pretty stack trace is added to crash logs, this
should avoid the dance of requesting the compiler output
- Moves the previous "**** DESERIALIZATION FAILURE ..." output to the
last pretty stack trace line
- Removes the module and compiler version notes added to the fatal
diagnostic
- Adds a new effective compiler version line for all frontend failure.
Somewhat duplicates the line from the driver, but adds in the
effective version
- Adds a new line for the full misc version of the module that failed.
May double up with previous "While reading from ..." lines that are
added in various deserialization methods, but better to have it
twice than not at all
It's currently not obvious in crash reports whether compiling with
errors is enabled or not. Since this option can make previously
"impossible" paths now possible, add a message to both the pretty stack
output and fatal deserialization diagnostics to point out that it is
enabled.
Previous output:
```
could not deserialize type for ‘foo(_:)': top-level value not found
Cross-reference to module ‘Bar_Private'
... Baz
```
New output:
```
Could not deserialize type for ‘foo(_:)'
Caused by: top-level value not found
Cross-reference to ‘Baz' in module 'Bar_Private'
```
In #30614, we started consuming XRefNonLoadedModuleErrors while loading
conformances, since a conformance to a type we cannot load usually
indicates we're trying to load a protocol that was declared in an
@_implementationOnly imported module.
We should also consume TypeErrors that we see where the underlying reason
is an XRefNonLoadedModuleError, since they're likely indicators of the
same thing.
The difference with `ModuleFile` is that `ModuleFileSharedCore` provides immutable data and is independent of a particular ASTContext.
It is designed to be able to be shared across multiple `ModuleFile`s of different `ASTContext`s in a thread-safe manner.
Add an alternative to getTopLevelDecls and getDeclChecked to limit which
decls are deserialized by first looking at their attributes. If the
attributes are accepted by a function passed as argument the decl is
fully deserialized, otherwise it is ignored.
The filter is included in the signature of existing functions in the
Serilalization services, but I’ve added new methods for it in FileUnit
and its subclasses to leave existing implementations untouched.
Once accessors are no longer listed as members of their parent context,
a failure to deserialize a VarDecl or SubscriptDecl needs to create a
MissingMemberDecl with the total number of vtable entries expected for
all of the accessors of the storage.
Note that until the accessor change actually lands, we always compute
the expected number of vtable entries as 0.
This silences the instances of the warning from Visual Studio about not all
codepaths returning a value. This makes the output more readable and less
likely to lose useful warnings. NFC.
Most of this patch is just removing special cases for materializeForSet
or other fairly mechanical replacements. Unfortunately, the rest is
still a fairly big change, and not one that can be easily split apart
because of the quite reasonable reliance on metaprogramming throughout
the compiler. And, of course, there are a bunch of test updates that
have to be sync'ed with the actual change to code-generation.
This is SR-7134.
Now that @inlinable is a supported feature, we need to handle cases
where a function is inlinable but it references some type that imports
differently in different Swift versions. To start, handle the case
where a SIL function's type is now invalid and therefore the entire
function can't be imported. This doesn't open up anything interesting
yet, but it's a start.
Part of rdar://problem/40899824
Introduce some metaprogramming of accessors and generally prepare
for storing less-structured accessor lists.
NFC except for a change to the serialization format.
We can encounter these when the compiler modifies an inlinable
function to break apart a struct and the struct uses a private
type for one of its fields. It's questionable whether we /should/
handle this, but meanwhile this /is/ a non-intrusive fix that
preserves the performance of non-resilient libraries.
(That is, it appears this worked in Swift 4.0, though perhaps
not all of the same optimizations kicked in.)
https://bugs.swift.org/browse/SR-6874
We would miscompile in mixed-language-version projects when a Swift class was compiled for one language version, while using Objective-C-imported types that are only available to that version, and then imported into a Swift module with a different language version that wasn't able to see all of the properties because of incompatible imported types. This manifested in a number of ways:
- We assumed we could re-derive the constant field offsets of the class's ivars from the layout, which is wrong if properties are missing, causing accesses to final properties or subclass properties to go to the wrong offsets.
- We assumed we could re-derive the instance size and alignment of a class instance in total, causing code to allocate the wrong amount of memory.
- We neglected to account for the space that stored properties take up in the field offset vector of the class object, causing us to load vtable entries for following subclass methods from the wrong offsets.
Eventually, resilience should reduce our exposure to these kinds of problems. As an incremental step in the right direction, when we look at a class from another module in IRGen, treat it as always variably-sized, so we don't try to hardcode offsets, size, or alignment of its instances. When we import a class, and we're unable to import a stored property, leave behind a new kind of MissingMemberDecl that records the number of field offset vector slots it will take up, so that we lay out subclass objects and compute vtable offsets correctly. Fixes rdar://problem/35330067.
A side effect of this is that the RemoteAST library is no longer able to provide fixed field offsets for class ivars. This doesn't appear to impact the lldb test suite, and they will ultimately need to use more abstract access patterns to get ivar offsets from resilient classes (if they aren't already), so I just removed the RemoteAST test cases that tested for class field offsets for now.
This shows up with swift_wrapper typedefs, which get imported into
Swift as structs. If someone makes an extension of a swift_wrapper
type, but the swift_wrapper is only applied in Swift 4 mode, that
extension will break any Swift 3 clients. Recover by just dropping
the extension entirely.
There's still more complexity around extensions---what if a
requirement can't be deserialized? what if something's depending on
the protocol conformance provided by the extension?---but the missing
base type case should be pretty safe. If you can't see the type at
all, things that depend on its conformances are already in trouble.
rdar://problem/33636733
Special DeclNames represent names that do not have an identifier in the
surface language. This implies serializing the information about whether
a name is special together with its identifier (if it is not special)
in both the module file and the swift lookup table.
Finishes up the "big four" non-type decl kinds. Unfortunately,
indiscriminately dropping members from a class affects the layout
of its vtable. That issue is tracked by rdar://problem/31878396.
Proof-of-concept for the above. This shouldn't be common---renames are
far more likely, and those we can track---but occurs when the
swift_wrapper attribute (the implementation of NS_STRING_ENUM) is
active in Swift 4 but not in Swift 3.
Note that this only checks the canonical interface type of the
declaration, because the non-canonical type may contain references to
the declaration's generic parameters.
The next commits will need access from ModuleFile.cpp.
Also stop pretending we care what kind of error we got back, or that
we might get back random other errors that weren't accounted
for. That's good for crash logs, but not actually interesting here.
(Nearly every place that previously looked for OverrideError will
also accept the upcoming TypeError.)
No effective functionality change.
