Harden more of the serialization functions to propagate errors for
the caller to handle these errors gracefully. This fixes a crash in
finishNormalConformance when indexing a system module with an
implementation-only import.
rdar://problem/52837313
This eliminates the entire 'lazy generic environment' concept;
essentially, all generic environments are now lazy, and since
each signature has exactly one environment, their construction
no longer needs to be co-ordinated with deserialization.
Now that GenericSignatures store their single unique GenericEnvironment,
we can remove similar logic from deserialization to preserve identity
of GenericEnvironments.
- Parameterize maybeReadGenericParams' BitstreamCursor so that we can read from
the correct cursor when trying to read the generic params of a SILFunction.
- Only serialize the context generic params for SILFunctions for which we're
serializing a complete definition. This fixes issues with us getting the
wrong archetypes forward-declared from references in other modules.
In this version of the patch, we adjust the deserialization point for the
generic param list to correctly come before we check if the SILFunction block
is empty, and we add a kludge to keep the JIT from crapping itself when it sees
the same transparent definition in multiple REPL lines' modules
<rdar://problem/16094902>.
The previous commit solves a problem this exposed at r14050 in inout deshadowing
that caused memory corruption when transparent functions were imported. This
should now be safe to commit.
Swift SVN r14109
- Parameterize maybeReadGenericParams' BitstreamCursor so that we can read from the correct cursor when trying to read the generic params of a SILFunction.
- Only serialize the context generic params for SILFunctions for which we're serializing a complete definition. This fixes issues with us getting the wrong archetypes forward-declared from references in other modules.
In this version of the patch, we adjust the deserialization point for the generic param list to correctly come before we check if the SILFunction block is empty, and we add a kludge to keep the JIT from crapping itself when it sees the same transparent definition in multiple REPL lines' modules <rdar://problem/16094902>.
Swift SVN r14030
- Parameterize maybeReadGenericParams' BitstreamCursor so that we can read from the correct cursor when trying to read the generic params of a SILFunction.
- Only serialize the context generic params for SILFunctions for which we're serializing a complete definition. This fixes issues with us getting the wrong archetypes forward-declared from references in other modules.
This gets me a clean build when applied against r13984.
Swift SVN r14005
Edge SILFunction one step closer to independence from SILFunctionType context by taking the generic param list as a separate constructor parameter, and serializing those params alongside the function record. For now we still pass in the context params from the SILFunctionType in most cases, because the logic for finding the generic params tends to be entangled in type lowering, but this pushes the problem up a step.
Thanks Jordan for helping work out the serialization changes needed.
Compared to r13036, this version of the patch includes the decls_block RecordKind enumerators for the GENERIC_PARAM_LIST layouts in the sil_block RecordKind enumerator, as Jordan had suggested before. r13036 caused buildbot failures when building for iOS, but I am unable to reproduce those failures locally now.
Swift SVN r13485
Edge SILFunction one step closer to independence from SILFunctionType context by taking the generic param list as a separate constructor parameter, and serializing those params alongside the function record. For now we still pass in the context params from the SILFunctionType in most cases, because the logic for finding the generic params tends to be entangled in type lowering, but this pushes the problem up a step.
Thanks Jordan for helping work out the serialization changes needed.
Swift SVN r13036
In general, this forces SILGen and IRGen code that's grabbing
a declaration to state whether it's doing so to define it.
Change SIL serialization to serialize the linkage of functions
and global variables, which means also serializing declarations.
Change the deserializer to use this stored linkage, even when
only deserializing a declaration, and to call a callback to
inform the client that it has deserialized a new entity.
Take advantage of that callback in the linking pass to alter
the deserialized linkage as appropriate for the fact that we
imported the declaration. This computation should really take
advantage of the relationship between modules, but currently
it does not.
Swift SVN r12090
Previously, cross-references just carried a chain of identifiers and a
top-level module, plus a type to validate against, a generic parameter index,
or an operator fixity. However, referencing "the first generic parameter
of the prefix function ++ that takes a ForwardIndex" requires /all three/
of these filters to unambiguously select the right declaration.
Now, cross-references consist of a chain of trailing records, one for each
link in the path. There are (currently) five kinds of links:
Type: a declaration that cannot have overloads
Value: a declaration that can have overloads (filtered by type)
Extension: filter to decls within extensions on another module
Operator:
- as the first path piece, an operator declaration
- as a later path piece, a fixity filter for operator functions
Generic Param: an indexed generic parameter of the previous result
This should allow us to uniquely cross-reference any Swift declaration we
need to.
Swift SVN r11399
NominalTypeDecls and ExtensionDecls have a list of members. Rather than
eagerly populating that list when the nominal or extension is just referenced,
just include a pointer back to the ModuleFile, so that they can be
deserialized when we actually look into the decl. The design here is
general enough that we could do something similar with imported Clang decls.
Clang is even more lazy here: the on-disk representation is a hash table,
so lookup only forces deserialization of members with the same name. We
probably want that some day, but this might be enough to get by for now.
This is groundwork for loading partial ASTs, where eagerly deserializing
members leads to circular references we're not able to handle.
Swift SVN r11219
Since we don't have soft-failure yet from deserialization, it's helpful to
at least know where to start looking when something crashes. There are some
rough edges here but it should be much better than nothing.
This also pulls the list of record nodes out into a separate file, so that
we can avoid repeating it.
Example crash:
1. While reading from ./CTypes.swiftmodule
2. While deserializing 'CBool' (StructDecl)
3. While deserializing decl #26 (XREF)
4. Cross-reference to 'LogicValue' in swift
(don't worry, this is an example where I'm tweaking things)
<rdar://problem/14838332>
Swift SVN r11057
Each loaded file gets added to the main module, rather than being a standalone
separate module. In theory, this will be used to assemble several partial
ASTs into a complete module. In practice, there's still a ways to go...but
this can already round-trip a single module file.
This also factors out the FileUnit-creating part of SerializedModuleLoader,
which should help clients like SourceKit that don't need to search for a
swiftmodule file associated with a particular import.
Swift SVN r10952
Allow archetype_method to look up a witness from a concrete ProtocolConformance record. This will allow generic specialization to apply to constrained generic functions independent of archetype_method devirtualization. <rdar://problem/14748543>
Swift SVN r10950
This completes the FileUnit refactoring. A module consists of multiple
FileUnits, which provide decls from various file-like sources. I say
"file-like" because the Builtin module is implemented with a single
BuiltinUnit, and imported Clang modules are just a single FileUnit source
within a module.
Most modules, therefore, contain a single file unit; only the main module
will contain multiple source files (and eventually partial AST files).
The term "translation unit" has been scrubbed from the project. To refer
to the context of declarations outside of any other declarations, use
"top-level" or "module scope". To refer to a .swift file or its DeclContext,
use "source file". To refer to a single unit of compilation, use "module",
since the model is that an entire module will be compiled with a single
driver call. (It will still be possible to compile a single source file
through the direct-to-frontend interface, but only in the context of the
whole module.)
Swift SVN r10837
Part of the FileUnit restructuring. A Clang module (whether from a framework
or a simple collection of headers) is now imported as a TranslationUnit
containing a single ClangModuleUnit.
One wrinkle in all this is that Swift very much wants to do searches on a
per-module basis, but Clang can only do lookups across the entire
TranslationUnit. Unless and until we get a better way to deal with this,
we're stuck with an inefficiency here. Previously, we used to hack around
this by ignoring the "per-module" bit and only performing one lookup into
all Clang modules, but that's not actually correct with respect to visibility.
Now, we're just taking the filtering hit for looking up a particular name,
and caching the results when we look up everything (for code completion).
This isn't ideal, but it doesn't seem to be costing too much in performance,
at least not right now, and it means we can get visibility correct.
In the future, it might make sense to include a ClangModuleUnit alongside a
SerializedASTFile for adapter modules, rather than having two separate
modules with the same name. I haven't really thought through this yet, though.
Swift SVN r10834
Part of the FileUnit restructuring. A serialized module is now represented as
a TranslationUnit containing a single SerializedASTFile.
As part of this change, the FileUnit interface has been made virtual, rather
than switching on the Kind in every accessor. We think the operations
performed on files are sufficiently high-level that this shouldn't affect us.
A nice side effect of all this is that we now properly model the visibility
of modules imported into source files. Previously, we would always consider
the top-level imports of all files within a target, whether re-exported or
not.
We may still end up wanting to distinguish properties of a complete Swift
module file from a partial AST file, but we can do that within
SerializedModuleLoader.
Swift SVN r10832
ModuleID is compatible with IdentifierID, but uses 0 to mean “the builtin module”
and 1 to mean “the current module”. Anything else is a top-level module name,
as an identifier. As an implementation detail, 1 is now never a valid IdentifierID.
(0 remains “the empty string”.)
Using this, simplify the encoding of the owner of a conformance.
Swift SVN r9944
Each one has a different kind of lookup cache anyway, and there's no real
reason to have them share storage at the cost of type-safety.
Swift SVN r9242
docs/Resilience.rst describes the notion of a resilience component:
if the current source file is in the same component as a module being
used, it can use fragile access for everything in the other module,
with the assumption that everything in a component will always be
recompiled together.
However, nothing is actually using this today, and the interface we
have is probably not what we'll want in 2.0, when we actually implement
resilience.
Swift SVN r9174
Add a SILLinkage mode "Deserialized" to make sure IRGen will emit
hidden symbols for deserialized SILFunction.
Inside SIL linker, set Linkage to external if we only have a declaration for
a callee function.
Both sil block and decl block in a module can emit an array of substitutions.
To share the serialization between SILSerializer and Serializer, we modify
the interface to pass in the abbreviation codes to write functions and to
pass in a cursor to read functions.
We now correctly handle the serialization of Substitutions in SpecializeInst.
For a deserialized SILFunction, we now temporarily set its SILLocation and
DebugScope to an empty FileLocation. Once mandatory inliner sets the SILLocation
to the location of ApplyInst, a null SILLocation and a null DebugScope
may work for a deserialized SILFunction.
Update testing cases to reflect that we are now inlining transparent functions
from modules, or to disable SILDeserializer for now (I am not sure how to update
those testing cases).
Swift SVN r8582
Add serialization/deserialization of the following SILInstructions:
BuiltinFunctionRefInst, IndexRawPointerInst, ModuleInst,
Conversion instructions:
RefToObjectPointerInst, UpcastInst, CoerceInst, AddressToPointerInst,
PointerToAddressInst, ObjectPointerToRefInst, RefToRawPointerInst,
RawPointerToRefInst, RefToUnownedInst, UnownedToRefInst
DestroyAddrInst, LoadInst, StrongReleaseInst, StrongRetainInst,
TupleElementAddrInst, TupleExtractInst
Make getModule in ModuleFile public to be used by SILDeserializer, also
make addModuleRef in Serializer public to be used by SILSerializer.
Update testing case to cover the above SILInstructions.
Swift SVN r8372
Use a worklist in SIL linking to traverse the newly serialized SILFunction.
Add serialization/deserialization of the following SILInstructions:
AllocArray, Apply, FunctionRef, IntegerLiteral, Metatype, StructExtract,
Struct and Tuple.
Make getDecl and getIdentifier in ModuleFile public to be used by
SILDeserializer, also make addDeclRef and addIdentifierRef in Serializer
public to be used by SILSerializer.
Update testing case to cover the above SILInstructions.
TODO: lookupSILFunction should replace the existing empty SILFunction instead
of creating a new SILFunction.
Swift SVN r8339
SerializedSILLoader to hold a list of SIL deserializers.
Also add an intial implementation of a linking pass that is run right after
SILGen to link the declaration of SILFunction to the actual definition in
the serialized module.
We add two blocks to the serialized module: a sil index block that
maps identifier to a function ID and also holds a list of function offsets,
and a sil block for the actual SILFunctions. We can probably use subblock
instead of two top-level blocks.
The serialization/de-serialization of the function hash table and the function
offsets are implemented. But we are missing handling of types (see FIXME in
the code).
ModuleFile::Serialized is made public to be used by SIL deserializer, as well
as ModuleFile::getType.
The SIL deserializer holds a pointer to the ModuleFile, it gets the sil cursor
and the sil index cursor from the ModuleFile. The other option is for SIL
deserializer to find the start of the two sil blocks within itself, thus having
less coupling with ModuleFile.
No testing case yet because we are missing handling of types.
Swift SVN r8206
getDisplayDecls() was introduced for ":print_module" and works slightly differently, e.g.
it will return the decls from a shadowed clang module, since we want to display them.
Swift SVN r7909
As a bring-up hack, the module serializer would write a special record,
FALL_BACK_TO_TRANSLATION_UNIT, if it encountered something it didn't know
how to serialize. This then directed the deserializer to ignore the
contents of the module file and instead reload the original source file.
Now that we can serialize pretty much everything*, though, we don't need
this, and instead we'd rather know where the serialization coverage has
gaps (by asserting).
Swift SVN r7752
This is basically the same as doing a :print_decl on every decl in the module,
except that it does not print extensions that come from other modules, and
/does/ print extensions and operators that come from this module.
Does not yet work for Clang modules or the Builtin module.
Swift SVN r7601
In Swift, a module is expected to know which libraries it needs, rather than
having this specified by an external module map. While we haven't quite
designed this yet (frameworks get this for free in Clang, for example),
we can at least provide a simple option for the common case of a module
associated with a single library.
This will probably change in the future, so I left in the more general
deserialization code I was working on before simplifying the use case.
A loaded module can in theory specify any arbitrary frameworks or libraries
as dependencies, not just a single dylib.
Swift SVN r7583
...instead of just those that are re-exported. This will be used for
autolinking (and probably few other places).
As part of this, we get two name changes:
(1) Module::getReexportedModules -> getImportedModules
(2) TranslationUnit::getImportedModules -> getImports
The latter doesn't just get modules-plus-access-paths; it also includes
whether or not the import is re-exported. Mainly, though, it just didn't
seem like a good idea to overload this name when the two functions aren't
really related.
No tests yet, will come with autolinking.
Swift SVN r7487
This will be used to resolve properties and method calls on objects with
dynamic-lookup ("id") type. For now, this is tested in swift-ide-test
by using the -dynamic-lookup-completion option and providing a
-code-completion-token value.
Caveats/TODOs:
- As before, since we're using the global method pool, this isn't scoped by
module. We could do a per-module filter, but I don't know if that will
actually buy us much.
- Again, Clang's method pool does not include methods from protocols.
- Lookup by selector name cannot find properties with a customized getter
name. <rdar://problem/14776565>
- The Clang-side method pool is keyed by selector, but Swift wants to look
things up by method name, which maps to the first selector piece, so we
end up having to do a scan of all the selectors in the pool.
Swift SVN r7330
With this, we can now get a list of all class members* available in the
current translation unit, which will be necessary for doing id-style
dynamic lookup (inferring which method you're referring to when the base
type is some magic "dynamic lookup" type).
* Including members of protocols, since a class we don't know about could
have implemented the protocol.
Since there is no code currently using this, I've added a new mode to
swift-ide-test to just dump all class members -- what will eventually
happen when you code complete on a dynamic lookup type. This mode will
go away once the other pieces of id-style lookup are in place.
Swift SVN r7287
...by adding a new callback to ModuleLoader: loadDeclsConformingTo.
This is used only when the type checker doesn't have enough contextual
information to resolve an expression involving a literal, so it's
possible many *LiteralConvertible types will never be loaded.
Deserialization of types with conversion methods is still eager, since
there's no easy hook to tell when they're needed, but the list has been
renamed to refer to any decls that need to be eagerly deserialized, in
case we need it for other purposes in the future.
This probably won't help much in a real program, but it cuts the test
run time by about 5-10% in my build.
Swift SVN r7268
