Currently symbol graphs are always written in files that contain 1 to 2
module names. It's possible for Swift module names to be very long, so
combining 2 of them in file name like `module1@module2...` in the same
path component means the name can be too long for some file systems. The
new option `-symbol-graph-shorten-output-names` changes the symbol graph
output files to use a MD5 hash of the module name(s) as the filename and
outputs an additional JSON file with the original names mapped to the
real filename. The module names JSON can be used to construct a VFS
overlay with the original naming scheme.
fix#83723
I considered using vfsoverlay, which seems like a viable solution, but
the vfsoverlay options don't seem to apply to any of the outputs from
the compiler. When I set an overlay to remap the symbol graph file
outputs, the remapped external paths aren't used so the root problem of
too long file names remains.
Reformatting everything now that we have `llvm` namespaces. I've
separated this from the main commit to help manage merge-conflicts and
for making it a bit easier to read the mega-patch.
This is phase-1 of switching from llvm::Optional to std::optional in the
next rebranch. llvm::Optional was removed from upstream LLVM, so we need
to migrate off rather soon. On Darwin, std::optional, and llvm::Optional
have the same layout, so we don't need to be as concerned about ABI
beyond the name mangling. `llvm::Optional` is only returned from one
function in
```
getStandardTypeSubst(StringRef TypeName,
bool allowConcurrencyManglings);
```
It's the return value, so it should not impact the mangling of the
function, and the layout is the same as `std::optional`, so it should be
mostly okay. This function doesn't appear to have users, and the ABI was
already broken 2 years ago for concurrency and no one seemed to notice
so this should be "okay".
I'm doing the migration incrementally so that folks working on main can
cherry-pick back to the release/5.9 branch. Once 5.9 is done and locked
away, then we can go through and finish the replacement. Since `None`
and `Optional` show up in contexts where they are not `llvm::None` and
`llvm::Optional`, I'm preparing the work now by going through and
removing the namespace unwrapping and making the `llvm` namespace
explicit. This should make it fairly mechanical to go through and
replace llvm::Optional with std::optional, and llvm::None with
std::nullopt. It's also a change that can be brought onto the
release/5.9 with minimal impact. This should be an NFC change.
This includes:
- bumping the SWIFT_SYMBOLGRAPH_FORMAT_MINOR version
- introduction of the "swift.extension" symbol and "extensionTo" relationship
- adding support for ExtensionDecl to the Symbol class
- adding a "typeKind" field to the symbol's extension mixin which indicates what kind
of symbol was extended
- intoduction of the -emit-extension-block-symbols flag, which enables the behavior
outlined below
- adaptions to SymbolGraphASTWalker that ensure a swift.extension symbol is emitted
for each extension to a type that does not exist in the local symbol graph
- adaptions to SymbolGraph and SymbolGraphASTWalker that ensure member and conformance
relationships are correctly associated with the swift.extension symbol instead of
the original type declaration's (extended nominal's) symbol where applicable
- adaptions to SymbolGraphASTWalker that ensure swift.extension symbols are connected
to their respective extended nominal's symbol using an extensionTo relationship
Testing:
- adds SymbolGraph tests that test behavior only relevant in
-emit-extension-block-symbols mode
- adapts some SymbolGraph tests to additionally test similar behavior for
extensions to external types in -emit-extension-block-symbols mode
- adapts some SymbolGraph tests to (additionally or exclusively) test the
behavior with -emit-extension-block-symbols mode enabled
Bugfixes:
- fixes a bug where some conformsTo relationships implicated by the conformances
declared on an extension to an external type were not emitted
(see test/SymbolGraph/Relationships/ConformsTo/Indirect.swift)
Further changes:
- documents the strategy for naming and associating children declared in extensions
to typealiases (see test/SymbolGraph/Relationships/MemberOf/Typealias.swift,
test/SymbolGraph/Symbols/Names.swift)
This currently doesn't check for inherited docs, ie. either the
imported declaration has docs or it doesn't. There's also a few odd
cases with mixed doc types and when each line is prefixed with '*', but
it's good enough for an initial implementation.
Moves UTF8 sanitisation out of ASTPrinter.h and into Unicode.h so that
it can be used here as well.
Resolves rdar://91388603.
When the SymbolGraph json is requested via (key.retrieve_symbol_graph: 1) this adds
a new field in the response that lists all the parent contexts of the symbol under
the cursor with their symbol graph kind and name, and their USR:
key.parent_contexts: [
{
key.kind: "swift.struct",
key.name: "Parent",
key.usr: "s:27cursor_symbol_graph_parents6ParentV"
},
...
]
}
Resolves rdar://problem/73904365
Adds a new 'key.retrieve_symbol_graph' option to the request. When set to 1 it
includes the JSON for a SymbolGraph containing a single node for the symbol at
the requested position.
This also extends the SymbolGraph library with a new entry point to get a graph
for a single symbol, and to additionally support type substitution to match the
existing CursorInfo behavior (e.g. so that when invoked on `first` in
`Array<Int>().first`, the type is given as `Int?` rather than `Element?`).
Resolves rdar://problem/70551509
Emit copies of default implementations in protocol extensions and superclass declarations in conforming types and subclasses respectively using a virtual USR, i.e. `${REAL_USR}::SYNTHESIZED::${CONFORMING_OR_SUBCLASS_TYPE_USR}`.
- Add a -skip-synthesized-members option to skip these synthesized members.
- Create a new wrapping `Symbol` type that can also contain a base type declaration as well as the inherited declaration for those synthesized cases. Move some symbol-specific APIs there.
- Doc comments can “cascade” down to protocol extensions or refinements in concrete types. When emitting the doc comment for a symbol, look up through to superclasses or protocol requirements for where a doc comment is actually written.
- Clean up filtering of implicitly private (e.g. “public underscored”) types
rdar://problem/59128787
When a module extends a type from another module, serialize those symbols into
separated files dedicated to those extended modules. This makes it easier to
ingest and categorize those symbols under the extended module if desired.
rdar://58941718
Adds a tool `swift-symbolgraph-extract` that reads an existing Swift
module and prints a platform- and language-agnostic JSON description of
the module, primarly for documentation.
Adds a small sub-library `SymbolGraphGen` which houses the core
implementation for collecting relevant information about declarations.
The main entry point is integrated directly into the driver as a mode:
the tool is meant to be run outside of the normal edit-compile-run/test
workflow to avoid impacting build times.
Along with common options for other tools, unique options include
`pretty-print` for debugging, and a `minimum-access-level` options for
including internal documentation.
A symbol graph is a directed graph where the nodes are symbols in a
module and the edges are relationships between them. For example, a
`struct S` may have a member `var x`. The graph would have two nodes for
`S` and `x`, and one "member-of" relationship edge. Other relationship
kinds include "inherits-from" or "conforms to". The data format for a
symbol graph is still under development and may change without notice
until a specificiation and versioning scheme is published.
Various aspects about a symbol are recorded in the nodes, such as
availability, documentation comments, or data needed for printing the
shapes of declarations without having to understand specifics about the
langauge.
Implicit and public-underscored stdlib declarations are not included by
default.
rdar://problem/55346798
