Previously we were using the same set of conditions
for serializing as for swiftdoc, so excluded them.
However it's reasonable to have them in the
swiftsourceinfo.
* Remove dead ModuleSourceInfoFilename parameters
These were never actually used; we might find a way to bring them back later.
* Introduce SerializedModuleBaseName
This is intended to replace the _n_ filename parameters that tend to get passed around in the SerializedModuleLoader classes.
* Manipulate currPath in SerializedModuleLoader less often
* Don’t pass raw paths around SerializedModuleLoader
Only pass base names.
* Regularize module file opening functions
SwiftSourceInfo files provide source location information for decls coming from
loaded modules. For most IDE use cases it either has an undesirable impact on
performance with no benefit (code completion), results in stale locations being
used instead of more up-to-date indexer locations (cursor info), or has no
observable effect (live diagnostics, which are filtered to just those with a
location in the primary file).
For non-IDE clients of SourceKit though, cursor info providing declaration
locations for symbols from other modules is useful, so add a global
configuration option (and a new request to set it) to control whether
.swiftsourceinfo files are loaded or not based on use case (they are loaded by
default).
After setting up the .swiftsourceinfo file, this patch starts to actually serialize
and de-serialize source locations for declaration. The binary format of .swiftsourceinfo
currently contains these three records:
BasicDeclLocs: a hash table mapping from a USR ID to a list of basic source locations. The USR id
could be retrieved from the following DeclUSRs record using an actual decl USR. The basic source locations
include a file ID and the results from Decl::getLoc(), ValueDecl::getNameLoc(), Decl::getStartLoc() and Decl::getEndLoc().
The file ID could be used to retrieve the actual file name from the following SourceFilePaths record.
Each location is encoded as a line:column pair.
DeclUSRS: a hash table mapping from USR to a USR ID used by location records.
SourceFilePaths: a hash table mapping from a file ID to actual file name.
BasicDeclLocs should be sufficient for most diagnostic cases. If additional source locations
are needed, we could always add new source location records without breaking the backward compatibility.
When de-serializing the source location from a module-imported decl, we calculate its USR, retrieve the USR ID
from the DeclUSRS record, and use the USR ID to look up the basic location list in the BasicDeclLocs record.
For more details about .swiftsourceinfo file: https://forums.swift.org/t/proposal-emitting-source-information-file-during-compilation
We previously shied away from this in order to not /accidentally/
depend on it, but it becomes interesting again with textual
interfaces, which can certainly be read by humans. The cross-file
order is the order of input files, which is at least controllable by
users.
Store leading a trailing "trivia" around a token, such as whitespace,
comments, doc comments, and escaping backticks. These are syntactically
important for preserving formatting when printing ASTs but don't
semantically affect the program.
Tokens take all trailing trivia up to, but not including, the next
newline. This is important to maintain checks that statements without
semicolon separators start on a new line, among other things.
Trivia are now data attached to the ends of tokens, not tokens
themselves.
Create a new Syntax sublibrary for upcoming immutable, persistent,
thread-safe ASTs, which will contain only the syntactic information
about source structure, as well as for generating new source code, and
structural editing. Proactively move swift::Token into there.
Since this patch is getting a bit large, a token fuzzer which checks
for round-trip equivlence with the workflow:
fuzzer => token stream => file1
=> Lexer => token stream => file 2 => diff(file1, file2)
Will arrive in a subsequent commit.
This patch does not change the grammar.
Let's say I am a good citizen and document my private symbols:
/** My TOP SECRET DOCUMENTATION */
private class Foo {
}
When I go to distribute the compiled binary, I find out my private
documentation is distributed as well:
$ swiftc test.swift -emit-module -module-name "test"
$ strings test.swiftdoc
My TOP SECRET DOCUMENTATION
/** My TOP SECRET DOCUMENTATION */
If a client can't use a symbol (e.g. it's private [or internal and not
-enable-testing]) don't emit the documentation for a symbol in the
swiftdoc.
Fixes: SR-762, rdar://21453624
The test coverage implements this truth table:
| visibility | -enable-testing | documentation? |
|------------|-----------------|----------------|
| private | no | ❌ |
| internal | no | ❌ |
| public | no | ✅ |
| private | yes | ❌ |
| internal | yes | ✅ |
| public | yes | ✅ |
Modified the existing comments test coverage to expect non-public
documentation not to be emitted.
Don't rely on existing comment structure
Refuse to emit comments if the decl cannot actually have one. To
accomplish this, we move `canHaveComment` into the Decl instance. It
must also be marked `const`, since one of its existing usages operates
on a const pointer.
Perform fewer checks when serializing the standard library.
We did not serialize them because getting USR for extensions is tricky (USRs are
usually for value decls). This commit starts to make up an USR for an extension by combining
the extended nominal's USR with the USR of the first value member of the extension. We use
this made-up USR to associate doc comments when (de)serializing them.
Most tests were using %swift or similar substitutions, which did not
include the target triple and SDK. The driver was defaulting to the
host OS. Thus, we could not run the tests when the standard library was
not built for OS X.
Swift SVN r24504
The driver infers the filename from the module file by replacing the extension,
and passes the explicit path to the swiftdoc file to the frontend. But there
is no option in the driver to control emission of swiftdoc (it is always
emitted, and name is always inferred from the swiftmodule name).
The swiftdoc file consists of a single table that maps USRs to {brief comment,
raw comment}. In order to look up a comment for decl we generate the USR
first. We hope that the performance hit will not be that bad, because most
declarations come from Clang. The advantage of this design is that the
swiftdoc file is not locked to the swiftmodule file, and can be updated,
replaced, and even localized.
Swift SVN r14914
