Moved all the threading code to one place. Added explicit support for
Darwin, Linux, Pthreads, C11 threads and Win32 threads, including new
implementations of Once for Linux, Pthreads, C11 and Win32.
rdar://90776105
Moved all the threading code to one place. Added explicit support for
Darwin, Linux, Pthreads, C11 threads and Win32 threads, including new
implementations of Once for Linux, Pthreads, C11 and Win32.
rdar://90776105
Create a new diagnostics serialization library, splitting out of
swiftAST to reduce the overheads for building the tooling for
cross-compiling the toolchain. This should reduce the build time for
swift-serialize-diagnostics to enable cross-compilation.
Move the source files from ParserSIL into the SIL library and remove the
ParserSIL library. ParsersSIL doesn't need to be its own library and this change will
simplify our builds.
Adds a tool `swift-symbolgraph-extract` that reads an existing Swift
module and prints a platform- and language-agnostic JSON description of
the module, primarly for documentation.
Adds a small sub-library `SymbolGraphGen` which houses the core
implementation for collecting relevant information about declarations.
The main entry point is integrated directly into the driver as a mode:
the tool is meant to be run outside of the normal edit-compile-run/test
workflow to avoid impacting build times.
Along with common options for other tools, unique options include
`pretty-print` for debugging, and a `minimum-access-level` options for
including internal documentation.
A symbol graph is a directed graph where the nodes are symbols in a
module and the edges are relationships between them. For example, a
`struct S` may have a member `var x`. The graph would have two nodes for
`S` and `x`, and one "member-of" relationship edge. Other relationship
kinds include "inherits-from" or "conforms to". The data format for a
symbol graph is still under development and may change without notice
until a specificiation and versioning scheme is published.
Various aspects about a symbol are recorded in the nodes, such as
availability, documentation comments, or data needed for printing the
shapes of declarations without having to understand specifics about the
langauge.
Implicit and public-underscored stdlib declarations are not included by
default.
rdar://problem/55346798
Separate out the host build of SwiftRemoteMirror from the "target" build
(the host for the standard library may be different from the host for
the compiler). Restructure the build to ensure that we use the correct
compiler for building the SwiftRemoteMirror for the host. This fixes a
build issue when building for Linux AArch64.
We would build two copies of swiftReflection, one for the host and one
for the target. However, the compiler configuration cannot be swapped
out in the middle as we were attempting to do. This would result in a
failure to build sometimes due to the missing dependency on the wanted
compiler. More importantly, it would also use the wrong compiler when
building the library. Although this duplicates the source paths,
correctness is preserved.
This refactors DWARFImporter to become a part of ClangImporter, since
it needs access to many of its implementation details anyway. The
DWARFImporterDelegate is just another mechanism for deserializing
Clang ASTs and once we have a Clang AST, the processing is effectively
the same.
Since the generated headers are in 'include/swift/Parse' they are globally accessible so make sure they are created early on.
If these get moved inside 'lib/Parse' then we can remove this dependency setting.
Instead of creating syntax nodes directly, modify the parser to invoke an abstract interface 'SyntaxParseActions' while it is parsing the source code.
This decouples the act of parsing from the act of forming a syntax tree representation.
'SyntaxTreeCreator' is an implementation of SyntaxParseActions that handles the logic of creating a syntax tree.
To enforce the layering separation of parsing and syntax tree creation, a static library swiftSyntaxParse is introduced to compose the two.
This decoupling is important for introducing a syntax parser library for SwiftSyntax to directly access parsing.
When debugging Objective-C or C++ code on Darwin, the debug info
collected by dsymutil in the .dSYM bundle is entirely
self-contained. It is possible to debug a program, set breakpoints and
print variables even without having the complete original source code
or a matching SDK available. With Swift, this is currently not the
case. Even though .dSYM bundles contain the binary .swiftmodule for
all Swift modules, any Clang modules that the Swift modules depend on,
still need to be imported from source to even get basic LLDB
functionality to work. If ClangImporter fails to import a Clang
module, effectively the entire Swift module depending on it gets
poisoned.
This patch is addressing this issue by introducing a ModuleLoader that
can ask queries about Clang Decls to LLDB, since LLDB knows how to
reconstruct Clang decls from DWARF and clang -gmodules producxes full
debug info for Clang modules that is embedded into the .dSYM budle.
This initial version does not contain any advanced functionality at
all, it merely produces an empty ModuleDecl. Intertestingly, even this
is a considerable improvement over the status quo. LLDB can now print
Swift-only variables in modules with failing Clang depenecies, and
becuase of fallback mechanisms that were implemented earlier, it can
even display the contents of pure Objective-C objects that are
imported into Swift. C structs obviously don't work yet.
rdar://problem/36032653
...finally breaking the dependency of Parse on Sema.
There are still some unfortunate dependencies here -- Xi's working on
getting /AST/ not dependent on Sema -- but this is a step forward.
It is a little strange that parseIntoSourceFile is in ParseSIL, and
therefore that that's still a dependency for anyone trying to, well,
parse. However, nearly all clients that parse want to type-check as
well, and that requires Sema, Serialization, and the ClangImporter...
and Serialization and SIL currently require each other as well
(another circular dependency). So it's not actively causing us trouble
right now.
The Swift 4 Migrator is invoked through either the driver and frontend
with the -update-code flag.
The basic pipeline in the frontend is:
- Perform some list of syntactic fixes (there are currently none).
- Perform N rounds of sema fix-its on the primary input file, currently
set to 7 based on prior migrator seasons. Right now, this is just set
to take any fix-it suggested by the compiler.
- Emit a replacement map file, a JSON file describing replacements to a
file that Xcode knows how to understand.
Currently, the Migrator maintains a history of migration states along
the way for debugging purposes.
- Add -emit-remap frontend option
This will indicate the EmitRemap frontend action.
- Don't fork to a separte swift-update binary.
This is going to be a mode of the compiler, invoked by the same flags.
- Add -disable-migrator-fixits option
Useful for debugging, this skips the phase in the Migrator that
automatically applies fix-its suggested by the compiler.
- Add -emit-migrated-file-path option
This is used for testing/debugging scenarios. This takes the final
migration state's output text and writes it to the file specified
by this option.
- Add -dump-migration-states-dir
This dumps all of the migration states encountered during a migration
run for a file to the given directory. For example, the compiler
fix-it migration pass dumps the input file, the output file, and the
remap file between the two.
State output has the following naming convention:
${Index}-${MigrationPassName}-${What}.${extension}, such as:
1-FixitMigrationState-Input.swift
rdar://problem/30926261
Previously it was part of swiftBasic.
The demangler library does not depend on llvm (except some header-only utilities like StringRef). Putting it into its own library makes sure that no llvm stuff will be linked into clients which use the demangler library.
This change also contains other refactoring, like moving demangler code into different files. This makes it easier to remove the old demangler from the runtime library when we switch to the new symbol mangling.
Also in this commit: remove some unused API functions from the demangler Context.
fixes rdar://problem/30503344
Add an option to the lexer to go back and get a list of "full"
tokens, which include their leading and trailing trivia, which
we can index into from SourceLocs in the current AST.
This starts the Syntax sublibrary, which will support structured
editing APIs. Some skeleton support and basic implementations are
in place for types and generics in the grammar. Yes, it's slightly
redundant with what we have right now. lib/AST conflates syntax
and semantics in the same place(s); this is a first step in changing
that to separate the two concepts for clarity and also to get closer
to incremental parsing and type-checking. The goal is to eventually
extract all of the syntactic information from lib/AST and change that
to be more of a semantic/symbolic model.
Stub out a Semantics manager. This ought to eventually be used as a hub
for encapsulating lazily computed semantic information for syntax nodes.
For the time being, it can serve as a temporary place for mapping from
Syntax nodes to semantically full lib/AST nodes.
This is still in a molten state - don't get too close, wear appropriate
proximity suits, etc.
Store leading a trailing "trivia" around a token, such as whitespace,
comments, doc comments, and escaping backticks. These are syntactically
important for preserving formatting when printing ASTs but don't
semantically affect the program.
Tokens take all trailing trivia up to, but not including, the next
newline. This is important to maintain checks that statements without
semicolon separators start on a new line, among other things.
Trivia are now data attached to the ends of tokens, not tokens
themselves.
Create a new Syntax sublibrary for upcoming immutable, persistent,
thread-safe ASTs, which will contain only the syntactic information
about source structure, as well as for generating new source code, and
structural editing. Proactively move swift::Token into there.
Since this patch is getting a bit large, a token fuzzer which checks
for round-trip equivlence with the workflow:
fuzzer => token stream => file1
=> Lexer => token stream => file 2 => diff(file1, file2)
Will arrive in a subsequent commit.
This patch does not change the grammar.
swift-reflection-test is now the test that forks a swift executable
and performs remote reflection, making it runnable on other targets,
such as the iOS simulator.
swift-reflection-dump is now a host-side tool that dumps the remote
reflection sections for any platform binary and will continue to
link in LLVM object file support.
This necessitates finally moving lib/Refleciton into stdlib/public,
since we're linking target-specific versions of the test tool and
we would eventually like to adopt some of this functionality in
the runtime anyway.
These likely don't have Swift type metadata but might be useful to
record or instantiate based on what kind of metadata is encountered for
the sake of memory tools.
- Don't depend on LLVM Support and swiftBasic as this will bring in
llvmSupport and other heavy dependencies, which we don't build for
non-host architectures right now anyway.
- Add a reader template that works with the same (albeit somewhat
clunky) interface in-process with the runtime and remotely, by having
the memory tool supply a function that will copy data from the remote
process.
- Add a Buffer type to abstract indirecting pointers in a remote address
space, which is handled by the memory reader.
- Start sketching out the C remote mirrors interface.
This adds the various TypeRef classes that will hold the
type references prior to substitution. There is also some
basic TypeRef visitor and dumping support.
rdar://problem/24173679
This commit adds a number of compression routines:
1. A dictionary based compression.
2. Huffman based compression.
3. A compression algorithm for swift names that's based on the other two.
This commit also adds two large autogenerated files: CBCTables.h and HuffTables.h.
These files contain the autogenerated string tables and auto-generated code for
fast compression/decompression. The internal tree data structures are lowered
into code that does the variable length encoding/decoding and searching of
fragments in the codebook. The files were generated by processing the symbols
from several large swift applications (stdlib, unittests, simd, ui app, etc).
The list of the programs is listed as part of the output of the tool in the
header file.
I decided to commit the auto-generated files for two reasons. First, we have a
cyclic dependency problem where we need to analyze the output of the compiler
(swift files) in order to generate the tables. And second, these tables will
become a part of the Swift ABI and should remain constant.
It should be possible to split the code that generates the Trie-based data
structure and auto-generate it as part of the Swift build process.
(libraries now)
It has been generally agreed that we need to do this reorg, and now
seems like the perfect time. Some major pass reorganization is in the
works.
This does not have to be the final word on the matter. The consensus
among those working on the code is that it's much better than what we
had and a better starting point for future bike shedding.
Note that the previous organization was designed to allow separate
analysis and optimization libraries. It turns out this is an
artificial distinction and not an important goal.
Replace ReST-flavored documentation comments with Markdown.
rdar://problem/20180412
In addition to full Markdown support, the following extensions are
supported. These appear as lists at the top level of the comment's
"document". All of these extensions are matched without regard to
case.
Parameter Outlines
------------------
- Parameters:
- x: ...
- y: ...
Separate Parameters
-------------------
- parameter x: ...
- parameter y: ...
- Note:
Parameter documentation may be broken up across the entire comment,
with a mix of parameter documentation kinds - they'll be consolidated
in the end.
Returns
-------
- returns: ...
The following extensions are also list items at the top level, which
will also appear in Xcode QuickHelp as first-class citizens:
- Attention: ...
- Author: ...
- Authors: ...
- Bug: ...
- Complexity: ...
- Copyright: ...
- Date: ...
- Experiment: ...
- Important: ...
- Invariant: ...
- Note: ...
- Postcondition: ...
- Precondition: ...
- Remark: ...
- Remarks: ...
- See: ...
- Since: ...
- Todo: ...
- Version: ...
- Warning: ...
These match most of the extra fields in Doxygen, plus a few more per request.
Other changes
-------------
- Remove use of rawHTML for all markup AST nodes except for those
not representable by the Xcode QuickHelp XSLT - <h>, <hr/>, and of
course inline/block HTML itself.
- Update the doc comment RNG schema to more accurately reflect Xcode
QuickHelp.
- Clean up cmark CMake configuration.
- Rename "FullComment" to "DocComment"
- Update the Swift Standard Documentation (in a follow-up commit)
- Update SourceKit for minor changes and link against cmark
(in a follow-up commit).
Swift SVN r27727
This library becomes a permanent interface that we would need to support
in the long term, so we should get dylib names and APIs right.
rdar://20418214
Swift SVN r26957
OptimizeARC does not only contain an optimize arc pass: the library also
includes aa. What this really is a repository of the extra passes and
infrastructure that we inject into LLVM. Thus LLVMPasses is a more descriptive
name. It also matches SILPasses.
I also taught lit how to use the new llvm-opt driver for running swift llvm
passes through opt without having to remember how to setup the dynamic swift
llvm pass dylib. You can use this in lit tests by using the substitution
%llvm-opt.
Swift SVN r21654
This is completely non-functional at the moment, but will eventually
turn into a side-car data file that we can use to describe various
properties of C/Objective-C entities that can be fed into the Clang
importer. It's provided as a separate library because this information
might eventually be useful for Clang as well.
Swift SVN r19987
This allows swiftFrontend to drop its dependency on swiftDriver, and could
someday allow us to move the integrated frontend's option parsing out of
swiftFrontend (which would allow other tools which use swiftFrontend to
exclude the option table entirely).
Swift SVN r19824
It does not yet understand a lot of ReST constructs, in particular, it does not
understand inline markup at all, but it is good enough to extract brief
comments.
The parser in BriefParser.cpp does not construct any ASTs or emit any warnings.
This is a special parser for brief comments that should be as fast as possible.
Real ReST parser will come later.
Swift SVN r15046
