* Move `AvailabilitySpec` handling logic to AST, so they can be shared
between libParse and ASTGen
* Requestify '-define-availability' arguments parsing and parse them
with 'SwiftParser' according to the 'ParserASTGen' feature flag
* Implement 'AvailableAttr' generation in ASTGen
We're not caching this now, but it lets us dodge annoying layering
issues because ASTGen (where this is available) sits on top of the C++
parser, which needs to call it.
This is phase-1 of switching from llvm::Optional to std::optional in the
next rebranch. llvm::Optional was removed from upstream LLVM, so we need
to migrate off rather soon. On Darwin, std::optional, and llvm::Optional
have the same layout, so we don't need to be as concerned about ABI
beyond the name mangling. `llvm::Optional` is only returned from one
function in
```
getStandardTypeSubst(StringRef TypeName,
bool allowConcurrencyManglings);
```
It's the return value, so it should not impact the mangling of the
function, and the layout is the same as `std::optional`, so it should be
mostly okay. This function doesn't appear to have users, and the ABI was
already broken 2 years ago for concurrency and no one seemed to notice
so this should be "okay".
I'm doing the migration incrementally so that folks working on main can
cherry-pick back to the release/5.9 branch. Once 5.9 is done and locked
away, then we can go through and finish the replacement. Since `None`
and `Optional` show up in contexts where they are not `llvm::None` and
`llvm::Optional`, I'm preparing the work now by going through and
removing the namespace unwrapping and making the `llvm` namespace
explicit. This should make it fairly mechanical to go through and
replace llvm::Optional with std::optional, and llvm::None with
std::nullopt. It's also a change that can be brought onto the
release/5.9 with minimal impact. This should be an NFC change.
Avoid parsing the syntax tree up-front, and instead
only parse it when required, which happens when either:
1. ASTGen parsing is enabled (currently disabled
by default)
2. Round trip checking is enabled for a primary
file (enabled by default in a debug build,
except when dep scanning or doing an IDE
operation)
3. We need to evaluate a macro in that file
This change therefore means that we now no longer
need to parse the syntax tree for secondary files
by default unless we specifically need to evaluate
a macro in them (e.g if we need to lookup a member
on a decl with an attached macro). And the same
for primaries in release builds.
rdar://109283847
In the Swift grammar, the top-level of a source file is a mix of three
different kinds of "items": declarations, statements, and expressions.
However, the existing parser forces all of these into declarations at
parse time, wrapping statements and expressions in TopLevelCodeDecls,
so the primary API for getting the top-level entities in source files
is based on getting declarations.
Start generalizing the representation by storing ASTNode instances at
the top level, rather than declaration pointers, updating many (but
not all!) uses of this API. The walk over declarations is a (cached)
filter to pick out all of the declarations. Existing parsed files are
unaffected (the parser still creates top-level code declarations), but
the new "macro expansion" source file kind skips creating top-level
code declarations so we get the pure parse tree. Additionally, some
generalized clients (like ASTScope lookup) will now look at the list
of items, so they'll be able to walk into statements and expressions
without the intervening TopLevelCodeDecl.
Over time, I'd like to phase out `getTopLevelDecls()` entirely,
relying on the new `getTopLevelItems()` for parsed content. We can
introduce TopLevelCodeDecls more lazily for semantic walks.
Currently when parsing a SourceFile, the parser
gets handed pointers so that it can write the
interface hash and collected tokens directly into
the file. It can also call `setSyntaxRoot` at
the end of parsing to set the syntax tree.
In preparation for the removal of
`performParseOnly`, this commit formalizes these
values as outputs of `ParseSourceFileRequest`,
ensuring that the file gets parsed when the
interface hash, collected tokens, or syntax tree
is queried.
Split off the notion of "recording" dependencies from the notion of
"collecting" dependencies. This corrects an oversight in the previous
design where dependency replay and recording were actually not "free" in
WMO where we actually never track dependencies. This architecture also
lays the groundwork for the removal of the referenced name trackers.
The algorithm builds upon the infrastructure for dependency sources and
sinks laid down during the cut over to request-based dependency tracking
in #30723.
The idea of the naive algorithm is this:
For a chain of requests A -> B* -> C -> D* -> ... -> L where L is a lookup
request and all starred requests are cached, once L writes into the
dependency collector, the active stack is walked and at each cache-point
the results of dependency collection are associated with the request
itself (in this example, B* and D* have all the names L found associated
with them). Subsequent evaluations of these cached requests (B* and D*
et al) will then *replay* the previous lookup results from L into the
active referenced name tracker. One complication is, suppose the
evaluation of a cached request involves multiple downstream name
lookups. More concretely, suppose we have the following request trace:
A* -> B -> L
|
-> C -> L
|
-> D -> L
|
-> ...
Then A* must see the union of the results of each L. If this reminds
anyone of a union-find, that is no accident! A persistent union-find
a la Conchon and Filliatre is probably in order to help bring down peak
heap usage...
Define a new type DependencyCollector that abstracts over the
incremental dependency gathering logic. This will insulate the
request-based name tracking code from future work on private,
intransitive dependencies.
Code Completion operates on a CompilerInstance that passes a primary
file down for type checking. This means it creates and registers
dependencies in the referenced name trackers. Despite the fact that
those dependencty edges are unused,
because the would-be swiftdeps file is never written to disk,
it is still a dependency source that should participate in the
request-based dependency tracking refactor.
Rework the lazy function body parsing mechanism to use the
request-evaluator, so that asking for the body of a function will
initiate parsing. Clean up a number of callers to
AbstractFunctionDecl::getBody() that don't actually need the body, so
we don't perform unnecessary parsing.
This change does not delay parsing of function bodies in the general
case; rather, it sets up the infrastructure to always delay parsing of
function bodies.
Rework the lazy function body parsing mechanism to use the
request-evaluator, so that asking for the body of a function will
initiate parsing. Clean up a number of callers to
AbstractFunctionDecl::getBody() that don't actually need the body, so
we don't perform unnecessary parsing.
This change does not delay parsing of function bodies in the general
case; rather, it sets up the infrastructure to always delay parsing of
function bodies.
Ensure that lazy parsing of the members of nominal type definitions
and extensions is handled through a request. Most of the effort here
is in establishing a new request zone for parser requests.
