The original module names themselves must still be valid unescaped identifiers; most of the serialization logic in the compiler depends on the name of a module matching its name on the file system, and it would be very complex to turn escaped identifiers into file-safe names.
Raw identifiers are backtick-delimited identifiers that can contain any
non-identifier character other than the backtick itself, CR, LF, or other
non-printable ASCII code units, and which are also not composed entirely
of operator characters.
This is phase-1 of switching from llvm::Optional to std::optional in the
next rebranch. llvm::Optional was removed from upstream LLVM, so we need
to migrate off rather soon. On Darwin, std::optional, and llvm::Optional
have the same layout, so we don't need to be as concerned about ABI
beyond the name mangling. `llvm::Optional` is only returned from one
function in
```
getStandardTypeSubst(StringRef TypeName,
bool allowConcurrencyManglings);
```
It's the return value, so it should not impact the mangling of the
function, and the layout is the same as `std::optional`, so it should be
mostly okay. This function doesn't appear to have users, and the ABI was
already broken 2 years ago for concurrency and no one seemed to notice
so this should be "okay".
I'm doing the migration incrementally so that folks working on main can
cherry-pick back to the release/5.9 branch. Once 5.9 is done and locked
away, then we can go through and finish the replacement. Since `None`
and `Optional` show up in contexts where they are not `llvm::None` and
`llvm::Optional`, I'm preparing the work now by going through and
removing the namespace unwrapping and making the `llvm` namespace
explicit. This should make it fairly mechanical to go through and
replace llvm::Optional with std::optional, and llvm::None with
std::nullopt. It's also a change that can be brought onto the
release/5.9 with minimal impact. This should be an NFC change.
While skipping, if we encounter a token that looks
like it could be the start of a `/.../` regex
literal, fall back to parsing the function or type
body normally, as such a token could become a
regex literal. As such, it could treat `{` and
`}` as literal, or otherwise have contents that
would be lexically invalid Swift.
To avoid falling back in too many cases, we apply
the existing regex literal heuristics. Cases that
pass the heuristic fall back to regular parsing.
Cases that fail the heuristic are further checked
to make sure they wouldn't contain an unbalanced
`{` or `}`, but otherwise are allowed to be
skipped. This allows us to continue skipping for
most occurrences of infix and prefix `/`.
This is meant as a lower risk workaround to fix the
the issue, we ought to go back to handling regex
literals in the lexer.
Resolves rdar://95354010
Teach the lexer not to consider `/` an operator
character when attempting to re-lex a regex
literal. This allows us to split off a prefix
operator.
Previously this was done after-the-fact in the
parser, but that didn't cover the unapplied infix
operator case, and didn't form a `tok::amp_prefix`
for `foo(&/.../)`, which led to a suboptimal
diagnostic.
This also now means we'll split an operator for
cases such as `foo(!/^/)` rather than treating it
as an unapplied infix operator.
rdar://92469917
Queue up diagnostics when lexing, waiting until
`Lexer::lex` is called before emitting them. This
allows us to re-lex without having to deal with
previously invalid tokens.
This reverts commit a67a0436f7, reversing
changes made to 9965df76d0.
This commit or the earlier commit this commit is based on (#40531) broke the
incremental bot.
Update the lexing implementation to defer to the
regex library, which will pass back the pointer
from to resume lexing, and update the emission to
call the new `Regex(_regexString:version:)`
overload, that will accept the regex string with
delimiters.
Because this uses the library's lexing
implementation, the delimiters are now `'/.../'`
and `'|...|'` instead of plain `'...'`.
With `-enable-experimental-string-processing`,
start lexing `'` delimiters as regex literals (this
is just a placeholder delimiter for now). The
contents of which gets passed to the libswift
library, which can return an error string to be
emitted, or null for success.
The libswift side isn't yet hooked up to the Swift
regex parser, so for now just emit a dummy
diagnostic for regexes starting with quantifiers.
If successful, build an AST node which will be
emitted as an implicit call to an
`init(_regexString:)` initializer of an in-scope
`Regex` decl (which will eventually be a known
stdlib decl).
Previously, when we reached the maximum nesting level, we changed the current token’s kind to an EOF token. A lot of places in the parser are not set up to expect this token change. The intended workaround was to check whether pushing a structure marker failed (which would change the token kind) and bail out parsing if this happened. This was fragile and caused assertion failures in assert builds.
Instead of changing the current token’s kind, and failing to push the structure marker, let the lexer know that it should cut off lexing, essentially making the input buffer stop at the current position. The parser will continue to consume its current token (`Parser.Tok`) and the next token that’s already lexed in the lexer (`Lexer.NextToken`) before reaching the emulated EOF token. Thus two more tokens are parsed than before, but that shouldn’t make much of a difference.
* Implement 'getDiagnosticSeverity()' and 'getDiagnosticMessage()' on
'CodeCompletionResult'
* Differentiate 'RedundantImportIndirect' from 'RedundantImport'
* Make non-Sendable check respects '-warn-concurrency'
rdar://76129658
This is an intermediate state in which the lexer delegates the
responsibility for trivia lexing to the parser. Later, the parser will
delegate this responsibility to SyntaxParsingContext which will hand it
over to SyntaxParseAction, which will only lex the pieces if it is
really necessary to do so.
The lexer is only responsible for skipping over trivia and noting their
length. A separate TriviaLexer can be invoked to split the raw trivia
string into its pieces.
Since most of the time the trivia pieces aren't needed, this will allow
us to later only parse trivia into pieces when they are explicitly
needed.
When the location given to getLocForStartOfLine was an empty line, it
would actually return the location of the next line rather than the
given location as it should.
If the location given to getLocForEndOfLine was inside a token on a line
that was either empty or contained whitespace, it would skip to the end
of that token and then return the location for the next line. This was
an issue for multiline strings, where the string is a single token but
it's over multiple lines.
* WIP implementation
* Cleanup implementation
* Install backedge rather than storing array reference
* Add diagnostics
* Add missing parameter to ResultFinderForTypeContext constructor
* Fix tests for correct fix-it language
* Change to solution without backedge, change lookup behavior
* Improve diagnostics for weak captures and captures under different names
* Remove ghosts of implementations past
* Address review comments
* Reorder member variable initialization
* Fix typos
* Exclude value types from explicit self requirements
* Add tests
* Add implementation for AST lookup
* Add tests
* Begin addressing review comments
* Re-enable AST scope lookup
* Add fixme
* Pull fix-its into a separate function
* Remove capturedSelfContext tracking from type property initializers
* Add const specifiers to arguments
* Address review comments
* Fix string literals
* Refactor implicit self diagnostics
* Add comment
* Remove trailing whitespace
* Add tests for capture list across multiple lines
* Add additional test
* Fix typo
* Remove use of ?: to fix linux build
* Remove second use of ?:
* Rework logic for finding nested self contexts
Allow the use of declarations whose names start with $ in all
modes. However, normal code cannot define new entities with names that
start with $: only the implementation can do that, e.g., for property
delegates.
Previously, the Lexer kept a single flag whether we’re lexing Swift or SIL. Instead, keep track if we’re parsing Swift, SIL, or a Swiftinterface file. .swiftinterface files allow $-prefixed identifiers anywhere.
We've been running doxygen with the autobrief option for a couple of
years now. This makes the \brief markers into our comments
redundant. Since they are a visual distraction and we don't want to
encourage more \brief markers in new code either, this patch removes
them all.
Patch produced by
for i in $(git grep -l '\\brief'); do perl -pi -e 's/\\brief //g' $i & done
Multiline string literal at attribute message position was disallowed in
59778f8ecb.
Reworked to try to at least get multiline strings working which might be
useful as messages for attributes (for example a detailed “unavailable”
annotation) minus the code which read off the start of the StringRef buffer.
Lexer::getEncodedStringSegment (now getEncodedStringSegmentImpl)
assumes that it can read one byte past the end of a string segment in
order to avoid bounds-checks on things like "is this a \r\n
sequence?". However, the function was being used for strings that did
not come from source where this assumption was not always valid.
Change the reusable form of the function to always copy into a
temporary buffer, allowing the fast path to continue to be used for
normal parsing.
Caught by ASan!
rdar://problem/44306756
Having this be a single buffer hardcoded in the SourceManager and set
by all clients is silly. SourceFiles with the 'Main' kind are allowed
to have hashbang lines (`#!`), other files are not. And anyone
manually setting up a Lexer can decide for themselves.
No intended behavioral change.
