Commit Graph

103 Commits

Author SHA1 Message Date
Hamish Knight
81dc7d87ed [Parse] Remove TypeChecker and SIL options parameter from ParserUnit
Providing these is a bit of a layering violation,
the parser shouldn't care about these options (there
does seem to be one current use of `TypeCheckerOpts`
in the parser for designated operator types, but
that's a legacy feature that was never officially
supported).
2024-10-17 14:04:34 +01:00
Ben Barham
9779c18da3 Rename startswith to starts_with
LLVM is presumably moving towards `std::string_view` -
`StringRef::startswith` is deprecated on tip. `SmallString::startswith`
was just renamed there (maybe with some small deprecation inbetween, but
if so, we've missed it).

The `SmallString::startswith` references were moved to
`.str().starts_with()`, rather than adding the `starts_with` on
`stable/20230725` as we only had a few of them. Open to switching that
over if anyone feels strongly though.
2024-03-13 22:25:47 -07:00
Ben Barham
ef8825bfe6 Migrate llvm::Optional to std::optional
LLVM has removed llvm::Optional, move over to std::optional. Also
clang-format to fix up all the renamed #includes.
2024-02-21 11:20:06 -08:00
pinkjuice66
50d6b1f663 [Parse] Add test cases for validating UTF-8 correctness 2024-01-09 19:31:00 +09:00
Evan Wilde
65efe663e6 Replace hasValue with std::optional::has_value
Optional changed to std::optional, so there is no hasValue. Replacing it
so that it builds.
2023-08-03 10:06:44 -07:00
Ben Barham
3ec878d918 Update llvm::Optional API uses
Use the std-equivalent names as the LLVM ones are now deprecated
(eventually `llvm::Optional` will disappear):
  - `getValue` -> `value`
  - `getValueOr` -> `value_or`
  - `hasValue` -> `has_value`

Follow up from ab1b343dad and
7d8bf37e5e with some missing cases.
2023-01-25 16:28:10 -08:00
Alex Hoppen
fe2ae72ad2 [IDE] Rename CodeCompletion to IDEInspection in cases where the code path no longer exclusively applies to code completion
The code completio infrastructure is also being used for cursor info now, so it should no longer be called code completion.

rdar://103251187
2022-12-13 11:41:05 +01:00
Robert Widmann
37e7052c68 Remove -emit-syntax and -verify-syntax-tree 2022-11-16 15:07:48 -08:00
Robert Widmann
d1cadb86c5 Remove Explicit Trivia from Lexer 2022-11-16 14:52:28 -08:00
Robert Widmann
25a8655773 Delete swift-syntax-test and Syntax Unit Tests 2022-11-16 13:38:25 -08:00
Robert Widmann
91c262bcb7 Move TokenKinds.def.gyb to AST 2022-11-16 13:38:25 -08:00
Doug Gregor
0f9a70601a Parse and record top-level "items" rather than always forcing declarations.
In the Swift grammar, the top-level of a source file is a mix of three
different kinds of "items": declarations, statements, and expressions.
However, the existing parser forces all of these into declarations at
parse time, wrapping statements and expressions in TopLevelCodeDecls,
so the primary API for getting the top-level entities in source files
is based on getting declarations.

Start generalizing the representation by storing ASTNode instances at
the top level, rather than declaration pointers, updating many (but
not all!) uses of this API. The walk over declarations is a (cached)
filter to pick out all of the declarations. Existing parsed files are
unaffected (the parser still creates top-level code declarations), but
the new "macro expansion" source file kind skips creating top-level
code declarations so we get the pure parse tree. Additionally, some
generalized clients (like ASTScope lookup) will now look at the list
of items, so they'll be able to walk into statements and expressions
without the intervening TopLevelCodeDecl.

Over time, I'd like to phase out `getTopLevelDecls()` entirely,
relying on the new `getTopLevelItems()` for parsed content. We can
introduce TopLevelCodeDecls more lazily for semantic walks.
2022-11-01 08:04:15 -07:00
Robert Widmann
18b79ffcfd Resolve a Layering Violation in libBasic
Basic should not be allowed to link Parse, yet it was doing so
to allow Version to provide a constructor that would conveniently
parse a StringRef. This entrypoint also emitted diagnostics, so it
pulled in libAST.

Sink the version parser entrypoint down into Parse where it belongs
and point all the clients to the right place.
2022-09-09 00:21:30 -07:00
Rintaro Ishizaki
a673043737 Terminology change: 'garbage' -> 'unexpected'
There are no "garbage" characters in Swift code. They are just
"unexpected."
2022-08-15 14:32:28 -07:00
Robert Widmann
003b362a65 Differentiate Shebang from Garbage
Add #! as an explicit kind of trivia.
2022-07-08 18:28:57 -07:00
Becca Royal-Gordon
3843c7cd5e Update SWIFT_COMPILER_VERSION language features
The `SWIFT_COMPILER_VERSION` define is used to stamp a vendor’s version number into a Swift compiler binary. It can be queried from Swift code using `#if _compiler_version` and from Clang by using a preprocessor definition called `__SWIFT_COMPILER_VERSION`. These are unsupported compiler-internal features used primarily by Apple Swift.

In Swift 1.0 through 5.5, Apple Swift used a scheme for `SWIFT_COMPILER_VERSION` where the major version matched the embedded clang (e.g. 1300 for Apple Clang 13.0.0) and the minor version was ignored. Starting in Swift 5.6, Apple Swift started using major and minor version numbers that matched the Swift.org version number. This makes them easier to understand, but it means that version 1300.0.x was followed by version 5.6.x. Not only did version numbers go backwards, but also the old logic to ignore minor versions was now a liability, because it meant you would not be able to target a change to 5.7.x compilers but not 5.6.x compilers.

This commit addresses the problem by:

* Modifying the existing `#if _compiler_version(string-literal)` feature so it transforms the major version into a major and minor that will compare correctly to new version numbers. For instance, “1300.*” is transformed into “1.300”, which will compare correctly to a “5.6” or “5.7” version even if it doesn’t really capture the fact that “1300” was a Swift 5.5 compiler. As a bonus, this allows you to use the feature to backwards-compatibly test new compilers using the existing feature: “5007.*” will be seen by compilers before 5.7 as an unknown future version, but will be seen by 5.7 compilers as targeting them.

* Modifying the `__SWIFT_COMPILER_VERSION` clang define similarly so that, to preprocessor conditions written for the old scheme, a 5.7 compiler will appear to have major version 5007.

* Adding a new variant of `#if _compiler_version` with the same syntax as `#if swift` and `#if compiler`—that is, taking a comparison operator and a bare set of dotted version numbers, rather than a string literal. Going forward, this will be how version checks are written once compatibility with compilers before this change is no longer a concern.

These changes are only lightly tested because tests have to work without any compiler version defined (the default in most configurations), but I’ve tested what I can.

Fixes rdar://89841295.
2022-04-27 18:27:52 -07:00
Hamish Knight
080d59b3df [Lexer] Delay token diagnostics
Queue up diagnostics when lexing, waiting until
`Lexer::lex` is called before emitting them. This
allows us to re-lex without having to deal with
previously invalid tokens.
2022-04-12 16:03:47 +01:00
Meghana Gupta
f458d9b490 Fix unnecessary one-time recompile of stdlib with -enable-ossa-flag (#39516)
* Fix unnecessary one-time recompile of stdlib with -enable-ossa-flag

This includes a bit in the module format to represent if the module was
compiled with -enable-ossa-modules flag. When compiling a client module
with -enable-ossa-modules flag, all dependent modules are checked for this bit,
if not on, recompilation is triggered with -enable-ossa-modules.

* Updated tests
2021-10-04 18:46:40 -07:00
Alex Hoppen
a7641a7cd2 [Lexer] Adjust tests for new delayed trivia lexing 2021-02-05 08:15:54 +01:00
Alexey Komnin
6a4c5cf061 Fix SR-13520: return string instead of address of local temporary 2020-09-22 19:33:43 +03:00
Owen Voorhees
45bc578ae5 [SourceManager] Rename line and column APIs for clarity 2020-05-21 12:54:07 -05:00
Hamish Knight
011f4f1584 Requestify SourceFile parsing
Add ParseSourceFileRequest that parses a SourceFile
for its top-level decls.
2020-03-03 15:53:18 -08:00
Hamish Knight
312f7ddc50 Parse Swift decls in one shot
Instead of interleaving typechecking and parsing
for SIL files, first parse the file for Swift
decls by skipping over any intermixed SIL decls.
Then we can perform type checking, and finally SIL
parsing where we now skip over Swift decls.

This is an intermediate step to requestifying the
parsing of a source file for its Swift decls.
2020-02-04 13:04:50 -08:00
Robert Widmann
48805b1d44 Give ASTContext TypeCheckerOptions
Strip TypeChecker of all of this state.
2019-11-12 09:56:01 -08:00
Owen Voorhees
8a6711769e [Diagnostics] Refactor DiagnosticConsumer interface
DiagnosticInfo now holds all the information needed to consume
a diagnostic, so remove unneeded parameters from handleDiagnostic.
2019-10-29 13:52:12 -07:00
swift-ci
7245d76875 Merge remote-tracking branch 'origin/master' into master-rebranch 2019-08-14 11:44:30 -07:00
Adrian Prantl
c08a62764a Move DWARFImporterDelegate into ClangImporter and remove DWARFImporter.
This refactors DWARFImporter to become a part of ClangImporter, since
it needs access to many of its implementation details anyway. The
DWARFImporterDelegate is just another mechanism for deserializing
Clang ASTs and once we have a Clang AST, the processing is effectively
the same.
2019-08-14 10:28:50 -07:00
Gwen Mittertreiner
67cfef2d60 SectionRef::getContents() now returns Expected
Updated uses of object::SectionRef::getContents() since it now returns
an Expected<StringRef> instead of modifying the one it's passed.

See also: git-svn-id:
https://llvm.org/svn/llvm-project/llvm/trunk@360892
91177308-0d34-0410-b5e6-96231b3b80d8
2019-05-20 16:02:41 -07:00
David Ungar
44daa88ebd Format 2019-04-03 12:53:31 -07:00
David Ungar
9cc3a4a9d8 Rename defaultDiagnosticLoc to bufferIndirectlyCausingDiagnostic. 2019-04-03 12:52:49 -07:00
David Ungar
c139c5909a Pass defaultDiagnosticLoc to handleDiagnostic, not currentPrimaryInput. 2019-04-02 22:27:55 -07:00
David Ungar
7a0e0ffc8a Store current primary in diagnostic engine, pass it down via handleDiagnostic. Unformmated. 2019-04-02 00:43:28 -07:00
Harlan Haskins
f5fc6f0c57 [Lexer] Handle SwiftInterface files as well as SIL
Previously, the Lexer kept a single flag whether we’re lexing Swift or SIL. Instead, keep track if we’re parsing Swift, SIL, or a Swiftinterface file. .swiftinterface files allow $-prefixed identifiers anywhere.
2019-01-22 11:02:36 -08:00
Argyrios Kyrtzidis
5b1aab1cc6 [unittests/Parse] Update unit tests to accomodate ParsedTrivia introduction 2019-01-17 13:47:29 -08:00
Adrian Prantl
d63debeb60 Experimental: Extend ClangImporter to import clang modules from DWARF
When debugging Objective-C or C++ code on Darwin, the debug info
collected by dsymutil in the .dSYM bundle is entirely
self-contained. It is possible to debug a program, set breakpoints and
print variables even without having the complete original source code
or a matching SDK available. With Swift, this is currently not the
case. Even though .dSYM bundles contain the binary .swiftmodule for
all Swift modules, any Clang modules that the Swift modules depend on,
still need to be imported from source to even get basic LLDB
functionality to work. If ClangImporter fails to import a Clang
module, effectively the entire Swift module depending on it gets
poisoned.

This patch is addressing this issue by introducing a ModuleLoader that
can ask queries about Clang Decls to LLDB, since LLDB knows how to
reconstruct Clang decls from DWARF and clang -gmodules producxes full
debug info for Clang modules that is embedded into the .dSYM budle.

This initial version does not contain any advanced functionality at
all, it merely produces an empty ModuleDecl. Intertestingly, even this
is a considerable improvement over the status quo. LLDB can now print
Swift-only variables in modules with failing Clang depenecies, and
becuase of fallback mechanisms that were implemented earlier, it can
even display the contents of pure Objective-C objects that are
imported into Swift. C structs obviously don't work yet.

rdar://problem/36032653
2018-12-05 13:54:13 -08:00
Rintaro Ishizaki
1f563b3716 [incrParse] Fix SyntaxParsingCache::translateToPreEditPosition()
If the position is in the region that is inserted by the edits,
'pre-edit' position shouldn't exist. So we cannot reuse the node at the
position.

rdar://problem/45259469
https://bugs.swift.org/browse/SR-8995
2018-11-06 00:27:19 +09:00
Alex Hoppen
63becb530b [incrParse] Fix bug in translation of post-edit to pre-edit positions 2018-10-11 17:54:17 +02:00
Jordan Rose
e180bf9d4e [Parse] Tweak a utility function that relies on reading past the end (#19230)
Lexer::getEncodedStringSegment (now getEncodedStringSegmentImpl)
assumes that it can read one byte past the end of a string segment in
order to avoid bounds-checks on things like "is this a \r\n
sequence?". However, the function was being used for strings that did
not come from source where this assumption was not always valid.
Change the reusable form of the function to always copy into a
temporary buffer, allowing the fast path to continue to be used for
normal parsing.

Caught by ASan!

rdar://problem/44306756
2018-09-11 20:11:58 -07:00
Xi Ge
24b0eac9a4 Parser: parse members in extension decls incrementally. 2018-09-05 17:00:39 -07:00
Jordan Rose
63cd1258ea Stop using SourceManager::getBufferIdentifierForLoc to find buffer IDs
The right way is findBufferContainingLoc. getBufferIdentifierForLoc is
both slower and wrong in the presence of #sourceLocation.

I couldn't come up with a test for the change in IDE/Utils.cpp because
refactoring still seems to be broken around #sourceLocation. I'll file
bugs for that.
2018-08-29 11:46:41 -07:00
Jordan Rose
fc9ea1e329 Add Lexer::IsHashbangAllowed, drop SourceManager::getHashbangBufferID (#18534)
Having this be a single buffer hardcoded in the SourceManager and set
by all clients is silly. SourceFiles with the 'Main' kind are allowed
to have hashbang lines (`#!`), other files are not. And anyone
manually setting up a Lexer can decide for themselves.

No intended behavioral change.
2018-08-07 08:25:05 -07:00
swift-ci
7cde895b95 Merge remote-tracking branch 'origin/master' into master-llvm-swift5-transition 2018-03-07 21:18:13 -08:00
omochimetaru
967a48cb77 [Parse] refactor Lexer initialization 2018-03-08 11:17:51 +09:00
omochimetaru
1588fb914b [Parse] test current buggy behavior 2018-03-08 11:17:51 +09:00
swift-ci
149a29cb4d Merge remote-tracking branch 'origin/master' into master-llvm-swift5-transition 2018-03-03 15:57:51 -08:00
omochimetaru
d12542503f [Syntax] test diagnostics in Lexer with libSyntax (#14954) 2018-03-04 08:53:54 +09:00
swift-ci
5712998273 Merge remote-tracking branch 'origin/master' into master-next 2017-12-27 19:29:23 -08:00
omochimetaru
ebd5323b42 [Syntax] add UTF-8 BOM support to libSyntax 2017-12-28 01:26:09 +09:00
omochimetaru
c8a6ef13da [Parse] add test about BOM + trivia 2017-12-27 18:06:18 +09:00
swift-ci
734f1b8fb1 Merge remote-tracking branch 'origin/master' into master-next 2017-12-22 00:09:03 -08:00