When looking for a Swift module on disk, we were scanning all module search paths if they contain the module we are searching for. In a setup where each module is contained in its own framework search path, this scaled quadratically with the number of modules being imported. E.g. a setup with 100 modules being imported form 100 module search paths could cause on the order of 10,000 checks of `FileSystem::exists`. While these checks are fairly fast (~10µs), they add up to ~100ms.
To improve this, perform a first scan of all module search paths and list the files they contain. From this, create a lookup map that maps filenames to the search paths they can be found in. E.g. for
```
searchPath1/
Module1.framework
searchPath2/
Module1.framework
Module2.swiftmodule
```
we create the following lookup table
```
Module1.framework -> [searchPath1, searchPath2]
Module2.swiftmodule -> [searchPath2]
```
The effect of passing -enable-copy-propagation is both to enable the
CopyPropagation pass to shorten object lifetimes and also to enable
lexical lifetimes to ensure that object lifetimes aren't shortened while
a variable is still in scope and used.
Add a new flag, -enable-lexical-borrow-scopes=true to override
-enable-copy-propagation's effect (setting it to ::ExperimentalLate) on
SILOptions::LexicalLifetimes that sets it to ::Early even in the face of
-enable-copy-propagation. The old flag -disable-lexical-lifetimes is
renamed to -enable-lexical-borrow-scopes=false but continues to set that
option to ::Off even when -enable-copy-propagation is passed.
Previously, both swift-frontend and sil-opt put lexical lifetimes behind
a flag named -enable-experimental-lexical-lifetimes. That's redundant.
Here, the experimental portion of the name is dropped.
We noticed some Swift clients rely on the serialized search paths in the module to
find dependencies and droping these paths altogether can lead to build failures like
rdar://85840921.
This change teaches the serialization to obfuscate the search paths and the deserialization
to recover them. This allows clients to keep accessing these paths without exposing
them when shipping the module to other users.
We've recently added the -experimental-hermetic-seal-at-link compiler flag,
which turns on aggressive dead-stripping optimizations and assumes that library
code can be optimized against client code because all users of the library
code/types are present at link/LTO time. This means that any module that's
built with -experimental-hermetic-seal-at-link requires all clients of this
module to also use -experimental-hermetic-seal-at-link. This PR enforces that
by storing a bit in the serialized module, and checking the bit when importing
modules.
These modules are part of the experimental declarative string processing feature. If accepted to the Standard Library, _StringProcessing will be available via implicit import just like _Concurrency, though _MatchingEngine will still be hidden as an implementation detail.
`_MatchingEngine` will contain the general-purpose pattern matching engine ISA, bytecode, and executor. `_StringProcessing` will contain regular expression and pattern matching APIs whose implementation depends on the matching engine..
Also consolidates frontend flag `-enable-experimental-regex` as `-enable-experimental-string-processing`.
Resolves rdar://85478647.
The reason why I am doing this is that we are going to be enabling lexical
lifetimes early in the pipeline so that I can use it for the move operator's
diagnostics.
To make it easy for passes to know whether or not they should support lexical
lifetimes, I included a query on SILOptions called
supportsLexicalLifetimes. This will return true if the pass (given the passed in
option) should insert the lexical lifetime flag. This ensures that passes that
run in both pipelines (e.x.: AllocBoxToStack) know whether or not to set the
lexical lifetime flag without having to locally reason about it.
This is just chopping off layers of a larger patch I am upstreaming.
NOTE: This is technically NFC since it leaves the default alone of not inserting
lexical lifetimes at all.
With PE/COFF, one cannot reference a data symbol directly across the
binary module boundary. Instead, the reference must be indirected
through the Import Address Table (IAT) to allow for position
independence.
When generating a reference to a AsyncFunctionPointer ({i8*, i32}), we
tag the pointer as being indirected by tagging bit 1 (with the
assumption that native alignment will ensure 4/8 byte alignment, freeing
the bottom 2 bits at least for bit-packing). We tweak the
v-table/witness table emission such that all references to the
AsyncFunctionPointer are replaced with the linker synthetic import
symbol with the bit packing:
~~~
.quad __imp_$s1L1CC1yyYaKFTu+1
~~~
rather than
~~~
.quad $s1L1CC1yyYaKFTu
~~~
Upon access of the async function pointer reference, we open-code the
check for the following:
~~~
pointer = (pointer & 1) ? *(void **)(pointer & ~1) : pointer;
~~~
Thanks to @DougGregor for the discussion and the suggestion for the
pointer tagging. Thanks to @aschwaighofer for pointers to the code that
I had missed. Also, thanks to @SeanROlszewski for the original code
sample that led to the reduced test case.
Fixes: SR-15399
These include _move and @_noImplicitCopy. I still need to wire up the parsing of
those behind this feature.
The reason that I am adding this now is that I am going to now need to make some
changes behind a feature flag and I have not yet needed to add one. The specific
reason I needed to add one here is to ensure that I properly guard inside _move
the call to Builtin.move so as to prevent a "cond_fail" incident.
P.S.: This work depends on experimental lexical lifetimes being enabled as well,
so I did that at the same time in this PR.
Introduce a few changes to the logic for watchOS concurrency
back-deployment with respect to the async frame pointer.
* Only apply the change to watchOS device targets, not simulator targets
* Only introduce the override when no specific
`-swift-async-frame-pointer=<value>` option is provided on the
command line
* Only override the default when deploying to watchOS < 8, and
* Use "never" for the default rather than "always".
This represents a different but safer trade-off than before. Setting
the async bit in the frame pointer can cause older APIs (such as
backtrace APIs in the OS) to crash when they encounter such frame
pointers. So, with this change we never set the bit when back-deploying
for watchOS device, to avoid said crashes.
The trade-off here is that a back-deployed watchOS app will never have
the async frame pointer bit set, so async backtraces will be
unavailable even when running on watchOS 8 or newer.
The asm definition of `swift_async_extendedFramePointerFlags` prevents
the use of bitcode with the back-deployment libraries, so remove the
definition and use of this symbol from watchOS binaries entirely.
Instead, always force the async frame bit to be set. This trades off
backtraces on older OS's for debuggability of newer ones. If it causes
problems, it can be disabled via the option
`-swift-async-frame-pointer=never`.
Fixes rdar://84687579.
Previously, the flag was a LangOptioins. That didn't make much sense because
this isn't really a user-facing behavior. More importantly, as a member
of that type type it couldn't be accessed when setting up pass
pipelines. Here, the flag is moved to SILOptions.
Intro the frontend flag `-check-api-availability-only` that limits
availability checking to the API and SPI. This mode doesn't check the
availability of non-inlinable function bodies and of non-ABI-public decl
signatures.
This mode goal is to check all that is printed in the swiftinterface
file. It should be used in place of the wider
`-disable-availability-checking` when generating an interface for
platforms with no binaries.
rdar://81679692
Introduce a compiler flag that warnings about any public types defined in
a module that are neither explicitly `Sendable` nor explicitly
non-`Sendable` (the latter of which has no spelling currently), which
is intended to help with auditing a module for Sendable conformances.
* Fix unnecessary one-time recompile of stdlib with -enable-ossa-flag
This includes a bit in the module format to represent if the module was
compiled with -enable-ossa-modules flag. When compiling a client module
with -enable-ossa-modules flag, all dependent modules are checked for this bit,
if not on, recompilation is triggered with -enable-ossa-modules.
* Updated tests
This enables optimizing / dead-stripping of witness methods across modules at
LTO time.
- Under -internalize-at-link, restrict visibility of wtables to linkage unit.
- Emit thunks for cross-module wcalls when WME is enabled.
- Use thunks for wcalls across modules when WME is enabled.
- Adjust TBDGen to account for witness method thunks when WME is enabled.
- Add an IR test to check that thunks are used when doing cross-module calls.
- Add an end-to-end test case for cross-module WME.
A new LLVM IR affordance that allows expressing conditions under which globals
can be removed/dropped (even when marked with @llvm.used) is being discussed at:
- <https://reviews.llvm.org/D104496>
- <https://lists.llvm.org/pipermail/llvm-dev/2021-September/152656.html>
This is a preliminary implementation that marks runtime lookup records (namely
protocol records, type descriptors records and protocol conformance records)
with the !llvm.used.conditional descriptors. That allows link-time / LTO-time
removal of these records (by GlobalDCE) based on whether they're actually used
within the linkage unit. Effectively, this allows libraries that have a limited
and known set of clients, to be optimized against the client at LTO time, and
significantly reduce the code size of that library.
Parts of the implementation:
- New -conditional-runtime-records frontend flag to enable using !llvm.used.conditional
- IRGen code that emits these records can now emit these either as a single contiguous
array (asContiguousArray = true, the old way), which is used for JIT mode, or
as indivial globals (asContiguousArray = false), which is necessary for the
!llvm.used.conditional stripping to work.
- When records are emitted as individual globals, they have new names of
"\01l_protocol_" + mangled name of the protocol descriptor, and similarly for
other records.
- Fixed existing tests to account for individual records instead of a single array
- Added an IR level test, and an end-to-end execution test to demonstrate that
the !llvm.used.conditional-based stripping actually works.
"add inits to toplevel" and "call pattern heuristics" are only used in
code completion. Move them from LangOptions to CodeCompletionContext so
that they don't affect compiler arguments.
Added ForceStructTypeLayouts. When enabled, IRGen will lower structs using the
aligned group of TypeLayout rather than using TypeInfos. This potentially leads
to a size increase as TypeInfos currently produce better code than the
TypeLayout route.
- Under -internalize-at-link, stop unconditionally marking all globals as used.
- Under -internalize-at-link, restrict visibility of vtables to linkage unit.
- Emit virtual method thunks for cross-module vcalls when VFE is enabled.
- Use thunks for vcalls across modules when VFE is enabled.
- Adjust TBDGen to account for virtual method thunks when VFE is enabled.
- Add an end-to-end test case for cross-module VFE.
