Iterating child tasks depends on knowing the size of AsyncTask, and changing the size of the task broke it. Instead of relying on mirroring the full structure in our out-of-process definitions, add a debug variable to libswift_Concurrency that contains the size of AsyncTask.
While we're there, add some more validation to child task enumeration. Check each child task's metadata pointer to make sure that it actually points to the AsyncTask metadata, and have the inner loop also increment and check ChildTaskLoopCount to stop runaway iteration in that loop.
rdar://148836760
(cherry picked from commit e3057031da)
Adding `SWIFT_ENABLE_SWIFT_IN_SWIFT` option to enable or disable the
parts of Swift that require a Swift compiler to build. This is meant for
bootstrapping compilers on new platforms and is not guaranteed to result
in a compiler that will pass the test suite.
This option is on by default so that folks won't forget.
If the option is off, the resulting compiler does not include the Swift
optimizer sources in SwiftCompilerSources nor does the resulting
compiler have swift macro support.
We're using C++ 14+ at least now across the project so these should not be necessary. It's also kind of wild that the definitions we had across different subsystems had different definitions.
**Background:**
Each RemoteMirror test runs two processes: A host process runs the
RemoteMirror library and queries memory data from a target process. The host
and target communicate by passing strings back and forth with requests to read
particular information in memory. The target is pure Swift; the host is
C/C++.
**What this change does:**
Without this, the host makes very many small requests to the target.
Each of those requests has to be individually parsed by the target
using String operations from the debug stdlib. Actually transferring
the raw bytes is relatively quick.
With this, the host requests and caches "pages" of about 4k and
satisfies most requests from previously-fetched data. This dramatically
reduces the total number of operations.
**Performance notes:**
The following notes only count the time to compile and run the 78 tests in the
validation-tests/Reflection directory using `lit.py`; it does not include time
to rebuild the project before running tests.
**Performance with debug stdlib:**
(That is: `build-script -ra --debug-swift-stdlib`) Before this change, I got
tired of waiting after about 15 minutes and it was about 1/4 done. Some very
simplistic profiling showed >99% of the time being spent in stdlib String
operations in the target process. Activity Monitor shows that individual tests
run for about 6 minutes of CPU time each.
With this change, the tests run in about 70s of wall time. Almost all of the
time seems to spent compiling the tests; the tests themselves run too quickly to
even show up in the Activity Monitor.
**Performance with release stdlib:**
(That is: `build-script -ra`)
The tests run in about 70s of wall time with or without this change.
Mostly this just means adding `Musl` as a module dependency of various
things and making sure that we build things for `swift_static` even
if `SWIFT_BUILD_STATIC_STDLIB` isn't enabled.
There's also a slight difference in the declaration of `memcmp()`;
musl's declaration is more like the one we have on Darwin.
rdar://123508245
This change adds support for WASI in stdlib tests. Some tests that expect a crash to happen had to be disabled, since there's currently no way to observe such crash from a WASI host.
We remove the existing `swift_reflection_iterateAsyncTaskAllocations` API that attempts to provide all necessary information about a tasks's allocations starting from the task. Instead, we split it into two pieces: `swift_reflection_asyncTaskSlabPointer` to get the first slab for a task, and `+swift_reflection_asyncTaskSlabAllocations` to get the allocations in a slab, and a pointer to the next slab.
We also add a dummy metadata pointer to the beginning of each slab. This allows tools to identify slab allocations on the heap without needing to locate every single async task object. They can then use `swift_reflection_asyncTaskSlabAllocations` on such allocations to find out about the contents.
rdar://82549631
We remove the existing `swift_reflection_iterateAsyncTaskAllocations` API that attempts to provide all necessary information about a tasks's allocations starting from the task. Instead, we split it into two pieces: `swift_reflection_asyncTaskSlabPointer` to get the first slab for a task, and `+swift_reflection_asyncTaskSlabAllocations` to get the allocations in a slab, and a pointer to the next slab.
We also add a dummy metadata pointer to the beginning of each slab. This allows tools to identify slab allocations on the heap without needing to locate every single async task object. They can then use `swift_reflection_asyncTaskSlabAllocations` on such allocations to find out about the contents.
rdar://82549631
Previously, we used SWIFT_INCLUDE_TESTS to control if we created build rules for
binaries just used to test as well as running the tests as well. In this commit,
I changed this behavior by adding an option called SWIFT_INCLUDE_TEST_BINARIES.
This will allow for the stage 1 swift build to build and install test binaries
into the just built toolchain for use by the stage 2 swift that builds the
stdlib.
Implement a version of projectExistential tailored for LLDB. There are 2
differences when projecting existentials for LLDB:
1 - When it comes to existentials, LLDB stores the address of the error
pointer, which must be dereferenced.
2 - When the existential wraps a class type, LLDB expects the address
returned is the class instance itself and not the address of the
reference.
This patch also adapts the swift reflection test machinery to test
projectExistentialAndUnwrapClass as well. This is done by exposing
the new functionality from swift reflection test. It is tested in
existentials.swift, and ensures that the typeref information is
exactly the same as what is expected from projectExistential,
except the out address.
(cherry picked from commit 55e971e06750c3ba29722d558cc5400298f6bdaf)
If execv fails, then there is no valid indication of what failed. This
may manifest itself as an unexpected EOF in `collectBytesFromPipe` when
in fact this is a problem with `execv`. Check the return status and be
noisy if the exec fails.
Now, on other platforms, it may not be a runtime error to call
`execv(..., NULL)`, despite the manual pages and standards requesting
that a valid array be passed. Furthermore, other platforms may require
that argv[0] be populated, and with a valid executable name.
Ensuring that these conditions are fulfilled is more correct, and
permits the Reflection validation tests to successfully run and pass on
OpenBSD.
This code rearchitects and simplifies the projectEnumValue support by
introducing a new `TypeInfo` subclass for each kind of enum, including trivial,
no-payload, single-payload, and three different classes for multi-payload enums:
* "UnsupportedEnum" that we don't understand. This returns "don't know" answers for all requests in cases where the runtime lacks enough information to accurately handle a particular enum.
* MP Enums that only use a separate tag value. This includes generic enums and other dynamic layouts, as well as enums whose payloads have no spare bits.
* MP Enums that use spare bits, possibly in addition to a separate tag. This logic can only be used, of course, if we can in fact compute a spare bit mask that agrees with the compiler.
The final challenge is to choose one of the above three handlings for every MPE. Currently, we do not have an accurate source of information for the spare bit mask, so we never choose the third option above. We use the second option for dynamic MPE layouts (including generics) and the first for everything else.
TODO: Once we can arrange for the compiler to expose spare bit mask data, we'll be able to use that to drive more MPE cases.
Teach RemoteMirror how to project enum values
This adds two new functions to the SwiftRemoteMirror
facility that support inspecting enum values.
Currently, these support non-payload enums and
single-payload enums, including nested enums and
payloads with struct, tuple, and reference payloads.
In particular, it handles nested `Optional` types.
TODO: Multi-payload enums use different strategies for
encoding the cases that aren't yet supported by this
code.
Note: This relies on information from dataLayoutQuery
to correctly decode invalid pointer values that are used
to encode enums. Existing clients will need to augment
their DLQ functions before using these new APIs.
Resolves rdar://59961527
```
/// Projects the value of an enum.
///
/// Takes the address and typeref for an enum and determines the
/// index of the currently-selected case within the enum.
///
/// Returns true iff the enum case could be successfully determined.
/// In particular, note that this code may fail for valid in-memory data
/// if the compiler is using a strategy we do not yet understand.
SWIFT_REMOTE_MIRROR_LINKAGE
int swift_reflection_projectEnumValue(SwiftReflectionContextRef ContextRef,
swift_addr_t EnumAddress,
swift_typeref_t EnumTypeRef,
uint64_t *CaseIndex);
/// Finds information about a particular enum case.
///
/// Given an enum typeref and index of a case, returns:
/// * Typeref of the associated payload or zero if there is no payload
/// * Name of the case if known.
///
/// The Name points to a freshly-allocated C string on the heap. You
/// are responsible for freeing the string (via `free()`) when you are finished.
SWIFT_REMOTE_MIRROR_LINKAGE
int swift_reflection_getEnumCaseTypeRef(SwiftReflectionContextRef ContextRef,
swift_typeref_t EnumTypeRef,
unsigned CaseIndex,
char **CaseName,
swift_typeref_t *PayloadTypeRef);
```
Co-authored-by: Mike Ash <mikeash@apple.com>
This reverts commit beb8ecd8cc. Add a
workaround for the dependency issue.
It is unclear why `sourcekitd` is getting added improperly as a
dependency on `lib/sourcekitd.framework/sourcekitd`. This workaround
adjusts the dependency such that we end up with a dependency on
`lib/sourcekitd.framework/Versions/A/sourcekitd` as an order-only
dependency. This should fix the compile issue. I am unable to
reproduce this issue with the `add_library` usage for adding a Swift
library. This allows us to cleave the host and target libraries, and so
the workaround is sufficient to make progress and the problem will be
resolved with the migration towards CMake for handling the dependencies.
The link dependency is not sufficient, because this code depends on the
sourcekitd header and module map, which are produced during the build
process. This started failing after edbe22b63c, where the dependency
on `sourcekitd-test` was removed. That dependency was itself wrong,
but it happened to be a good enough approximation that we never saw
issues.
This moves the swift-lang library out of the tools/SourceKit directory
and into the stdlib directory. As it stands, the library required that
the standard library and the SDK overlay was being built. This is
implicitly done when building the stdlib subdirectory. Furthermore, it
had an odd tie between the flags for Swift and SourceKit which now have
the logic separated. This clearly delineates the host and target
components in the repository.
PE/COFF does not permit weak linking semantics. This means that the
expression `&symbol == NULL` is tautologically false. In order to
support this semantic, the symbol needs to be dynamically looked up at
runtime. Since the ObjC runtime is not in use on Windows currently,
just ignore this for the time being. Silences a warning from clang
about the tautological comparison.
Also have swift-reflection-test check if the symbol exists. This allows swift-reflection-test to work with older Remote Mirror dylibs that don't have it.
rdar://problem/50030805
This was used for the swift-reflection-test tool. Instead of using fat
binaries, use the target binary itself. This simplifies the build logic
as well as paves the road to sub-builds for each target rather than a
monolithic build as we have today.
Originally, the swift-reflection-test was a host tool that linked
against the target libraries since it was testing the target layout.
Now that the tool has been split into a host and target component
(5ea5bb06a3) and we have target and host
libraries that we can link against appropriately, we do not need to link
against the FAT binary. Since there was exactly one use of this
functionality, switching that from fat linking to regular linking allows
us to remove this functionality entirely. Switch to regular linking and
remove the option.
Recent Swift uses 2 as the is-Swift bit when running on newer versions, and 1 on older versions. Since it's difficult or impossible to know what we'll be running on at build time, make the selection at runtime.
