Due to the custom build system implemented in CMake for Swift, we cannot
properly detect the target and set flags appropriately. Instead, assume
that if the primary variant is an Apple target, that all targets are
Apple variants. This fixes cross-compilation on macOS.