averello/swift-mirror
1
0
Fork 0
mirror of https://github.com/apple/swift.git synced 2025-12-14 20:36:38 +01:00
Code Issues Packages Projects Releases Wiki Activity

[docs] Add First Pull Request guide and Getting Started guide.

Browse Source 
This improves upon the existing documentation to provide a clearer end-to-end
workflow for new contributors and people who wish to build the toolchain
locally but do not intend to submit patches.

We also provide more directions for systematically utilizing our existing
documentation.
This commit is contained in:
Varun Gandhi
2020-05-23 17:32:49 -07:00
parent 9b01139f26
commit 3fe3500d98
7 changed files with 915 additions and 253 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
Brewfile
View File

@@ -1,2 +1,3 @@
brew "cmake"
brew "ninja"
brew "sccache"

270
README.md
View File

@@ -50,14 +50,9 @@ To learn more about the programming language, visit [swift.org](https://swift.or
- [Contributing to Swift](#contributing-to-swift)
- [Getting Started](#getting-started)
- [System Requirements](#system-requirements)
- [Getting Sources for Swift and Related Projects](#getting-sources-for-swift-and-related-projects)
- [Building Swift](#building-swift)
- [Swift Toolchains](#swift-toolchains)
- [Build Failures](#build-failures)
- [Testing Swift](#testing-swift)
- [Learning More](#learning-more)
- [Build Dependencies](#build-dependencies)
## Contributing to Swift
@@ -77,206 +72,15 @@ well. For more, see the [Code of Conduct](https://swift.org/community/#code-of-c
## Getting Started
These instructions give the most direct path to a working Swift development
environment. To build from source you will need about 2 GB of disk space for the
source code and up to 70 GB of disk space for the build artifacts with full
debugging. Depending on your machine, a clean build can take a few minutes to
several hours. Naturally, incremental builds are much faster.
If you are interested in:
- Contributing fixes and features to the compiler: See our
[How to Submit Your First Pull Request guide](/docs/HowToGuides/FirstPullRequest.md).
- Building the compiler as a one-off: See our [Getting Started guide][].
- Building a toolchain as a one-off: Follow the [Getting Started guide][]
up until the "Building the project" section. After that, follow the
instructions in the [Swift Toolchains](#swift-toolchains) section below.
Once you are able to build things successfully and have a compile-test-debug
loop going, check out the [development tips](docs/DevelopmentTips.md) for
better productivity while working on the compiler.
You can also skim [docs/README.md](/docs/README.md) to understand what
high-level documentation is available.
### System Requirements
macOS, Ubuntu Linux LTS, and the latest Ubuntu Linux release are currently
supported as host development operating systems.
Please make sure you use Python 2.x. Python 3.x is not supported currently.
#### macOS
To build for macOS, you need [Xcode 12 beta 3](https://developer.apple.com/xcode/resources/).
The required version of Xcode changes frequently, and is often a beta release.
Check this document or the host information on <https://ci.swift.org> for the
current required version.
Swift's build tooling is meant to support spaces in the paths passed to them,
but using spaces sometimes tickles bugs in Swift's build scripts or the tools
they rely on. For example, [SR-13441](https://bugs.swift.org/browse/SR-13441)
is caused by a space in the Xcode path used on macOS. If you see Swift's build
tooling misbehave due to a space in a path, please
[report the bug on the Swift bug tracker](https://swift.org/contributing/#reporting-bugs)
and then change the path to work around it.
You will also need [CMake](https://cmake.org) and [Ninja](https://ninja-build.org),
which can be installed via a package manager:
**[Homebrew](https://brew.sh/)**
brew install cmake ninja
You can also use [homebrew-bundle](https://github.com/Homebrew/homebrew-bundle)
from the root of this repository's working directory to install all of these
dependencies:
brew bundle
**[MacPorts](https://macports.org)**
sudo port install cmake ninja
Instructions for installing CMake and Ninja directly can be found [below](#build-dependencies).
#### Linux
For Ubuntu, you'll need the following development dependencies:
```
sudo apt-get install \
clang \
cmake \
git \
icu-devtools \
libcurl4-openssl-dev \
libedit-dev \
libicu-dev \
libncurses5-dev \
libpython-dev \
libsqlite3-dev \
libxml2-dev \
ninja-build \
pkg-config \
python \
python-six \
rsync \
swig \
systemtap-sdt-dev \
tzdata \
uuid-dev
```
**Note:** LLDB currently requires at least `swig-1.3.40` but will successfully build
with version 2 shipped with Ubuntu.
**Note:** For Ubuntu 20.04, use `libpython2-dev` in place of the libpython-dev package above.
### Getting Sources for Swift and Related Projects
First create a directory for all of the Swift sources:
mkdir swift-source
cd swift-source
**Note:** This is important since update-checkout (see below) checks out
repositories next to the Swift source directory. This means that if one clones
Swift and has other unrelated repositories, update-checkout may not clone those
repositories and will update them instead. Be aware that `update-checkout`
currently does not support paths with non-ASCII characters. If such characters
are present in the path to `swift-source`, `update-checkout` will fail.
**Via HTTPS** For those checking out sources as read-only, HTTPS works best:
git clone https://github.com/apple/swift.git
./swift/utils/update-checkout --clone
**Via SSH** For those who plan on regularly making direct commits,
cloning over SSH may provide a better experience (which requires
[uploading SSH keys to GitHub](https://help.github.com/articles/adding-a-new-ssh-key-to-your-github-account/)):
git clone git@github.com:apple/swift.git
./swift/utils/update-checkout --clone-with-ssh
### Building Swift
The `build-script` is a high-level build automation script that supports basic
options such as building a Swift-compatible LLDB, building the Swift Package
Manager, building for various platforms, running tests after builds, and more.
There are two primary build systems to use: Xcode and Ninja. The Xcode build
system allows you to work in Xcode, but Ninja is a bit faster and supports
more environments.
First, make sure that you're in the swift directory:
cd swift
To build using Ninja, run:
utils/build-script --release-debuginfo
When developing Swift, it helps to build what you're working on in a debug
configuration while building the rest of the project with optimizations. Below
are some examples of using debug variants:
utils/build-script --release-debuginfo --debug-swift # Swift frontend built in debug
utils/build-script --release-debuginfo --debug-swift-stdlib # Standard library built in debug
utils/build-script --release-debuginfo --debug-swift --force-optimized-typechecker # Swift frontend sans type checker built in debug
Limiting the amount of debug code in the compiler has a very large impact on
Swift compile times, and in turn the test execution time. If you want to build
the entire project in debug, you can run:
utils/build-script --debug
For documentation of all available arguments, as well as additional usage
information, see the inline help:
utils/build-script -h
#### Xcode
To build using Xcode, specify the `--xcode` argument on any of the above commands.
Xcode can be used to edit the Swift source code, but it is not currently
fully supported as a build environment for SDKs other than macOS. The generated
Xcode project does not integrate with the test runner, but the tests can be run
with the 'check-swift' target.
#### Build Products
All of the build products are placed in `swift-source/build/${TOOL}-${MODE}/${PRODUCT}-${PLATFORM}/`.
If macOS Swift with Ninja in DebugAssert mode was built, all of the products
would be in `swift-source/build/Ninja-DebugAssert/swift-macosx-x86_64/`. It
helps to save this directory as an environment variable for future use.
export SWIFT_BUILD_DIR="~/swift-source/build/Ninja-DebugAssert/swift-macosx-x86_64"
#### Ninja
Once the first build has completed, Ninja can perform fast incremental builds of
various products. These incremental builds are a big timesaver when developing
and debugging.
cd ${SWIFT_BUILD_DIR}
ninja swift-frontend
This will build the Swift compiler, but will not rebuild the standard library or
any other target. Building the `swift-stdlib` target as an additional layer of
testing from time to time is also a good idea. To build just the standard
library, run:
ninja swift-stdlib
It is always a good idea to do a full build after using `update-checkout`.
#### Using Xcode
To open the Swift project in Xcode, open `${SWIFT_BUILD_DIR}/Swift.xcodeproj`.
It will auto-create a *lot* of schemes for all of the available targets. A
common debug flow would involve:
- Select the 'swift-frontend' scheme.
- Pull up the scheme editor (⌘⇧<).
- Select the 'Arguments' tab and click the '+'.
- Add the command line options.
- Close the scheme editor.
- Build and run.
Another option is to change the scheme to "Wait for executable to be launched",
then run the build product in Terminal.
[Getting Started guide]: /docs/HowToGuides/GettingStarted.md
### Swift Toolchains
@@ -335,7 +139,12 @@ compiler crashes.
### Build Failures
Make sure you are using the [correct release](#macos) of Xcode.
Try the suggestions in
[Troubleshooting build issues](/docs/HowToGuides/GettingStarted.md#troubleshooting-build-issues).
Make sure you are using the
[correct release](/docs/HowToGuides/GettingStared.md#installing-dependencies)
of Xcode.
If you have changed Xcode versions but still encounter errors that appear to
be related to the Xcode version, try passing `--clean` to `build-script`.
@@ -343,18 +152,11 @@ be related to the Xcode version, try passing `--clean` to `build-script`.
When a new version of Xcode is released, you can update your build without
recompiling the entire project by passing the `--reconfigure` option.
Make sure all repositories are up to date with the `update-checkout` command
described above.
## Testing Swift
See [docs/Testing.md](docs/Testing.md), in particular the section on [lit.py](docs/Testing.md#using-litpy).
## Learning More
Be sure to look through the [docs](https://github.com/apple/swift/tree/master/docs)
directory for more information about the compiler. In particular, the documents
titled [Debugging the Swift Compiler](docs/DebuggingTheCompiler.md) and
Be sure to look at the [documentation index](/docs/README.md) for a bird's eye
view of the available documentation. In particular, the documents titled
[Debugging the Swift Compiler](docs/DebuggingTheCompiler.md) and
[Continuous Integration for Swift](docs/ContinuousIntegration.md) are very
helpful to understand before submitting your first PR.
@@ -378,41 +180,3 @@ Another source of documentation is the standard library itself, located in
`stdlib`. Much of the language is actually implemented in the library
(including `Int`), and the standard library gives some examples of what can be
expressed today.
## Build Dependencies
### CMake
[CMake](https://cmake.org) is the core infrastructure used to configure builds of
Swift and its companion projects; at least version 3.16.5 is required.
On macOS, you can download the [CMake Binary Distribution](https://cmake.org/download),
bundled as an application, copy it to `/Applications`, and add the embedded
command line tools to your `PATH`:
export PATH=/Applications/CMake.app/Contents/bin:$PATH
On Linux, if you have not already installed Swift's [development
dependencies](#linux), you can download and install the CMake
package separately using the following command:
sudo apt-get install cmake
### Ninja
[Ninja](https://ninja-build.org) is the current recommended build system
for building Swift and is the default configuration generated by CMake. [Pre-built
packages](https://github.com/ninja-build/ninja/wiki/Pre-built-Ninja-packages)
are available for macOS and Linux distributions. You can also clone Ninja
next to the other projects and it will be bootstrapped automatically:
**Via HTTPS**
git clone https://github.com/ninja-build/ninja.git && cd ninja
git checkout release
cat README
**Via SSH**
git clone git@github.com:ninja-build/ninja.git && cd ninja
git checkout release
cat README

BIN
docs/GitHubCreatePRScreenshot.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 62 KiB

101
docs/HowToGuides/FAQ.md Normal file
View File

@@ -0,0 +1,101 @@
# Frequently Asked Questions
## Build System and CMake Configuration
### How do I add a new file to CMake?
If you forget to add a new file to the CMake configuration, you may end up with
undefined symbol errors at link time.
There should be a `CMakeLists.txt` in the directory where you added the new
file, which has the list of different .h/.cpp files to be included in the build.
Add your new file to that list.
### How do I speed up iterating on changes to the build system?
The general idea is to build as little as you can.
- [Use `sccache`](/docs/HowToGuides/GettingStarted.md) or `ccache` if you
aren't doing so already.
- Use `build-script`'s various `--skip-*` flags to skip configuring for
platforms that you do not care about.
- If you're on macOS, use `--swift-darwin-supported-archs="x86_64"`.
- Use a release build without assertions (`--release --no-assertions`).
While debuginfo and assertions are valuable to enable when working on the
toolchain itself, they are not so useful if you are working only on changes
to the build system.
## Using a Locally Built Toolchain
### How do I use a locally built compiler to build X?
You can use the `SWIFT_EXEC` environment variable to use a locally
built compiler to compile both packages and Xcode projects.
1. For SwiftPM packages, pass the environment variable when invoking SwiftPM.
```sh
# Assuming the current working directory contains the package, build the
# package using a custom compiler.
SWIFT_EXEC=/path/to/swiftc swift build
```
2. For Xcode projects, select the project in the Project Navigator. In the
Build Settings tab, click '+' and then 'Add User-Defined Setting'.
Create a build setting `SWIFT_EXEC` with the value set to `/path/to/swiftc`.
If you now do a clean build, your locally built compiler will be used.
At the time of writing, in the latest Xcode 12 beta, `SWIFT_EXEC` does not
work for SwiftPM integration inside Xcode, so this will not work for Xcode
projects that depend on SwiftPM packages.
**Note:** Even thought the variable says 'SWIFT', it needs to point to
'swift**c**', not 'swift'. The extra 'c' is not a typo.
## Testing and CI
### How do I reproduce/fix a test that fails in CI but passes for me locally?
TODO: Write some tips here, point to Testing.md for simulator setup.
## Documentation
### Where can I find documentation on X?
This very depends on what X is, but some broad guidelines are:
1. Do a case-insensitive recursive string search.
- Use a specialized tool like [ripgrep](https://github.com/BurntSushi/ripgrep)
or [ag](https://github.com/ggreer/the_silver_searcher).
- Use 'Find in Workspace' in Xcode (<kbd>⌘</kbd>+<kbd>⇧</kbd>+<kbd>F</kbd>).
- Use `grep -i -r "mypattern" .`.
2. Go through the [Documentation Index](/docs/README.md).
## Pull Request Workflow
### How do I format my changes?
First, install `clang-format` using your system's package manager. This should
also install the `git-clang-format` script (try `git-clang-format --help`).
In case it doesn't, you can replace `git-clang-format` in the following
commands with `../llvm-project/clang/tools/clang-format/git-clang-format`.
Start out at the tip of the branch where you want to reformat the commits.
```
# If there is only one commit that needs to be reformatted.
git-clang-format HEAD~1
git add .
git commit --amend --no-edit
# Say the last N commits need to be reformatted.
# Mark them as 'edit' instead of 'pick'.
git rebase -i HEAD~N
# Re-run N times, reformatting each commit.
git-clang-format HEAD~1
git add .
git commit --amend --no-edit
git rebase --continue
```
### How do I clean up my git history?
TODO: Link to a beginner-friendly external resource, or (less preferably)
describe basic usage of rebase here.

179
docs/HowToGuides/FirstPullRequest.md Normal file
View File

@@ -0,0 +1,179 @@
# How to Submit Your First Pull Request
So you've decided to contribute to the Swift toolchain, welcome!
Maybe this is your first time contributing to an open source project, or maybe
you are an experienced open source contributor who is excited about Swift, or
maybe you are somewhere in-between. Regardless of your background, we are
excited to have you contribute and improve the developer experience for Swift
programmers all over the globe.
:sparkles::child::student::woman_technologist::technologist::man_technologist::sparkles:
This document provides a high-level overview of different parts of the
contribution process.
## How do I pick something to work on?
In case you don't have something specific you'd like to work on, such as
implementing something for a Swift Evolution pitch, you could start off by
working on a bug labeled `StarterBug` on [Swift JIRA][StarterBug]. If the
bug hasn't been assigned to someone, check the comments in case someone has
already started working on it. If not, feel free to assign it to yourself and
start working on it!
[StarterBug]: https://bugs.swift.org/issues/?jql=labels%20%3D%20StarterBug%20AND%20(status%20%3D%20Open%20OR%20status%20%3D%20Reopened)%20AND%20project%20%3D%20Swift
## Getting Help
Usually, Starter Bugs try to provide some instructions to help you get started.
In case those are missing, please ask the bug reporter for more detailed steps
and they will be happy to help.
Once you start working on the bug, you will inevitably end up having a lot of
questions. Don't be afraid to ask for help! The codebase is large and wrapping
your head around it will take time. For example, you might have questions like:
- Where can I find documentation on X?
- I'm seeing a cryptic error E when trying to build the compiler. How do I fix
it or work around it?
- I'm seeing very long build times even for incremental builds. How do I speed
up iteration time?
- I'm not sure how to implement X. Any suggestions on where I should start?
- What is the difference between types T1 and T2? They look very similar.
- Should I split my new X into a separate file?
- Should I create a new test file or update an existing test?
- How should I test that I actually fixed this bug?
- Test X is failing and I can't understand why. What might be going wrong here?
- Test X is failing in CI but passing locally. Any tips for debugging?
- I made some change but that seems to be not getting picked up. What should
I do to fix it?
- I need to update the CMake but I'm not familiar with CMake. Could you give me
more guidance?
- How do I do X in git?
Some of these already have answers in our [FAQ](/docs/HowToGuides/FAQ.md).
Maybe you have a question that's not on this list. That's fine.
We're here to help. There are a couple of options to ask for help:
- [Development category on the Swift forums](https://forums.swift.org/c/development):
Prefer using the forums for broad questions, such as those related to
building the toolchain, or understanding how something works at a high-level.
Since more people are likely to see and be able to answer your question, the
question is likely to get an answer sooner. Another benefit of asking in
public is that the answers you receive will be helpful to bystanders too.
- Bug report/Pull request comments: Prefer asking in the bug report/pull request
when the question involves additional context specific to the
bug report/pull request.
These are suggestions, not rules. For example, it's okay if you ask a broad
question in a bug report or a pull request.
When asking for help, prefer giving as much information as possible, while
highlighting the parts that you think are important.
Remember that the [Swift Code of Conduct][] applies whenever you are
participating in the Swift project.
[Swift Code of Conduct]: https://swift.org/community/#code-of-conduct
### I didn't get a response from someone. What should I do?
It's possible that you ask someone a question in a bug report/pull request and
you don't get a response as quickly as you'd like. Maybe they are juggling
several tasks and the discussion with you accidentally slipped by. Maybe they
are on vacation or on leave for some reason. If you don't get a response
within a week, it's okay to politely ping them using an `@` mention with a
reminder. If you don't get a response for 2-3 weeks in a row, please ping
someone else.
## Working on a change
Please see our [Getting Started guide][] to understand how to build the code,
make changes, run tests and debug issues.
[Getting Started guide]: /docs/HowToGuides/GettingStarted.md
## Submitting a pull request
### Tidying up
Alright! You've implemented a change and would like to submit it.
Double-check that you've tidied your Git history, such as squashing
work-in-progress commits, and that your commit messages provide context.
For example, if a commit fixes a bug, then include a "Fixes SR-NNNNN" with the
bug number in the commit message.
Next, [format your changes](/docs/HowToGuides/FAQ.md#how-do-i-format-my-changes)
using `clang-format`.
### Pushing and creating a pull request
Assuming you followed the steps in our [Getting Started guide][], you should now
be able to push your latest changes to GitHub using `git push`.
Next, [create a pull request][] (PR). Usually, if you navigate to
https://github.com/apple/swift right after pushing your change, GitHub will
show a helpful "Compare & Pull Request" button.
![Compare & Pull Request button in GitHub UI](/docs/GitHubCreatePRScreenshot.png)
[create a pull request]: https://docs.github.com/en/github/collaborating-with-issues-and-pull-requests/creating-a-pull-request#creating-the-pull-request
## Asking for code review
If you had an active discussion with someone on how to implement your change,
you can `@` mention them in the PR description and ask for code review.
If you directly implemented the change without any guidance from anyone else,
`@` mention someone from GitHub's suggested reviewers. If GitHub doesn't
make any suggestions, ask the [Code Owner](/CODE_OWNERS.txt) based on the
component for your change. Please ask someone though! We don't want you to get
stuck because you were not sure who to ask for code review.
At the beginning, contributors are not able to run the continuous integration
(CI) bot, which builds the project and runs tests. Please ask your code
reviewer(s) to invoke the bot for you.
## Responding to code review comments
During the process of code review, someone might suggest changes or have
questions about the implementation. If something is unclear, such as someone
using a technical term you don't recognize, check our
[Lexicon](/docs/Lexicon.md) or ask someone instead of trying to figure out
everything by yourself. Code review does not need to be a one-way
street. It is also a good opportunity for you to ask highly contextual
questions on topics that you struggled with or were unable to understand.
While making changes based on code review, if you are comfortable with
rebasing, prefer rebasing and force-pushing for small patches (say < 100 lines).
For larger patches, you can add fixup commits (`git commit --fixup ...`)
addressing the suggestions and rebase after it the patch has been approved
to clean up the history.
When you push again and want the tests to be re-run, please ask the reviewer
to invoke `swift-ci` for you.
At the end, once the tests are passing, the pull request is approved by
the reviewer, and you are satisfied with your changes, ask your reviewer
to merge your changes. :tada:
## I can't finish the contribution I started. :frowning_face:
That's totally okay! There is no shame in that. You only have limited time and
energy in a day. If you can, leave a comment on the bug report/pull request
that you will not be able to continue and unassign yourself from the bug on
JIRA. Don't worry about trying to explain _why_ you aren't able to contribute
further. We understand. Unanticipated things come up all the time and you
should do what _works for you_.
This point also applies if you don't have time right now but hope to get to
something in the near future. Please don't feel sad or apologetic!
## I submitted and merged my first pull request. What now?
Awesome! You could try fixing a few more Starter Bugs until you feel some
level of comfort working with the codebase. You could also start looking at
other bugs which interest you and you think you might be able to tackle.
Don't forget to ask for help if you need directions or you get stuck!
Once you've made multiple substantial contributions, you can
[ask for commit access](https://swift.org/contributing/#commit-access),
which will allow you to pick reviewers, trigger the CI bot and merge changes.

609
docs/HowToGuides/GettingStarted.md Normal file
View File

@@ -0,0 +1,609 @@
# How to Set Up an Edit-Build-Test-Debug Loop
This document describes how to set up a development loop for people interested
in contributing to Swift.
If you are only interested in building the
toolchain as a one-off, there are a couple of differences:
1. You can ignore the parts related to Sccache.
2. You can stop reading after
[Building the project for the first time](#building-the-project-for-the-first-time).
## Table of Contents
- [System Requirements](#system-requirements)
- [Cloning the project](#cloning-the-project)
- [Troubleshooting cloning issues](#troubleshooting-cloning-issues)
- [Installing dependencies](#installing-dependencies)
- [macOS](#macOS)
- [Ubuntu Linux](#ubuntu-linux)
- [Building the project for the first time](#building-the-project-for-the-first-time)
- [Spot check dependencies](#spot-check-dependencies)
- [Understanding the pieces](#understanding-the-pieces)
- [The actual build](#the-actual-build)
- [Troubleshooting build issues](#troubleshooting-build-issues)
- [Editing code](#editing-code)
- [Setting up your fork](#setting-up-your-fork)
- [First time Xcode setup](#first-time-xcode-setup)
- [Editing](#editing)
- [Incremental builds with Ninja](#incremental-builds-with-ninja)
- [Incremental builds with Xcode](#incremental-builds-with-xcode)
- [Spot checking an incremental build](#spot-checking-an-incremental-build)
- [Reproducing an issue](#reproducing-an-issue)
- [Running tests](#running-tests)
- [Debugging issues](#debugging-issues)
- [Print debugging](#print-debugging)
- [Debugging using LLDB](#debugging-using-lldb)
- [Next steps](#next-steps)
## System Requirements
1. Operating system:
The supported operating systems for developing the Swift toolchain are:
macOS, Ubuntu Linux LTS, and the latest Ubuntu Linux release.
At the moment, Windows is not supported as a host development operating
system. Experimental instructions for Windows are available under
[Windows.md](/docs/Windows.md).
2. Python 3: Several utility scripts are written in Python.
3. Disk space:
Make sure that you have enough available disk space before starting.
The source code, including full git history, requires about 3.5 GB.
Build artifacts take anywhere between 5 GB to 70 GB, depending on the
build settings.
4. Time:
Depending on your machine and build settings,
a from-scratch build can take a few minutes to several hours,
so you might want to grab a beverage while you follow the instructions.
Incremental builds are much faster.
## Cloning the project
1. Create a directory for the whole project:
```sh
mkdir -p swift-project/swift
cd swift-project/swift
```
2. Clone the sources:
- Via SSH (recommended):
If you plan on contributing regularly, cloning over SSH provides a better
experience. After you've [uploaded your SSH keys to GitHub][]:
```sh
git clone git@github.com:apple/swift.git .
utils/update-checkout --clone-with-ssh
```
- Via HTTPS:
If you want to check out the sources as read-only,
or are not familiar with setting up SSH,
you can use HTTPS instead:
```sh
git clone https://github.com/apple/swift.git .
utils/update-checkout --clone
```
**Note:** If you've already forked the project on GitHub at this stage,
**do not clone your fork** to start off. We describe
[how to setup your fork](#setting-up-your-fork) in a subsection below.
<!-- Recommending against cloning the fork due to SR-13476 and SR-13505. -->
3. Double-check that `swift`'s sibling directories are present.
```sh
ls ..
```
This should list directories like `llvm-project`, `swiftpm` and so on.
4. Checkout the right branch/tag:
If you are building the toolchain for local development, you can skip this
step, as Step 2 will checkout `swift`'s `master` branch and matching
branches for other projects. <!-- [TODO: master-to-main-rename] -->
If you are building the toolchain as a one-off, it is more likely that you
want a specific branch or a tag, often corresponding to a specific release
or a specific snapshot. You can update the branch/tag for all repositories
as follows:
```sh
utils/update-checkout --branch mybranchname
# OR
utils/update-checkout --tag mytagname
```
Detailed branching information, including names for release branches, can
be found in [Branches.md](/docs/Branches.md).
**Note:**
The commands used in the rest of this guide assumes that the absolute path
to your working directory is something like `/path/to/swift-project/swift`.
Double-check that running `pwd` prints a path ending with `swift`.
[uploaded your SSH keys to GitHub]: https://help.github.com/articles/adding-a-new-ssh-key-to-your-github-account/
### Troubleshooting cloning issues
- If `update-checkout` failed, double-check that the absolute path to your
working directory does not have non-ASCII characters.
- If `update-checkout` failed and the absolute path to your working directory
had spaces in it, please [file a bug report][Swift JIRA] and change the path
to work around it.
- Before running `update-checkout`, double-check that `swift` is the only
repository inside the `swift-project` directory. Otherwise,
`update-checkout` may not clone the necessary dependencies.
## Installing dependencies
### macOS
1. Install [Xcode 12 beta 3][Xcode] or newer:
The required version of Xcode changes frequently and is often a beta release.
Check this document or the host information on <https://ci.swift.org> for the
current required version.
2. Install [CMake][], [Ninja][] and [Sccache][]:
- Via [Homebrew][] (recommended):
```sh
brew install cmake ninja sccache
```
- Via [Homebrew Bundle][]:
```sh
brew bundle
```
[Xcode]: https://developer.apple.com/xcode/resources/
[CMake]: https://cmake.org
[Ninja]: https://ninja-build.org
[Sccache]: https://github.com/mozilla/sccache
[Homebrew]: https://brew.sh/
[Homebrew Bundle]: https://github.com/Homebrew/homebrew-bundle
### Ubuntu Linux
1. For Ubuntu 16.04 LTS and 18.04 LTS, run the following:
```sh
sudo apt-get install \
clang \
cmake \
git \
icu-devtools \
libcurl4-openssl-dev \
libedit-dev \
libicu-dev \
libncurses5-dev \
libpython3-dev \
libsqlite3-dev \
libxml2-dev \
ninja-build \
pkg-config \
python \
python-six \
rsync \
swig \
systemtap-sdt-dev \
tzdata \
uuid-dev
sudo snap install sccache --candidate --classic
```
**Note:** LLDB currently requires at least `swig-1.3.40` but will
successfully build with version 2 shipped with Ubuntu.
## Building the project for the first time
### Spot check dependencies
* Run `cmake --version`: This should be 3.18.1 or higher for macOS.
* Run `python3 --version`: Check that this succeeds.
* Run `ninja --version`: Check that this succeeds.
* Run `sccache --version`: Check that this succeeds.
### The roles of different tools
At this point, it is worthwhile to pause for a moment
to understand what the different tools do:
1. On macOS and Windows, IDEs (Xcode and Visual Studio resp.) serve as an
easy way to install development dependencies such as a C++ compiler,
a linker, header files, etc. The IDE's build system need not be used to
build Swift. On Linux, these dependencies are installed by the
distribution's package manager.
2. CMake is a cross-platform build system for C and C++.
It forms the core infrastructure used to _configure_ builds of
Swift and its companion projects.
3. Ninja is a low-level build system that can be used to _build_ the project,
as an alternative to Xcode's build system. Ninja is somewhat faster,
especially for incremental builds, and supports more build environments.
4. Sccache is a caching tool:
If you ever delete your build directory and rebuild from scratch
(i.e. do a "clean build"), Sccache can accelerate the new build
significantly. There are few things more satisfying than seeing Sccache
cut through build times.
5. `utils/update-checkout` is a script to help you work with all the individual
git repositories together, instead of manually cloning/updating each one.
6. `utils/build-script` (we will introduce this shortly)
is a high-level automation script that handles configuration (via CMake),
building (via Ninja or Xcode), caching (via Sccache), running tests and more.
> **Pro Tip**: Most tools support `--help` flags describing the options they
> support. Additionally, both Clang and the Swift compiler have hidden flags
> (`clang --help-hidden`/`swiftc --help-hidden`) and frontend flags
> (`clang -cc1 --help`/`swiftc -frontend --help`) and the Swift compiler
> even has hidden frontend flags (`swiftc -frontend --help-hidden`). Sneaky!
Phew, that's a lot to digest! Now let's proceed to the actual build itself!
### The actual build
1. Make sure you have Sccache running.
```sh
sccache --start-server
```
(Optional) Sccache defaults to a cache size of 10GB, which is relatively
small compared to build artifacts. You can bump it up, say by setting
`export SCCACHE_CACHE_SIZE="50G"` in your dotfile(s). For more details,
see the [Sccache README][Sccache].
2. Decide if you would like to build the toolchain using Ninja or using Xcode.
- If you use an editor other than Xcode and/or you want somewhat faster builds,
go with Ninja.
- If you are comfortable with using Xcode and would prefer to use it,
go with Xcode.
There is also a third option, which is somewhat more involved:
[using both Ninja and Xcode](#using-both-ninja-and-xcode).
3. Build the toolchain with optimizations, debuginfo, and assertions and run
the tests.
- Via Ninja:
```sh
utils/build-script --skip-build-benchmarks \
--skip-ios --skip-watchos --skip-tvos --swift-darwin-supported-archs "x86_64" \
--cmake-c-launcher="$(which sccache)" --cmake-cxx-launcher="$(which sccache)" \
--release-debuginfo --test
```
- Via Xcode:
```sh
utils/build-script --skip-build-benchmarks \
--skip-ios --skip-watchos --skip-tvos --swift-darwin-supported-archs "x86_64" \
--cmake-c-launcher="$(which sccache)" --cmake-cxx-launcher="$(which sccache)" \
--release-debuginfo --test \
--xcode
```
This will create a directory
`swift-project/build/Ninja-RelWithDebInfoAssert`
(with `Xcode` instead of `Ninja` if you used `--xcode`)
containing the build artifacts.
- If the build succeeds: Once the build is complete, the tests will run.
- If the tests are passing: Great! We can go to the next step.
- If some tests are failing:
- Consider [filing a bug report](https://swift.org/contributing/#reporting-bugs).
- Note down which tests are failing as a baseline. This baseline will be
handy later when you run the tests after making a change.
- If the build fails:
See [Troubleshooting build issues](#troubleshooting-build-issues).
In the following sections, for simplicity, we will assume that you are using a
`Ninja-RelWithDebInfoAssert` build on macOS running on an Intel-based Mac,
unless explicitly mentioned otherwise. You will need to slightly tweak the paths
for other build configurations.
#### Using both Ninja and Xcode
Some contributors find it more convenient to use both Ninja and Xcode.
Typically this configuration consists of:
1. A Ninja build created with `--release-debuginfo`.
2. An Xcode build created with `--release-debuginfo --debug-swift`.
The Ninja build can be used for fast incremental compilation and running tests
quickly. The Xcode build can be used for debugging with high fidelity.
The additional flexibility comes with two issues: (1) consuming much more disk
space and (2) you need to maintain the two builds in sync, which needs extra
care when moving across branches.
### Troubleshooting build issues
- Double-check that all projects are checked out at the right branches.
A common failure mode is using `git checkout` to change the branch only
for `swift` (often to a release branch), leading to an unsupported
configuration. See Step 4 of [Cloning the Project](#cloning-the-project)
on how to fix this.
- Double-check that all your dependencies
[meet the minimum required versions](#spot-check-dependencies).
- Check if there are spaces in the paths being used by `build-script` in
the log. While `build-script` should work with paths containing spaces,
sometimes bugs do slip through, such as
[SR-13441](https://bugs.swift.org/browse/SR-13441).
If this is the case, please [file a bug report][Swift JIRA] and change the path
to work around it.
- Check that your `build-script` invocation doesn't have typos. You can compare
the flags you passed against the supported flags listed by
`utils/build-script --help`.
- Check the error logs and see if there is something you can fix.
In many situations, there are several errors, so scrolling further back
and looking at the first error may be more helpful than simply looking
at the last error.
- Check if others have encountered the same issue on the Swift forums or on
[Swift JIRA][].
- Create a new Swift forums thread in the Development category. Include
information about your configuration and the errors you are seeing.
- You can [create a gist](https://gist.github.com) with the entire build
output and link it, while highlighting the most important part of the
build log in the post.
- Include the output of `utils/update-checkout --dump-hashes`.
[Swift JIRA]: https://bugs.swift.org
## Editing code
### Setting up your fork
If you are building the toolchain for development and submitting patches,
you will need to setup a GitHub fork.
First fork the `apple/swift` [repository](https://github.com/apple/swift.git),
using the "Fork" button in the web UI, near the top-right. This will create a
repository `username/swift` for your GitHub username. Next, add it as a remote:
```sh
# Using 'my-remote' as a placeholder name.
# If you set up SSH in step 2
git remote add my-remote git@github.com:username/swift.git
# If you used HTTPS in step 2
git remote add my-remote https://github.com/username/swift.git
```
Finally, create a new branch.
```sh
# Using 'my-branch' as a placeholder name
git checkout my-branch
git push --set-upstream my-remote my-branch
```
### First time Xcode setup
If you used `--xcode` earlier, you will see an Xcode project generated under
`../build/Xcode-RelWithDebInfoAssert/swift-macosx-x86_64`. When you open the
project, Xcode might helpfully suggest "Automatically Create Schemes". Most of
those schemes are not required in day-to-day work, so you can instead manually
select the following schemes:
- `swift-frontend`: If you will be working on the compiler.
- `check-swift-all`: This can be used to run the tests. The test runner does
not integrate with Xcode though, so it may be easier to run tests directly
on the commandline for more fine-grained control over which exact tests are
run.
<!-- TODO: Insert SourceKit/stdlib specific instructions? -->
### Editing
Make changes to the code as appropriate. Implement a shiny new feature!
Or fix a nasty bug! Update the documentation as you go! <!-- please 🙏 -->
The codebase is your oyster!
:construction::construction_worker::building_construction:
Now that you have made some changes, you will need to rebuild...
### Incremental rebuilds with Ninja
To rebuild the compiler:
```sh
ninja -C ../build/Ninja-RelWithDebInfoAssert/swift-macosx-x86_64 swift-frontend
```
To rebuild everything, including the standard library:
```sh
ninja -C ../build/Ninja-RelWithDebInfoAssert/swift-macosx-x86_64
```
### Incremental builds with Xcode
Rebuilding works the same way as with any other Xcode project; you can use
<kbd>⌘</kbd>+<kbd>B</kbd> or Product → Build.
### Spot checking an incremental build
As a quick test, go to `lib/Basic/Version.cpp` and tweak the version
printing code slightly. Next, do an incremental build as above. This incremental
build should be much faster than the from-scratch build at the beginning.
Now check if the version string has been updated:
```sh
../build/Ninja-RelWithDebInfoAssert/swift-macosx-x86_64/bin/swift-frontend --version
```
This should print your updated version string.
## Reproducing an issue
Starter bugs typically have small code examples that fit within a single file.
You can reproduce such an issue in various ways, such as compiling it from the
commandline using `/path/to/swiftc MyFile.swift`, pasting the code into
[Compiler Explorer][] (aka godbolt) or using an Xcode Playground.
[Compiler Explorer]: https://godbolt.org
For files using frameworks from an SDK bundled with Xcode, you need the pass
the SDK explicitly. Here are a couple of examples:
```sh
# Compile a file to an executable for your local machine.
xcrun -sdk macosx /path/to/swiftc MyFile.swift
# Say you are trying to compile a file importing an iOS-only framework.
xcrun -sdk iphoneos /path/to/swiftc -target arm64-apple-ios13.0 MyFile.swift
```
You can see the full list of `-sdk` options using `xcodebuild -showsdks`,
and check some potential `-target` options for different operating systems by
skimming the compiler's test suite under `test/`.
Sometimes bug reports come with SwiftPM packages or Xcode projects as minimal
reproducers. While we do not add packages or projects to the compiler's test
suite, it is generally helpful to first reproduce the issue in context before
trying to create a minimal self-contained test case. If that's the case with
the bug you're working on, check out our
[instructions on building packages and Xcode projects with a locally built compiler](/docs/HowToGuides/FAQ.md#how-do-i-use-a-locally-built-compiler-to-build-x).
## Running tests
There are two main ways to run tests:
1. `utils/run-test`: By default, `run-test` builds the tests' dependencies
before running them.
```sh
# Rebuild all test dependencies and run all tests under test/.
utils/run-test --lit ../llvm-project/llvm/utils/lit/lit.py \
../build/Ninja-RelWithDebInfoAssert/swift-macosx-x86_64/test-macosx-x86_64
# Rebuild all test dependencies and run tests containing "MyTest".
utils/run-test --lit ../llvm-project/llvm/utils/lit/lit.py \
../build/Ninja-RelWithDebInfoAssert/swift-macosx-x86_64/test-macosx-x86_64 \
--filter="MyTest"
```
2. `lit.py`: lit doesn't know anything about dependencies. It just runs tests.
```sh
# Run all tests under test/.
../llvm-project/llvm/utils/lit/lit.py -s -vv \
../build/Ninja-RelWithDebInfoAssert/swift-macosx-x86_64/test-macosx-x86_64
# Run tests containing "MyTest"
../llvm-project/llvm/utils/lit/lit.py -s -vv \
../build/Ninja-RelWithDebInfoAssert/swift-macosx-x86_64/test-macosx-x86_64 \
--filter="MyTest"
```
The `-s` and `-vv` flags print a progress bar and the executed commands
respectively.
If you making small changes to the compiler or some other component, you'll
likely want to [incrementally rebuild](#editing-code) only the relevant
Ninja/Xcode target and use `lit.py` with `--filter`. One potential failure
mode with this approach is accidental use of stale binaries. For example, say
that you want to rerun a SourceKit test but you only incrementally rebuilt the
compiler. Then your changes will not be reflected when the test runs because the
`sourcekitd` binary was not rebuilt. Using `run-test` instead is the safer
option, but it will lead to a longer feedback loop due to more things getting
rebuilt.
If you want to rerun all the tests, you can either rebuild the whole project
and use `lit.py` without `--filter` or use `run-test` to handle both aspects.
Recall the baseline failures mentioned in
[the build section](#the-actual-build). If your baseline had failing tests, make
sure you compare the failures seen after your changes to the baseline. If some
test failures look totally unrelated to your changes, there is a good chance
that they were already failing as part of the baseline.
For more details on running tests and understanding the various Swift-specific
lit customizations, see [Testing.md](/docs/Testing.md). Also check out the
[lit documentation](https://llvm.org/docs/CommandGuide/lit.html) to understand
how the different lit commands work.
## Debugging issues
In this section, we briefly describe two common ways of debugging: print
debugging and using LLDB.
Depending on the code you're interested in, LLDB may be significantly more
effective when using a debug build. Depending on what components you are
working on, you could turn off optimizations for only a few things.
Here are some example invocations:
```sh
# optimized Stdlib + debug Swiftc + optimized Clang/LLVM
utils/build-script --release-debuginfo --debug-swift # other flags...
# debug Stdlib + optimized Swiftc + optimized Clang/LLVM
utils/build-script --release-debuginfo --debug-swift-stdlib # other flags...
# optimized Stdlib + debug Swiftc (expect typechecker) + optimized Clang/LLVM
utils/build-script --release-debuginfo --debug-swift --force-optimized-typechecker
# Last resort option, it is highly unlikely that you will need this
# debug Stdlib + debug Swiftc + debug Clang/LLVM
utils/build-script --debug # other flags...
```
Debug builds have two major drawbacks:
- A debug compiler is much slower, leading to longer feedback loops in case you
need to repeatedly compile the Swift standard library and/or run a large
number of tests.
- The build artifacts consume a lot more disk space.
[DebuggingTheCompiler.md](/docs/DebuggingTheCompiler.md) goes into a LOT
more detail on how you can level up your debugging skills! Make sure you check
it out in case you're trying to debug a tricky issue and aren't sure how to
go about it.
### Print debugging
A large number of types have `dump(..)`/`print(..)` methods which can be used
along with `llvm::errs()` or other LLVM streams. For example, if you have a
variable `std::vector<CanType> canTypes` that you want to print, you could do:
```cpp
auto &e = llvm::errs();
e << "canTypes = [";
llvm::interleaveComma(canTypes, e, [&](auto ty) { ty.dump(e); });
e << "]\n";
```
You can also crash the compiler using `assert`/`llvm_unreachable`/
`llvm::report_fatal_error`, after accumulating the result in a stream:
```cpp
std::string msg; llvm::raw_string_ostream os(msg);
os << "unexpected canTypes = [";
llvm::interleaveComma(canTypes, os, [&](auto ty) { ty.dump(os); });
os << "] !!!\n";
llvm::report_fatal_error(os.str());
```
### Debugging using LLDB
When the compiler crashes, the commandline arguments passed to it will be
printed to stderr. It will likely look something like:
```
/path/to/swift-frontend <args>
```
- Using LLDB on the commandline: Copy the entire invocation and pass it to LLDB.
```sh
lldb -- /path/to/swift-frontend <args>
```
Now you can use the usual LLDB commands like `run`, `breakpoint set` and so
on. If you are new to LLDB, check out the [official LLDB documentation][] and
[nesono's LLDB cheat sheet][].
- Using LLDB within Xcode:
Select the current scheme 'swift-frontend' → Edit Scheme → Run phase →
Arguments tab. Under "Arguments Passed on Launch", copy-paste the `<args>`
and make sure that "Expand Variables Based On" is set to swift-frontend.
Close the scheme editor. If you now run the compiler
(<kbd>⌘</kbd>+<kbd>R</kbd> or Product → Run), you will be able to use the
Xcode debugger.
Xcode also has the ability to attach to and debug Swift processes launched
elsewhere. Under Debug → Attach to Process by PID or name..., you can enter
a compiler process's PID or name (`swift-frontend`) to debug a compiler
instance invoked elsewhere. This can be helpful if you have a single compiler
process being invoked by another tool, such as SwiftPM or another open Xcode
project.
> **Pro Tip**: Xcode 12's terminal does not support colors, so you may see
> explicit color codes printed by `dump()` methods on various types. To avoid
> color codes in dumped output, run `expr llvm::errs().enable_color(false)`.
[official LLDB documentation]: https://lldb.llvm.org
[nesono's LLDB cheat sheet]: https://www.nesono.com/sites/default/files/lldb%20cheat%20sheet.pdf
## Next steps
Make sure you check out the following resources:
* [LLVM Coding Standards](https://llvm.org/docs/CodingStandards.html): A style
guide followed by both LLVM and Swift. If there is a mismatch between the LLVM
Coding Standards and the surrounding code that you are editing, please match
the style of existing code.
* [LLVM Programmer's Manual](https://llvm.org/docs/ProgrammersManual.html):
A guide describing common programming idioms and data types used by LLVM and
Swift.
* [docs/README.md](/docs/README.md): Provides a bird's eye view of the available
documentation.
* [Lexicon.md](/docs/Lexicon.md): Provides definitions for jargon. If you run
into a term frequently that you don't recognize, it's likely that this file
has a definition for it.
* [Testing.md](/docs/Testing.md) and
[DebuggingTheCompiler.md](/docs/DebuggingTheCompiler.md): These cover more
ground on testing and debugging respectively.
* [Development Tips](/docs/DevelopmentTips.md): Tips for being more productive.
<!-- Link to Compiler Architecture.md once that is ready -->
If you see mistakes in the documentation (including typos, not just major
errors) or identify gaps that you could potentially improve the contributing
experience, please start a discussion on the forums, submit a pull request
or file a bug report on [Swift JIRA][]. Thanks!

8
docs/README.md
View File

@@ -55,6 +55,14 @@ documentation, please create a thread on the Swift forums under the
## How-To Guides
- [FAQ.md](/docs/HowToGuides/FAQ.md):
Answers "How do I do X?" for a variety of common tasks.
- [FirstPullRequest.md](/docs/HowToGuides/FirstPullRequest.md):
Describes how to submit your first pull request. This is the place to start
if you're new to the project!
- [GettingStarted.md](/docs/HowToGuides/GettingStarted.md):
Describes how to set up a working Swift development environment
for Linux and macOS, and get an edit-build-test-debug loop going.
- [DebuggingTheCompiler.md](/docs/DebuggingTheCompiler.md):
Describes a variety of techniques for debugging.
- Building for Android: