Apart from mildly speeding up indexing, this also keeps the compiler from running into issues with implementation-only imports that may not be present while we're trying to index. rdar://problem/52083709
