Implement span quoting for proc-macros

This PR implements span quoting, allowing proc-macros to produce spans
pointing *into their own crate*. This is used by the unstable
`proc_macro::quote!` macro, allowing us to get error messages like this:

```
error[E0412]: cannot find type `MissingType` in this scope
  --> $DIR/auxiliary/span-from-proc-macro.rs:37:20
   |
LL | pub fn error_from_attribute(_args: TokenStream, _input: TokenStream) -> TokenStream {
   | ----------------------------------------------------------------------------------- in this expansion of procedural macro `#[error_from_attribute]`
...
LL |             field: MissingType
   |                    ^^^^^^^^^^^ not found in this scope
   |
  ::: $DIR/span-from-proc-macro.rs:8:1
   |
LL | #[error_from_attribute]
   | ----------------------- in this macro invocation
```

Here, `MissingType` occurs inside the implementation of the proc-macro
`#[error_from_attribute]`. Previosuly, this would always result in a
span pointing at `#[error_from_attribute]`

This will make many proc-macro-related error message much more useful -
when a proc-macro generates code containing an error, users will get an
error message pointing directly at that code (within the macro
definition), instead of always getting a span pointing at the macro
invocation site.

This is implemented as follows:
* When a proc-macro crate is being *compiled*, it causes the `quote!`
  macro to get run. This saves all of the sapns in the input to `quote!`
  into the metadata of *the proc-macro-crate* (which we are currently
  compiling). The `quote!` macro then expands to a call to
  `proc_macro::Span::recover_proc_macro_span(id)`, where `id` is an
opaque identifier for the span in the crate metadata.
* When the same proc-macro crate is *run* (e.g. it is loaded from disk
  and invoked by some consumer crate), the call to
`proc_macro::Span::recover_proc_macro_span` causes us to load the span
from the proc-macro crate's metadata. The proc-macro then produces a
`TokenStream` containing a `Span` pointing into the proc-macro crate
itself.

The recursive nature of 'quote!' can be difficult to understand at
first. The file `src/test/ui/proc-macro/quote-debug.stdout` shows
the output of the `quote!` macro, which should make this eaier to
understand.

This PR also supports custom quoting spans in custom quote macros (e.g.
the `quote` crate). All span quoting goes through the
`proc_macro::quote_span` method, which can be called by a custom quote
macro to perform span quoting. An example of this usage is provided in
`src/test/ui/proc-macro/auxiliary/custom-quote.rs`

Custom quoting currently has a few limitations:

In order to quote a span, we need to generate a call to
`proc_macro::Span::recover_proc_macro_span`. However, proc-macros
support renaming the `proc_macro` crate, so we can't simply hardcode
this path. Previously, the `quote_span` method used the path
`crate::Span` - however, this only works when it is called by the
builtin `quote!` macro in the same crate. To support being called from
arbitrary crates, we need access to the name of the `proc_macro` crate
to generate a path. This PR adds an additional argument to `quote_span`
to specify the name of the `proc_macro` crate. Howver, this feels kind
of hacky, and we may want to change this before stabilizing anything
quote-related.

Additionally, using `quote_span` currently requires enabling the
`proc_macro_internals` feature. The builtin `quote!` macro
has an `#[allow_internal_unstable]` attribute, but this won't work for
custom quote implementations. This will likely require some additional
tricks to apply `allow_internal_unstable` to the span of
`proc_macro::Span::recover_proc_macro_span`.

compiler/rustc_expand/src/proc_macro_server.rs

@@ -1,4 +1,4 @@
-use crate::base::ExtCtxt;
+use crate::base::{ExtCtxt, ResolverExpand};
 
 use rustc_ast as ast;
 use rustc_ast::token;
@@ -7,6 +7,7 @@ use rustc_ast::token::NtIdent;
 use rustc_ast::tokenstream::{self, CanSynthesizeMissingTokens};
 use rustc_ast::tokenstream::{DelimSpan, Spacing::*, TokenStream, TreeAndSpacing};
 use rustc_ast_pretty::pprust;
+use rustc_data_structures::fx::FxHashMap;
 use rustc_data_structures::sync::Lrc;
 use rustc_errors::Diagnostic;
 use rustc_lint_defs::builtin::PROC_MACRO_BACK_COMPAT;
@@ -14,6 +15,8 @@ use rustc_lint_defs::BuiltinLintDiagnostics;
 use rustc_parse::lexer::nfc_normalize;
 use rustc_parse::{nt_to_tokenstream, parse_stream_from_source_str};
 use rustc_session::parse::ParseSess;
+use rustc_span::def_id::CrateNum;
+use rustc_span::hygiene::ExpnId;
 use rustc_span::hygiene::ExpnKind;
 use rustc_span::symbol::{self, kw, sym, Symbol};
 use rustc_span::{BytePos, FileName, MultiSpan, Pos, RealFileName, SourceFile, Span};
@@ -355,22 +358,34 @@ pub struct Literal {
 }
 
 pub(crate) struct Rustc<'a> {
+    resolver: &'a dyn ResolverExpand,
     sess: &'a ParseSess,
     def_site: Span,
     call_site: Span,
     mixed_site: Span,
     span_debug: bool,
+    krate: CrateNum,
+    expn_id: ExpnId,
+    rebased_spans: FxHashMap<usize, Span>,
 }
 
 impl<'a> Rustc<'a> {
-    pub fn new(cx: &'a ExtCtxt<'_>) -> Self {
+    pub fn new(cx: &'a ExtCtxt<'_>, krate: CrateNum) -> Self {
         let expn_data = cx.current_expansion.id.expn_data();
+        let def_site = cx.with_def_site_ctxt(expn_data.def_site);
+        let call_site = cx.with_call_site_ctxt(expn_data.call_site);
+        let mixed_site = cx.with_mixed_site_ctxt(expn_data.call_site);
+        let sess = cx.parse_sess();
         Rustc {
-            sess: &cx.sess.parse_sess,
-            def_site: cx.with_def_site_ctxt(expn_data.def_site),
-            call_site: cx.with_call_site_ctxt(expn_data.call_site),
-            mixed_site: cx.with_mixed_site_ctxt(expn_data.call_site),
+            resolver: cx.resolver,
+            sess,
+            def_site,
+            call_site,
+            mixed_site,
             span_debug: cx.ecfg.span_debug,
+            krate,
+            expn_id: cx.current_expansion.id,
+            rebased_spans: FxHashMap::default(),
         }
     }
 
@@ -713,6 +728,51 @@ impl server::Span for Rustc<'_> {
     fn source_text(&mut self, span: Self::Span) -> Option<String> {
         self.sess.source_map().span_to_snippet(span).ok()
     }
+    /// Saves the provided span into the metadata of
+    /// *the crate we are currently compiling*, which must
+    /// be a proc-macro crate. This id can be passed to
+    /// `recover_proc_macro_span` when our current crate
+    /// is *run* as a proc-macro.
+    ///
+    /// Let's suppose that we have two crates - `my_client`
+    /// and `my_proc_macro`. The `my_proc_macro` crate
+    /// contains a procedural macro `my_macro`, which
+    /// is implemented as: `quote! { "hello" }`
+    ///
+    /// When we *compile* `my_proc_macro`, we will execute
+    /// the `quote` proc-macro. This will save the span of
+    /// "hello" into the metadata of `my_proc_macro`. As a result,
+    /// the body of `my_proc_macro` (after expansion) will end
+    /// up containg a call that looks like this:
+    /// `proc_macro::Ident::new("hello", proc_macro::Span::recover_proc_macro_span(0))`
+    ///
+    /// where `0` is the id returned by this function.
+    /// When `my_proc_macro` *executes* (during the compilation of `my_client`),
+    /// the call to `recover_proc_macro_span` will load the corresponding
+    /// span from the metadata of `my_proc_macro` (which we have access to,
+    /// since we've loaded `my_proc_macro` from disk in order to execute it).
+    /// In this way, we have obtained a span pointing into `my_proc_macro`
+    fn save_span(&mut self, mut span: Self::Span) -> usize {
+        // Throw away the `SyntaxContext`, since we currently
+        // skip serializing `SyntaxContext`s for proc-macro crates
+        span = span.with_ctxt(rustc_span::SyntaxContext::root());
+        self.sess.save_proc_macro_span(span)
+    }
+    fn recover_proc_macro_span(&mut self, id: usize) -> Self::Span {
+        let resolver = self.resolver;
+        let krate = self.krate;
+        let expn_id = self.expn_id;
+        *self.rebased_spans.entry(id).or_insert_with(|| {
+            let raw_span = resolver.get_proc_macro_quoted_span(krate, id);
+            // Ignore the deserialized `SyntaxContext` entirely.
+            // FIXME: Preserve the macro backtrace from the serialized span
+            // For example, if a proc-macro crate has code like
+            // `macro_one!() -> macro_two!() -> quote!()`, we might
+            // want to 'concatenate' this backtrace with the backtrace from
+            // our current call site.
+            raw_span.with_def_site_ctxt(expn_id)
+        })
+    }
 }
 
 // See issue #74616 for details
@@ -722,7 +782,7 @@ fn ident_name_compatibility_hack(
     rustc: &mut Rustc<'_>,
 ) -> Option<(rustc_span::symbol::Ident, bool)> {
     if let NtIdent(ident, is_raw) = nt {
-        if let ExpnKind::Macro(_, macro_name) = orig_span.ctxt().outer_expn_data().kind {
+        if let ExpnKind::Macro { name: macro_name, .. } = orig_span.ctxt().outer_expn_data().kind {
             let source_map = rustc.sess.source_map();
             let filename = source_map.span_to_filename(orig_span);
             if let FileName::Real(RealFileName::Named(path)) = filename {