Auto merge of #101692 - cjgillot:generator-lazy-witness, r=oli-obk

Compute generator saved locals on MIR

Generators are currently type-checked by introducing a `witness` type variable, which is unified with a `GeneratorWitness(captured types)` whose purpose is to ensure that the auto traits correctly migrate from the captured types to the `witness` type.  This requires computing the captured types on HIR during type-checking, only to re-do it on MIR later.

This PR proposes to drop the HIR-based computation, and only keep the MIR one.  This is done in 3 steps.
1. During type-checking, the `witness` type variable is never unified.  This allows to stall all the obligations that depend on it until the end of type-checking.  Then, the stalled obligations are marked as successful, and saved into the typeck results for later verification.
2. At type-checking writeback, `witness` is replaced by `GeneratorWitnessMIR(def_id, substs)`.  From this point on, all trait selection involving `GeneratorWitnessMIR` will fetch the MIR-computed locals, similar to what opaque types do.  There is no lifetime to be preserved here: we consider all the lifetimes appearing in this witness type to be higher-ranked.
3. After borrowck, the stashed obligations are verified against the actually computed types, in the `check_generator_obligations` query.  If any obligation was wrongly marked as fulfilled in step 1, it should be reported here.

There are still many issues:
- ~I am not too happy having to filter out some locals from the checked bounds, I think this is MIR building that introduces raw pointers polluting the analysis;~ solved by a check specific to static variables.
- the diagnostics for captured types don't show where they are used/dropped;
- I do not attempt to support chalk.

cc `@eholk` `@jyn514` for the drop-tracking work
r? `@oli-obk` as you warned me of potential unsoundness

compiler/rustc_codegen_ssa/src/debuginfo/type_names.rs

@@ -414,6 +414,7 @@ fn push_debuginfo_type_name<'tcx>(
         | ty::Placeholder(..)
         | ty::Alias(..)
         | ty::Bound(..)
+        | ty::GeneratorWitnessMIR(..)
         | ty::GeneratorWitness(..) => {
             bug!(
                 "debuginfo: Trying to create type name for \

compiler/rustc_const_eval/src/const_eval/valtrees.rs

@@ -151,7 +151,7 @@ pub(crate) fn const_to_valtree_inner<'tcx>(
         // FIXME(oli-obk): we can probably encode closures just like structs
         | ty::Closure(..)
         | ty::Generator(..)
-        | ty::GeneratorWitness(..) => Err(ValTreeCreationError::NonSupportedType),
+        | ty::GeneratorWitness(..) |ty::GeneratorWitnessMIR(..)=> Err(ValTreeCreationError::NonSupportedType),
     }
 }
 
@@ -314,6 +314,7 @@ pub fn valtree_to_const_value<'tcx>(
         | ty::Closure(..)
         | ty::Generator(..)
         | ty::GeneratorWitness(..)
+        | ty::GeneratorWitnessMIR(..)
         | ty::FnPtr(_)
         | ty::RawPtr(_)
         | ty::Str

compiler/rustc_const_eval/src/interpret/intrinsics.rs

@@ -101,6 +101,7 @@ pub(crate) fn eval_nullary_intrinsic<'tcx>(
             | ty::Closure(_, _)
             | ty::Generator(_, _, _)
             | ty::GeneratorWitness(_)
+            | ty::GeneratorWitnessMIR(_, _)
             | ty::Never
             | ty::Tuple(_)
             | ty::Error(_) => ConstValue::from_machine_usize(0u64, &tcx),

compiler/rustc_const_eval/src/interpret/validity.rs

@@ -602,6 +602,7 @@ impl<'rt, 'mir, 'tcx: 'mir, M: Machine<'mir, 'tcx>> ValidityVisitor<'rt, 'mir, '
             | ty::Bound(..)
             | ty::Param(..)
             | ty::Alias(..)
+            | ty::GeneratorWitnessMIR(..)
             | ty::GeneratorWitness(..) => bug!("Encountered invalid type {:?}", ty),
         }
     }

compiler/rustc_const_eval/src/transform/validate.rs

@@ -372,12 +372,12 @@ impl<'a, 'tcx> Visitor<'tcx> for TypeChecker<'a, 'tcx> {
                                 return;
                             };
 
-                            let Some(&f_ty) = layout.field_tys.get(local) else {
+                            let Some(f_ty) = layout.field_tys.get(local) else {
                                 self.fail(location, format!("Out of bounds local {:?} for {:?}", local, parent_ty));
                                 return;
                             };
 
-                            f_ty
+                            f_ty.ty
                         } else {
                             let Some(f_ty) = substs.as_generator().prefix_tys().nth(f.index()) else {
                                 fail_out_of_bounds(self, location);

compiler/rustc_const_eval/src/util/type_name.rs

@@ -64,6 +64,7 @@ impl<'tcx> Printer<'tcx> for AbsolutePathPrinter<'tcx> {
             ty::Foreign(def_id) => self.print_def_path(def_id, &[]),
 
             ty::GeneratorWitness(_) => bug!("type_name: unexpected `GeneratorWitness`"),
+            ty::GeneratorWitnessMIR(..) => bug!("type_name: unexpected `GeneratorWitnessMIR`"),
         }
     }
 

compiler/rustc_hir/src/hir.rs

@@ -2106,8 +2106,8 @@ pub enum LocalSource {
 }
 
 /// Hints at the original code for a `match _ { .. }`.
-#[derive(Copy, Clone, PartialEq, Eq, Encodable, Hash, Debug)]
+#[derive(Copy, Clone, PartialEq, Eq, Hash, Debug)]
-#[derive(HashStable_Generic)]
+#[derive(HashStable_Generic, Encodable, Decodable)]
 pub enum MatchSource {
     /// A `match _ { .. }`.
     Normal,

compiler/rustc_hir_analysis/src/check/check.rs

@@ -14,7 +14,7 @@ use rustc_hir::{ItemKind, Node, PathSegment};
 use rustc_infer::infer::opaque_types::ConstrainOpaqueTypeRegionVisitor;
 use rustc_infer::infer::outlives::env::OutlivesEnvironment;
 use rustc_infer::infer::{DefiningAnchor, RegionVariableOrigin, TyCtxtInferExt};
-use rustc_infer::traits::Obligation;
+use rustc_infer::traits::{Obligation, TraitEngineExt as _};
 use rustc_lint::builtin::REPR_TRANSPARENT_EXTERNAL_PRIVATE_FIELDS;
 use rustc_middle::hir::nested_filter;
 use rustc_middle::middle::stability::EvalResult;
@@ -28,7 +28,7 @@ use rustc_span::{self, Span};
 use rustc_target::spec::abi::Abi;
 use rustc_trait_selection::traits::error_reporting::on_unimplemented::OnUnimplementedDirective;
 use rustc_trait_selection::traits::error_reporting::TypeErrCtxtExt as _;
-use rustc_trait_selection::traits::{self, ObligationCtxt};
+use rustc_trait_selection::traits::{self, ObligationCtxt, TraitEngine, TraitEngineExt as _};
 
 use std::ops::ControlFlow;
 
@@ -1460,7 +1460,8 @@ fn opaque_type_cycle_error(
                 for def_id in visitor.opaques {
                     let ty_span = tcx.def_span(def_id);
                     if !seen.contains(&ty_span) {
-                        err.span_label(ty_span, &format!("returning this opaque type `{ty}`"));
+                        let descr = if ty.is_impl_trait() { "opaque " } else { "" };
+                        err.span_label(ty_span, &format!("returning this {descr}type `{ty}`"));
                         seen.insert(ty_span);
                     }
                     err.span_label(sp, &format!("returning here with type `{ty}`"));
@@ -1507,3 +1508,34 @@ fn opaque_type_cycle_error(
     }
     err.emit()
 }
+
+pub(super) fn check_generator_obligations(tcx: TyCtxt<'_>, def_id: LocalDefId) {
+    debug_assert!(tcx.sess.opts.unstable_opts.drop_tracking_mir);
+    debug_assert!(matches!(tcx.def_kind(def_id), DefKind::Generator));
+
+    let typeck = tcx.typeck(def_id);
+    let param_env = tcx.param_env(def_id);
+
+    let generator_interior_predicates = &typeck.generator_interior_predicates[&def_id];
+    debug!(?generator_interior_predicates);
+
+    let infcx = tcx
+        .infer_ctxt()
+        // typeck writeback gives us predicates with their regions erased.
+        // As borrowck already has checked lifetimes, we do not need to do it again.
+        .ignoring_regions()
+        // Bind opaque types to `def_id` as they should have been checked by borrowck.
+        .with_opaque_type_inference(DefiningAnchor::Bind(def_id))
+        .build();
+
+    let mut fulfillment_cx = <dyn TraitEngine<'_>>::new(infcx.tcx);
+    for (predicate, cause) in generator_interior_predicates {
+        let obligation = Obligation::new(tcx, cause.clone(), param_env, *predicate);
+        fulfillment_cx.register_predicate_obligation(&infcx, obligation);
+    }
+    let errors = fulfillment_cx.select_all_or_error(&infcx);
+    debug!(?errors);
+    if !errors.is_empty() {
+        infcx.err_ctxt().report_fulfillment_errors(&errors, None);
+    }
+}

compiler/rustc_hir_analysis/src/check/mod.rs

@@ -105,6 +105,7 @@ pub fn provide(providers: &mut Providers) {
         region_scope_tree,
         collect_return_position_impl_trait_in_trait_tys,
         compare_impl_const: compare_impl_item::compare_impl_const_raw,
+        check_generator_obligations: check::check_generator_obligations,
         ..*providers
     };
 }

compiler/rustc_hir_analysis/src/coherence/inherent_impls.rs

@@ -240,6 +240,7 @@ impl<'tcx> InherentCollect<'tcx> {
             | ty::Closure(..)
             | ty::Generator(..)
             | ty::GeneratorWitness(..)
+            | ty::GeneratorWitnessMIR(..)
             | ty::Bound(..)
             | ty::Placeholder(_)
             | ty::Infer(_) => {

compiler/rustc_hir_analysis/src/variance/constraints.rs

@@ -295,12 +295,12 @@ impl<'a, 'tcx> ConstraintContext<'a, 'tcx> {
                 // types, where we use Error as the Self type
             }
 
-            ty::Placeholder(..) | ty::GeneratorWitness(..) | ty::Bound(..) | ty::Infer(..) => {
+            ty::Placeholder(..)
-                bug!(
+            | ty::GeneratorWitness(..)
-                    "unexpected type encountered in \
+            | ty::GeneratorWitnessMIR(..)
-                      variance inference: {}",
+            | ty::Bound(..)
-                    ty
+            | ty::Infer(..) => {
-                );
+                bug!("unexpected type encountered in variance inference: {}", ty);
             }
         }
     }

compiler/rustc_hir_typeck/src/cast.rs

@@ -130,6 +130,7 @@ impl<'a, 'tcx> FnCtxt<'a, 'tcx> {
             | ty::Float(_)
             | ty::Array(..)
             | ty::GeneratorWitness(..)
+            | ty::GeneratorWitnessMIR(..)
             | ty::RawPtr(_)
             | ty::Ref(..)
             | ty::FnDef(..)

compiler/rustc_hir_typeck/src/check.rs

@@ -130,7 +130,7 @@ pub(super) fn check_fn<'a, 'tcx>(
     let gen_ty = if let (Some(_), Some(gen_kind)) = (can_be_generator, body.generator_kind) {
         let interior = fcx
             .next_ty_var(TypeVariableOrigin { kind: TypeVariableOriginKind::MiscVariable, span });
-        fcx.deferred_generator_interiors.borrow_mut().push((body.id(), interior, gen_kind));
+        fcx.deferred_generator_interiors.borrow_mut().push((fn_id, body.id(), interior, gen_kind));
 
         let (resume_ty, yield_ty) = fcx.resume_yield_tys.unwrap();
         Some(GeneratorTypes {

compiler/rustc_hir_typeck/src/fn_ctxt/_impl.rs

@@ -517,16 +517,73 @@ impl<'a, 'tcx> FnCtxt<'a, 'tcx> {
     }
 
     pub(in super::super) fn resolve_generator_interiors(&self, def_id: DefId) {
-        let mut generators = self.deferred_generator_interiors.borrow_mut();
+        if self.tcx.sess.opts.unstable_opts.drop_tracking_mir {
-        for (body_id, interior, kind) in generators.drain(..) {
+            self.save_generator_interior_predicates(def_id);
+            return;
+        }
+
         self.select_obligations_where_possible(|_| {});
+
+        let mut generators = self.deferred_generator_interiors.borrow_mut();
+        for (_, body_id, interior, kind) in generators.drain(..) {
             crate::generator_interior::resolve_interior(self, def_id, body_id, interior, kind);
+            self.select_obligations_where_possible(|_| {});
+        }
+    }
+
+    /// Unify the inference variables corresponding to generator witnesses, and save all the
+    /// predicates that were stalled on those inference variables.
+    ///
+    /// This process allows to conservatively save all predicates that do depend on the generator
+    /// interior types, for later processing by `check_generator_obligations`.
+    ///
+    /// We must not attempt to select obligations after this method has run, or risk query cycle
+    /// ICE.
+    #[instrument(level = "debug", skip(self))]
+    fn save_generator_interior_predicates(&self, def_id: DefId) {
+        // Try selecting all obligations that are not blocked on inference variables.
+        // Once we start unifying generator witnesses, trying to select obligations on them will
+        // trigger query cycle ICEs, as doing so requires MIR.
+        self.select_obligations_where_possible(|_| {});
+
+        let generators = std::mem::take(&mut *self.deferred_generator_interiors.borrow_mut());
+        debug!(?generators);
+
+        for &(expr_hir_id, body_id, interior, _) in generators.iter() {
+            let expr_def_id = self.tcx.hir().local_def_id(expr_hir_id);
+            debug!(?expr_def_id);
+
+            // Create the `GeneratorWitness` type that we will unify with `interior`.
+            let substs = ty::InternalSubsts::identity_for_item(
+                self.tcx,
+                self.tcx.typeck_root_def_id(expr_def_id.to_def_id()),
+            );
+            let witness = self.tcx.mk_generator_witness_mir(expr_def_id.to_def_id(), substs);
+
+            // Unify `interior` with `witness` and collect all the resulting obligations.
+            let span = self.tcx.hir().body(body_id).value.span;
+            let ok = self
+                .at(&self.misc(span), self.param_env)
+                .eq(interior, witness)
+                .expect("Failed to unify generator interior type");
+            let mut obligations = ok.obligations;
+
+            // Also collect the obligations that were unstalled by this unification.
+            obligations
+                .extend(self.fulfillment_cx.borrow_mut().drain_unstalled_obligations(&self.infcx));
+
+            let obligations = obligations.into_iter().map(|o| (o.predicate, o.cause)).collect();
+            debug!(?obligations);
+            self.typeck_results
+                .borrow_mut()
+                .generator_interior_predicates
+                .insert(expr_def_id, obligations);
         }
     }
 
     #[instrument(skip(self), level = "debug")]
-    pub(in super::super) fn select_all_obligations_or_error(&self) {
+    pub(in super::super) fn report_ambiguity_errors(&self) {
-        let mut errors = self.fulfillment_cx.borrow_mut().select_all_or_error(&self);
+        let mut errors = self.fulfillment_cx.borrow_mut().collect_remaining_errors();
 
         if !errors.is_empty() {
             self.adjust_fulfillment_errors_for_expr_obligation(&mut errors);

compiler/rustc_hir_typeck/src/inherited.rs

@@ -56,7 +56,7 @@ pub struct Inherited<'tcx> {
     pub(super) deferred_asm_checks: RefCell<Vec<(&'tcx hir::InlineAsm<'tcx>, hir::HirId)>>,
 
     pub(super) deferred_generator_interiors:
-        RefCell<Vec<(hir::BodyId, Ty<'tcx>, hir::GeneratorKind)>>,
+        RefCell<Vec<(hir::HirId, hir::BodyId, Ty<'tcx>, hir::GeneratorKind)>>,
 
     pub(super) body_id: Option<hir::BodyId>,
 

compiler/rustc_hir_typeck/src/lib.rs

@@ -294,14 +294,24 @@ fn typeck_with_fallback<'tcx>(
         // Before the generator analysis, temporary scopes shall be marked to provide more
         // precise information on types to be captured.
         fcx.resolve_rvalue_scopes(def_id.to_def_id());
-        fcx.resolve_generator_interiors(def_id.to_def_id());
 
         for (ty, span, code) in fcx.deferred_sized_obligations.borrow_mut().drain(..) {
             let ty = fcx.normalize(span, ty);
             fcx.require_type_is_sized(ty, span, code);
         }
 
-        fcx.select_all_obligations_or_error();
+        fcx.select_obligations_where_possible(|_| {});
+
+        debug!(pending_obligations = ?fcx.fulfillment_cx.borrow().pending_obligations());
+
+        // This must be the last thing before `report_ambiguity_errors`.
+        fcx.resolve_generator_interiors(def_id.to_def_id());
+
+        debug!(pending_obligations = ?fcx.fulfillment_cx.borrow().pending_obligations());
+
+        if let None = fcx.infcx.tainted_by_errors() {
+            fcx.report_ambiguity_errors();
+        }
 
         if let None = fcx.infcx.tainted_by_errors() {
             fcx.check_transmutes();

compiler/rustc_hir_typeck/src/writeback.rs

@@ -545,6 +545,10 @@ impl<'cx, 'tcx> WritebackCx<'cx, 'tcx> {
         assert_eq!(fcx_typeck_results.hir_owner, self.typeck_results.hir_owner);
         self.typeck_results.generator_interior_types =
             fcx_typeck_results.generator_interior_types.clone();
+        for (&expr_def_id, predicates) in fcx_typeck_results.generator_interior_predicates.iter() {
+            let predicates = self.resolve(predicates.clone(), &self.fcx.tcx.def_span(expr_def_id));
+            self.typeck_results.generator_interior_predicates.insert(expr_def_id, predicates);
+        }
     }
 
     #[instrument(skip(self), level = "debug")]

compiler/rustc_infer/src/infer/canonical/canonicalizer.rs

@@ -435,6 +435,7 @@ impl<'cx, 'tcx> TypeFolder<'tcx> for Canonicalizer<'cx, 'tcx> {
             ty::Closure(..)
             | ty::Generator(..)
             | ty::GeneratorWitness(..)
+            | ty::GeneratorWitnessMIR(..)
             | ty::Bool
             | ty::Char
             | ty::Int(..)

compiler/rustc_infer/src/infer/canonical/query_response.rs

@@ -17,7 +17,7 @@ use crate::infer::region_constraints::{Constraint, RegionConstraintData};
 use crate::infer::{InferCtxt, InferOk, InferResult, NllRegionVariableOrigin};
 use crate::traits::query::{Fallible, NoSolution};
 use crate::traits::{Obligation, ObligationCause, PredicateObligation};
-use crate::traits::{PredicateObligations, TraitEngine};
+use crate::traits::{PredicateObligations, TraitEngine, TraitEngineExt};
 use rustc_data_structures::captures::Captures;
 use rustc_index::vec::Idx;
 use rustc_index::vec::IndexVec;

compiler/rustc_infer/src/infer/freshen.rs

@@ -209,6 +209,7 @@ impl<'a, 'tcx> TypeFolder<'tcx> for TypeFreshener<'a, 'tcx> {
             | ty::Foreign(..)
             | ty::Param(..)
             | ty::Closure(..)
+            | ty::GeneratorWitnessMIR(..)
             | ty::GeneratorWitness(..) => t.super_fold_with(self),
 
             ty::Placeholder(..) | ty::Bound(..) => bug!("unexpected type {:?}", t),

compiler/rustc_infer/src/infer/outlives/components.rs

@@ -112,7 +112,7 @@ fn compute_components<'tcx>(
             }
 
             // All regions are bound inside a witness
-            ty::GeneratorWitness(..) => (),
+            ty::GeneratorWitness(..) | ty::GeneratorWitnessMIR(..) => (),
 
             // OutlivesTypeParameterEnv -- the actual checking that `X:'a`
             // is implied by the environment is done in regionck.

compiler/rustc_infer/src/traits/engine.rs

@@ -36,11 +36,19 @@ pub trait TraitEngine<'tcx>: 'tcx {
         obligation: PredicateObligation<'tcx>,
     );
 
-    fn select_all_or_error(&mut self, infcx: &InferCtxt<'tcx>) -> Vec<FulfillmentError<'tcx>>;
-
     fn select_where_possible(&mut self, infcx: &InferCtxt<'tcx>) -> Vec<FulfillmentError<'tcx>>;
 
+    fn collect_remaining_errors(&mut self) -> Vec<FulfillmentError<'tcx>>;
+
     fn pending_obligations(&self) -> Vec<PredicateObligation<'tcx>>;
+
+    /// Among all pending obligations, collect those are stalled on a inference variable which has
+    /// changed since the last call to `select_where_possible`. Those obligations are marked as
+    /// successful and returned.
+    fn drain_unstalled_obligations(
+        &mut self,
+        infcx: &InferCtxt<'tcx>,
+    ) -> Vec<PredicateObligation<'tcx>>;
 }
 
 pub trait TraitEngineExt<'tcx> {
@@ -49,6 +57,8 @@ pub trait TraitEngineExt<'tcx> {
         infcx: &InferCtxt<'tcx>,
         obligations: impl IntoIterator<Item = PredicateObligation<'tcx>>,
     );
+
+    fn select_all_or_error(&mut self, infcx: &InferCtxt<'tcx>) -> Vec<FulfillmentError<'tcx>>;
 }
 
 impl<'tcx, T: ?Sized + TraitEngine<'tcx>> TraitEngineExt<'tcx> for T {
@@ -61,4 +71,13 @@ impl<'tcx, T: ?Sized + TraitEngine<'tcx>> TraitEngineExt<'tcx> for T {
             self.register_predicate_obligation(infcx, obligation);
         }
     }
+
+    fn select_all_or_error(&mut self, infcx: &InferCtxt<'tcx>) -> Vec<FulfillmentError<'tcx>> {
+        let errors = self.select_where_possible(infcx);
+        if !errors.is_empty() {
+            return errors;
+        }
+
+        self.collect_remaining_errors()
+    }
 }

compiler/rustc_interface/src/passes.rs

@@ -893,6 +893,15 @@ fn analysis(tcx: TyCtxt<'_>, (): ()) -> Result<()> {
         }
     });
 
+    if tcx.sess.opts.unstable_opts.drop_tracking_mir {
+        tcx.hir().par_body_owners(|def_id| {
+            if let rustc_hir::def::DefKind::Generator = tcx.def_kind(def_id) {
+                tcx.ensure().mir_generator_witnesses(def_id);
+                tcx.ensure().check_generator_obligations(def_id);
+            }
+        });
+    }
+
     sess.time("layout_testing", || layout_test::test_layout(tcx));
 
     // Avoid overwhelming user with errors if borrow checking failed.

compiler/rustc_lint/src/types.rs

@@ -1107,6 +1107,7 @@ impl<'a, 'tcx> ImproperCTypesVisitor<'a, 'tcx> {
             | ty::Closure(..)
             | ty::Generator(..)
             | ty::GeneratorWitness(..)
+            | ty::GeneratorWitnessMIR(..)
             | ty::Placeholder(..)
             | ty::FnDef(..) => bug!("unexpected type in foreign function: {:?}", ty),
         }

compiler/rustc_metadata/src/rmeta/decoder/cstore_impl.rs

@@ -202,6 +202,7 @@ provide! { tcx, def_id, other, cdata,
     thir_abstract_const => { table }
     optimized_mir => { table }
     mir_for_ctfe => { table }
+    mir_generator_witnesses => { table }
     promoted_mir => { table }
     def_span => { table }
     def_ident_span => { table }

compiler/rustc_metadata/src/rmeta/encoder.rs

@@ -1414,6 +1414,10 @@ impl<'a, 'tcx> EncodeContext<'a, 'tcx> {
             debug!("EntryBuilder::encode_mir({:?})", def_id);
             if encode_opt {
                 record!(self.tables.optimized_mir[def_id.to_def_id()] <- tcx.optimized_mir(def_id));
+
+                if let DefKind::Generator = self.tcx.def_kind(def_id) {
+                    record!(self.tables.mir_generator_witnesses[def_id.to_def_id()] <- tcx.mir_generator_witnesses(def_id));
+                }
             }
             if encode_const {
                 record!(self.tables.mir_for_ctfe[def_id.to_def_id()] <- tcx.mir_for_ctfe(def_id));

compiler/rustc_metadata/src/rmeta/mod.rs

@@ -376,6 +376,7 @@ define_tables! {
     object_lifetime_default: Table<DefIndex, LazyValue<ObjectLifetimeDefault>>,
     optimized_mir: Table<DefIndex, LazyValue<mir::Body<'static>>>,
     mir_for_ctfe: Table<DefIndex, LazyValue<mir::Body<'static>>>,
+    mir_generator_witnesses: Table<DefIndex, LazyValue<mir::GeneratorLayout<'static>>>,
     promoted_mir: Table<DefIndex, LazyValue<IndexVec<mir::Promoted, mir::Body<'static>>>>,
     // FIXME(compiler-errors): Why isn't this a LazyArray?
     thir_abstract_const: Table<DefIndex, LazyValue<ty::Const<'static>>>,

compiler/rustc_middle/src/mir/mod.rs

@@ -902,6 +902,8 @@ pub enum LocalInfo<'tcx> {
     AggregateTemp,
     /// A temporary created during the pass `Derefer` to avoid it's retagging
     DerefTemp,
+    /// A temporary created for borrow checking.
+    FakeBorrow,
 }
 
 impl<'tcx> LocalDecl<'tcx> {

compiler/rustc_middle/src/mir/query.rs

@@ -135,11 +135,20 @@ rustc_index::newtype_index! {
     pub struct GeneratorSavedLocal {}
 }
 
+#[derive(Clone, Debug, TyEncodable, TyDecodable, HashStable, TypeFoldable, TypeVisitable)]
+pub struct GeneratorSavedTy<'tcx> {
+    pub ty: Ty<'tcx>,
+    /// Source info corresponding to the local in the original MIR body.
+    pub source_info: SourceInfo,
+    /// Whether the local should be ignored for trait bound computations.
+    pub ignore_for_traits: bool,
+}
+
 /// The layout of generator state.
 #[derive(Clone, TyEncodable, TyDecodable, HashStable, TypeFoldable, TypeVisitable)]
 pub struct GeneratorLayout<'tcx> {
     /// The type of every local stored inside the generator.
-    pub field_tys: IndexVec<GeneratorSavedLocal, Ty<'tcx>>,
+    pub field_tys: IndexVec<GeneratorSavedLocal, GeneratorSavedTy<'tcx>>,
 
     /// Which of the above fields are in each variant. Note that one field may
     /// be stored in multiple variants.

compiler/rustc_middle/src/query/mod.rs

@@ -471,6 +471,17 @@ rustc_queries! {
         }
     }
 
+    query mir_generator_witnesses(key: DefId) -> mir::GeneratorLayout<'tcx> {
+        arena_cache
+        desc { |tcx| "generator witness types for `{}`", tcx.def_path_str(key) }
+        cache_on_disk_if { key.is_local() }
+        separate_provide_extern
+    }
+
+    query check_generator_obligations(key: LocalDefId) {
+        desc { |tcx| "verify auto trait bounds for generator interior type `{}`", tcx.def_path_str(key.to_def_id()) }
+    }
+
     /// MIR after our optimization passes have run. This is MIR that is ready
     /// for codegen. This is also the only query that can fetch non-local MIR, at present.
     query optimized_mir(key: DefId) -> &'tcx mir::Body<'tcx> {

compiler/rustc_middle/src/traits/mod.rs

@@ -37,7 +37,7 @@ pub use self::chalk::{ChalkEnvironmentAndGoal, RustInterner as ChalkRustInterner
 
 /// Depending on the stage of compilation, we want projection to be
 /// more or less conservative.
-#[derive(Debug, Copy, Clone, PartialEq, Eq, Hash, HashStable)]
+#[derive(Debug, Copy, Clone, PartialEq, Eq, Hash, HashStable, Encodable, Decodable)]
 pub enum Reveal {
     /// At type-checking time, we refuse to project any associated
     /// type that is marked `default`. Non-`default` ("final") types
@@ -90,7 +90,8 @@ pub enum Reveal {
 ///
 /// We do not want to intern this as there are a lot of obligation causes which
 /// only live for a short period of time.
-#[derive(Clone, Debug, PartialEq, Eq, Lift)]
+#[derive(Clone, Debug, PartialEq, Eq, Lift, HashStable, TyEncodable, TyDecodable)]
+#[derive(TypeVisitable, TypeFoldable)]
 pub struct ObligationCause<'tcx> {
     pub span: Span,
 
@@ -197,14 +198,16 @@ impl<'tcx> ObligationCause<'tcx> {
     }
 }
 
-#[derive(Clone, Debug, PartialEq, Eq, Hash, Lift)]
+#[derive(Clone, Debug, PartialEq, Eq, Hash, Lift, HashStable, TyEncodable, TyDecodable)]
+#[derive(TypeVisitable, TypeFoldable)]
 pub struct UnifyReceiverContext<'tcx> {
     pub assoc_item: ty::AssocItem,
     pub param_env: ty::ParamEnv<'tcx>,
     pub substs: SubstsRef<'tcx>,
 }
 
-#[derive(Clone, PartialEq, Eq, Hash, Lift, Default)]
+#[derive(Clone, PartialEq, Eq, Hash, Lift, Default, HashStable)]
+#[derive(TypeVisitable, TypeFoldable, TyEncodable, TyDecodable)]
 pub struct InternedObligationCauseCode<'tcx> {
     /// `None` for `ObligationCauseCode::MiscObligation` (a common case, occurs ~60% of
     /// the time). `Some` otherwise.
@@ -239,7 +242,8 @@ impl<'tcx> std::ops::Deref for InternedObligationCauseCode<'tcx> {
     }
 }
 
-#[derive(Clone, Debug, PartialEq, Eq, Hash, Lift)]
+#[derive(Clone, Debug, PartialEq, Eq, Hash, Lift, HashStable, TyEncodable, TyDecodable)]
+#[derive(TypeVisitable, TypeFoldable)]
 pub enum ObligationCauseCode<'tcx> {
     /// Not well classified or should be obvious from the span.
     MiscObligation,
@@ -447,7 +451,8 @@ pub enum ObligationCauseCode<'tcx> {
 /// This information is used to obtain an `hir::Ty`, which
 /// we can walk in order to obtain precise spans for any
 /// 'nested' types (e.g. `Foo` in `Option<Foo>`).
-#[derive(Copy, Clone, Debug, PartialEq, Eq, Hash, HashStable)]
+#[derive(Copy, Clone, Debug, PartialEq, Eq, Hash, HashStable, Encodable, Decodable)]
+#[derive(TypeVisitable, TypeFoldable)]
 pub enum WellFormedLoc {
     /// Use the type of the provided definition.
     Ty(LocalDefId),
@@ -464,7 +469,8 @@ pub enum WellFormedLoc {
     },
 }
 
-#[derive(Clone, Debug, PartialEq, Eq, Hash, Lift)]
+#[derive(Clone, Debug, PartialEq, Eq, Hash, Lift, HashStable, TyEncodable, TyDecodable)]
+#[derive(TypeVisitable, TypeFoldable)]
 pub struct ImplDerivedObligationCause<'tcx> {
     pub derived: DerivedObligationCause<'tcx>,
     pub impl_def_id: DefId,
@@ -518,7 +524,8 @@ impl<'tcx> ty::Lift<'tcx> for StatementAsExpression {
     }
 }
 
-#[derive(Clone, Debug, PartialEq, Eq, Hash, Lift)]
+#[derive(Clone, Debug, PartialEq, Eq, Hash, Lift, HashStable, TyEncodable, TyDecodable)]
+#[derive(TypeVisitable, TypeFoldable)]
 pub struct MatchExpressionArmCause<'tcx> {
     pub arm_block_id: Option<hir::HirId>,
     pub arm_ty: Ty<'tcx>,
@@ -534,7 +541,7 @@ pub struct MatchExpressionArmCause<'tcx> {
 }
 
 #[derive(Copy, Clone, Debug, PartialEq, Eq, Hash)]
-#[derive(Lift, TypeFoldable, TypeVisitable)]
+#[derive(Lift, TypeFoldable, TypeVisitable, HashStable, TyEncodable, TyDecodable)]
 pub struct IfExpressionCause<'tcx> {
     pub then_id: hir::HirId,
     pub else_id: hir::HirId,
@@ -544,7 +551,8 @@ pub struct IfExpressionCause<'tcx> {
     pub opt_suggest_box_span: Option<Span>,
 }
 
-#[derive(Clone, Debug, PartialEq, Eq, Hash, Lift)]
+#[derive(Clone, Debug, PartialEq, Eq, Hash, Lift, HashStable, TyEncodable, TyDecodable)]
+#[derive(TypeVisitable, TypeFoldable)]
 pub struct DerivedObligationCause<'tcx> {
     /// The trait predicate of the parent obligation that led to the
     /// current obligation. Note that only trait obligations lead to

compiler/rustc_middle/src/ty/adt.rs

@@ -188,7 +188,7 @@ impl<'tcx> AdtDef<'tcx> {
     }
 }
 
-#[derive(Copy, Clone, Debug, Eq, PartialEq, Hash, TyEncodable, TyDecodable)]
+#[derive(Copy, Clone, Debug, Eq, PartialEq, Hash, HashStable, TyEncodable, TyDecodable)]
 pub enum AdtKind {
     Struct,
     Union,

compiler/rustc_middle/src/ty/codec.rs

@@ -157,6 +157,14 @@ impl<'tcx, E: TyEncoder<I = TyCtxt<'tcx>>> Encodable<E> for AllocId {
     }
 }
 
+impl<'tcx, E: TyEncoder<I = TyCtxt<'tcx>>> Encodable<E> for ty::ParamEnv<'tcx> {
+    fn encode(&self, e: &mut E) {
+        self.caller_bounds().encode(e);
+        self.reveal().encode(e);
+        self.constness().encode(e);
+    }
+}
+
 #[inline]
 fn decode_arena_allocable<
     'tcx,
@@ -280,8 +288,17 @@ impl<'tcx, D: TyDecoder<I = TyCtxt<'tcx>>> Decodable<D> for ty::SymbolName<'tcx>
     }
 }
 
+impl<'tcx, D: TyDecoder<I = TyCtxt<'tcx>>> Decodable<D> for ty::ParamEnv<'tcx> {
+    fn decode(d: &mut D) -> Self {
+        let caller_bounds = Decodable::decode(d);
+        let reveal = Decodable::decode(d);
+        let constness = Decodable::decode(d);
+        ty::ParamEnv::new(caller_bounds, reveal, constness)
+    }
+}
+
 macro_rules! impl_decodable_via_ref {
-    ($($t:ty),+) => {
+    ($($t:ty,)+) => {
         $(impl<'tcx, D: TyDecoder<I = TyCtxt<'tcx>>> Decodable<D> for $t {
             fn decode(decoder: &mut D) -> Self {
                 RefDecodable::decode(decoder)
@@ -373,6 +390,15 @@ impl<'tcx, D: TyDecoder<I = TyCtxt<'tcx>>> RefDecodable<'tcx, D> for ty::List<ty
     }
 }
 
+impl<'tcx, D: TyDecoder<I = TyCtxt<'tcx>>> RefDecodable<'tcx, D> for ty::List<ty::Predicate<'tcx>> {
+    fn decode(decoder: &mut D) -> &'tcx Self {
+        let len = decoder.read_usize();
+        let predicates: Vec<_> =
+            (0..len).map::<ty::Predicate<'tcx>, _>(|_| Decodable::decode(decoder)).collect();
+        decoder.interner().intern_predicates(&predicates)
+    }
+}
+
 impl_decodable_via_ref! {
     &'tcx ty::TypeckResults<'tcx>,
     &'tcx ty::List<Ty<'tcx>>,
@@ -382,7 +408,8 @@ impl_decodable_via_ref! {
     &'tcx mir::UnsafetyCheckResult,
     &'tcx mir::BorrowCheckResult<'tcx>,
     &'tcx mir::coverage::CodeRegion,
-    &'tcx ty::List<ty::BoundVariableKind>
+    &'tcx ty::List<ty::BoundVariableKind>,
+    &'tcx ty::List<ty::Predicate<'tcx>>,
 }
 
 #[macro_export]
@@ -519,6 +546,8 @@ macro_rules! impl_binder_encode_decode {
 impl_binder_encode_decode! {
     &'tcx ty::List<Ty<'tcx>>,
     ty::FnSig<'tcx>,
+    ty::Predicate<'tcx>,
+    ty::TraitPredicate<'tcx>,
     ty::ExistentialPredicate<'tcx>,
     ty::TraitRef<'tcx>,
     Vec<ty::GeneratorInteriorTypeCause<'tcx>>,

compiler/rustc_middle/src/ty/context.rs

@@ -1306,6 +1306,7 @@ impl<'tcx> TyCtxt<'tcx> {
                     Placeholder,
                     Generator,
                     GeneratorWitness,
+                    GeneratorWitnessMIR,
                     Dynamic,
                     Closure,
                     Tuple,
@@ -1815,6 +1816,11 @@ impl<'tcx> TyCtxt<'tcx> {
         self.mk_mut_ref(self.lifetimes.re_erased, context_ty)
     }
 
+    #[inline]
+    pub fn mk_generator_witness_mir(self, id: DefId, substs: SubstsRef<'tcx>) -> Ty<'tcx> {
+        self.mk_ty(GeneratorWitnessMIR(id, substs))
+    }
+
     #[inline]
     pub fn mk_ty_var(self, v: TyVid) -> Ty<'tcx> {
         self.mk_ty_infer(TyVar(v))

compiler/rustc_middle/src/ty/error.rs

@@ -325,7 +325,8 @@ impl<'tcx> Ty<'tcx> {
             ty::Dynamic(..) => "trait object".into(),
             ty::Closure(..) => "closure".into(),
             ty::Generator(def_id, ..) => tcx.generator_kind(def_id).unwrap().descr().into(),
-            ty::GeneratorWitness(..) => "generator witness".into(),
+            ty::GeneratorWitness(..) |
+            ty::GeneratorWitnessMIR(..) => "generator witness".into(),
             ty::Tuple(..) => "tuple".into(),
             ty::Infer(ty::TyVar(_)) => "inferred type".into(),
             ty::Infer(ty::IntVar(_)) => "integer".into(),
@@ -373,7 +374,7 @@ impl<'tcx> Ty<'tcx> {
             ty::Dynamic(..) => "trait object".into(),
             ty::Closure(..) => "closure".into(),
             ty::Generator(def_id, ..) => tcx.generator_kind(def_id).unwrap().descr().into(),
-            ty::GeneratorWitness(..) => "generator witness".into(),
+            ty::GeneratorWitness(..) | ty::GeneratorWitnessMIR(..) => "generator witness".into(),
             ty::Tuple(..) => "tuple".into(),
             ty::Placeholder(..) => "higher-ranked type".into(),
             ty::Bound(..) => "bound type variable".into(),

compiler/rustc_middle/src/ty/fast_reject.rs

@@ -32,6 +32,7 @@ pub enum SimplifiedType {
     ClosureSimplifiedType(DefId),
     GeneratorSimplifiedType(DefId),
     GeneratorWitnessSimplifiedType(usize),
+    GeneratorWitnessMIRSimplifiedType(DefId),
     FunctionSimplifiedType(usize),
     PlaceholderSimplifiedType,
 }
@@ -108,6 +109,7 @@ pub fn simplify_type<'tcx>(
         ty::FnDef(def_id, _) | ty::Closure(def_id, _) => Some(ClosureSimplifiedType(def_id)),
         ty::Generator(def_id, _, _) => Some(GeneratorSimplifiedType(def_id)),
         ty::GeneratorWitness(tys) => Some(GeneratorWitnessSimplifiedType(tys.skip_binder().len())),
+        ty::GeneratorWitnessMIR(def_id, _) => Some(GeneratorWitnessMIRSimplifiedType(def_id)),
         ty::Never => Some(NeverSimplifiedType),
         ty::Tuple(tys) => Some(TupleSimplifiedType(tys.len())),
         ty::FnPtr(f) => Some(FunctionSimplifiedType(f.skip_binder().inputs().len())),
@@ -139,7 +141,8 @@ impl SimplifiedType {
             | ForeignSimplifiedType(d)
             | TraitSimplifiedType(d)
             | ClosureSimplifiedType(d)
-            | GeneratorSimplifiedType(d) => Some(d),
+            | GeneratorSimplifiedType(d)
+            | GeneratorWitnessMIRSimplifiedType(d) => Some(d),
             _ => None,
         }
     }
@@ -208,6 +211,7 @@ impl DeepRejectCtxt {
             | ty::Closure(..)
             | ty::Generator(..)
             | ty::GeneratorWitness(..)
+            | ty::GeneratorWitnessMIR(..)
             | ty::Placeholder(..)
             | ty::Bound(..)
             | ty::Infer(_) => bug!("unexpected impl_ty: {impl_ty}"),
@@ -306,7 +310,7 @@ impl DeepRejectCtxt {
 
             ty::Error(_) => true,
 
-            ty::GeneratorWitness(..) | ty::Bound(..) => {
+            ty::GeneratorWitness(..) | ty::GeneratorWitnessMIR(..) | ty::Bound(..) => {
                 bug!("unexpected obligation type: {:?}", obligation_ty)
             }
         }

compiler/rustc_middle/src/ty/flags.rs

@@ -125,6 +125,16 @@ impl FlagComputation {
                 self.bound_computation(ts, |flags, ts| flags.add_tys(ts));
             }
 
+            &ty::GeneratorWitnessMIR(_, ref substs) => {
+                let should_remove_further_specializable =
+                    !self.flags.contains(TypeFlags::STILL_FURTHER_SPECIALIZABLE);
+                self.add_substs(substs);
+                if should_remove_further_specializable {
+                    self.flags -= TypeFlags::STILL_FURTHER_SPECIALIZABLE;
+                }
+                self.add_flags(TypeFlags::HAS_TY_GENERATOR);
+            }
+
             &ty::Closure(_, substs) => {
                 let substs = substs.as_closure();
                 let should_remove_further_specializable =

compiler/rustc_middle/src/ty/layout.rs

@@ -645,6 +645,7 @@ where
                 | ty::Never
                 | ty::FnDef(..)
                 | ty::GeneratorWitness(..)
+                | ty::GeneratorWitnessMIR(..)
                 | ty::Foreign(..)
                 | ty::Dynamic(_, _, ty::Dyn) => {
                     bug!("TyAndLayout::field({:?}): not applicable", this)

compiler/rustc_middle/src/ty/opaque_types.rs

@@ -3,6 +3,7 @@ use crate::ty::fold::{TypeFolder, TypeSuperFoldable};
 use crate::ty::subst::{GenericArg, GenericArgKind};
 use crate::ty::{self, Ty, TyCtxt, TypeFoldable};
 use rustc_data_structures::fx::FxHashMap;
+use rustc_span::def_id::DefId;
 use rustc_span::Span;
 
 /// Converts generic params of a TypeFoldable from one
@@ -47,6 +48,47 @@ impl<'tcx> ReverseMapper<'tcx> {
         assert!(!self.do_not_error);
         kind.fold_with(self)
     }
+
+    fn fold_closure_substs(
+        &mut self,
+        def_id: DefId,
+        substs: ty::SubstsRef<'tcx>,
+    ) -> ty::SubstsRef<'tcx> {
+        // I am a horrible monster and I pray for death. When
+        // we encounter a closure here, it is always a closure
+        // from within the function that we are currently
+        // type-checking -- one that is now being encapsulated
+        // in an opaque type. Ideally, we would
+        // go through the types/lifetimes that it references
+        // and treat them just like we would any other type,
+        // which means we would error out if we find any
+        // reference to a type/region that is not in the
+        // "reverse map".
+        //
+        // **However,** in the case of closures, there is a
+        // somewhat subtle (read: hacky) consideration. The
+        // problem is that our closure types currently include
+        // all the lifetime parameters declared on the
+        // enclosing function, even if they are unused by the
+        // closure itself. We can't readily filter them out,
+        // so here we replace those values with `'empty`. This
+        // can't really make a difference to the rest of the
+        // compiler; those regions are ignored for the
+        // outlives relation, and hence don't affect trait
+        // selection or auto traits, and they are erased
+        // during codegen.
+
+        let generics = self.tcx.generics_of(def_id);
+        self.tcx.mk_substs(substs.iter().enumerate().map(|(index, kind)| {
+            if index < generics.parent_count {
+                // Accommodate missing regions in the parent kinds...
+                self.fold_kind_no_missing_regions_error(kind)
+            } else {
+                // ...but not elsewhere.
+                self.fold_kind_normally(kind)
+            }
+        }))
+    }
 }
 
 impl<'tcx> TypeFolder<'tcx> for ReverseMapper<'tcx> {
@@ -104,59 +146,20 @@ impl<'tcx> TypeFolder<'tcx> for ReverseMapper<'tcx> {
     fn fold_ty(&mut self, ty: Ty<'tcx>) -> Ty<'tcx> {
         match *ty.kind() {
             ty::Closure(def_id, substs) => {
-                // I am a horrible monster and I pray for death. When
+                let substs = self.fold_closure_substs(def_id, substs);
-                // we encounter a closure here, it is always a closure
-                // from within the function that we are currently
-                // type-checking -- one that is now being encapsulated
-                // in an opaque type. Ideally, we would
-                // go through the types/lifetimes that it references
-                // and treat them just like we would any other type,
-                // which means we would error out if we find any
-                // reference to a type/region that is not in the
-                // "reverse map".
-                //
-                // **However,** in the case of closures, there is a
-                // somewhat subtle (read: hacky) consideration. The
-                // problem is that our closure types currently include
-                // all the lifetime parameters declared on the
-                // enclosing function, even if they are unused by the
-                // closure itself. We can't readily filter them out,
-                // so here we replace those values with `'empty`. This
-                // can't really make a difference to the rest of the
-                // compiler; those regions are ignored for the
-                // outlives relation, and hence don't affect trait
-                // selection or auto traits, and they are erased
-                // during codegen.
-
-                let generics = self.tcx.generics_of(def_id);
-                let substs = self.tcx.mk_substs(substs.iter().enumerate().map(|(index, kind)| {
-                    if index < generics.parent_count {
-                        // Accommodate missing regions in the parent kinds...
-                        self.fold_kind_no_missing_regions_error(kind)
-                    } else {
-                        // ...but not elsewhere.
-                        self.fold_kind_normally(kind)
-                    }
-                }));
-
                 self.tcx.mk_closure(def_id, substs)
             }
 
             ty::Generator(def_id, substs, movability) => {
-                let generics = self.tcx.generics_of(def_id);
+                let substs = self.fold_closure_substs(def_id, substs);
-                let substs = self.tcx.mk_substs(substs.iter().enumerate().map(|(index, kind)| {
-                    if index < generics.parent_count {
-                        // Accommodate missing regions in the parent kinds...
-                        self.fold_kind_no_missing_regions_error(kind)
-                    } else {
-                        // ...but not elsewhere.
-                        self.fold_kind_normally(kind)
-                    }
-                }));
-
                 self.tcx.mk_generator(def_id, substs, movability)
             }
 
+            ty::GeneratorWitnessMIR(def_id, substs) => {
+                let substs = self.fold_closure_substs(def_id, substs);
+                self.tcx.mk_generator_witness_mir(def_id, substs)
+            }
+
             ty::Param(param) => {
                 // Look it up in the substitution list.
                 match self.map.get(&ty.into()).map(|k| k.unpack()) {

compiler/rustc_middle/src/ty/parameterized.rs

@@ -117,6 +117,7 @@ macro_rules! parameterized_over_tcx {
 parameterized_over_tcx! {
     crate::middle::exported_symbols::ExportedSymbol,
     crate::mir::Body,
+    crate::mir::GeneratorLayout,
     ty::Ty,
     ty::FnSig,
     ty::GenericPredicates,

compiler/rustc_middle/src/ty/print/mod.rs

@@ -265,6 +265,7 @@ fn characteristic_def_id_of_type_cached<'a>(
         ty::FnDef(def_id, _)
         | ty::Closure(def_id, _)
         | ty::Generator(def_id, _, _)
+        | ty::GeneratorWitnessMIR(def_id, _)
         | ty::Foreign(def_id) => Some(def_id),
 
         ty::Bool

compiler/rustc_middle/src/ty/print/pretty.rs

@@ -811,6 +811,28 @@ pub trait PrettyPrinter<'tcx>:
             ty::GeneratorWitness(types) => {
                 p!(in_binder(&types));
             }
+            ty::GeneratorWitnessMIR(did, substs) => {
+                p!(write("["));
+                if !self.tcx().sess.verbose() {
+                    p!("generator witness");
+                    // FIXME(eddyb) should use `def_span`.
+                    if let Some(did) = did.as_local() {
+                        let span = self.tcx().def_span(did);
+                        p!(write(
+                            "@{}",
+                            // This may end up in stderr diagnostics but it may also be emitted
+                            // into MIR. Hence we use the remapped path if available
+                            self.tcx().sess.source_map().span_to_embeddable_string(span)
+                        ));
+                    } else {
+                        p!(write("@"), print_def_path(did, substs));
+                    }
+                } else {
+                    p!(print_def_path(did, substs));
+                }
+
+                p!("]")
+            }
             ty::Closure(did, substs) => {
                 p!(write("["));
                 if !self.should_print_verbose() {

compiler/rustc_middle/src/ty/relate.rs

@@ -473,6 +473,16 @@ pub fn super_relate_tys<'tcx, R: TypeRelation<'tcx>>(
             Ok(tcx.mk_generator_witness(types))
         }
 
+        (&ty::GeneratorWitnessMIR(a_id, a_substs), &ty::GeneratorWitnessMIR(b_id, b_substs))
+            if a_id == b_id =>
+        {
+            // All GeneratorWitness types with the same id represent
+            // the (anonymous) type of the same generator expression. So
+            // all of their regions should be equated.
+            let substs = relation.relate(a_substs, b_substs)?;
+            Ok(tcx.mk_generator_witness_mir(a_id, substs))
+        }
+
         (&ty::Closure(a_id, a_substs), &ty::Closure(b_id, b_substs)) if a_id == b_id => {
             // All Closure types with the same id represent
             // the (anonymous) type of the same closure expression. So

compiler/rustc_middle/src/ty/structural_impls.rs

@@ -201,6 +201,7 @@ TrivialTypeTraversalAndLiftImpls! {
     bool,
     usize,
     ::rustc_target::abi::VariantIdx,
+    u16,
     u32,
     u64,
     String,
@@ -655,6 +656,9 @@ impl<'tcx> TypeSuperFoldable<'tcx> for Ty<'tcx> {
                 ty::Generator(did, substs.try_fold_with(folder)?, movability)
             }
             ty::GeneratorWitness(types) => ty::GeneratorWitness(types.try_fold_with(folder)?),
+            ty::GeneratorWitnessMIR(did, substs) => {
+                ty::GeneratorWitnessMIR(did, substs.try_fold_with(folder)?)
+            }
             ty::Closure(did, substs) => ty::Closure(did, substs.try_fold_with(folder)?),
             ty::Alias(kind, data) => ty::Alias(kind, data.try_fold_with(folder)?),
 
@@ -700,6 +704,7 @@ impl<'tcx> TypeSuperVisitable<'tcx> for Ty<'tcx> {
             }
             ty::Generator(_did, ref substs, _) => substs.visit_with(visitor),
             ty::GeneratorWitness(ref types) => types.visit_with(visitor),
+            ty::GeneratorWitnessMIR(_did, ref substs) => substs.visit_with(visitor),
             ty::Closure(_did, ref substs) => substs.visit_with(visitor),
             ty::Alias(_, ref data) => data.visit_with(visitor),
 

compiler/rustc_middle/src/ty/sty.rs

@@ -571,9 +571,9 @@ impl<'tcx> GeneratorSubsts<'tcx> {
     ) -> impl Iterator<Item = impl Iterator<Item = Ty<'tcx>> + Captures<'tcx>> {
         let layout = tcx.generator_layout(def_id).unwrap();
         layout.variant_fields.iter().map(move |variant| {
-            variant
+            variant.iter().map(move |field| {
-                .iter()
+                ty::EarlyBinder(layout.field_tys[*field].ty).subst(tcx, self.substs)
-                .map(move |field| ty::EarlyBinder(layout.field_tys[*field]).subst(tcx, self.substs))
+            })
         })
     }
 
@@ -2175,6 +2175,7 @@ impl<'tcx> Ty<'tcx> {
             | ty::Dynamic(..)
             | ty::Closure(..)
             | ty::GeneratorWitness(..)
+            | ty::GeneratorWitnessMIR(..)
             | ty::Never
             | ty::Tuple(_)
             | ty::Error(_)
@@ -2210,6 +2211,7 @@ impl<'tcx> Ty<'tcx> {
             | ty::Ref(..)
             | ty::Generator(..)
             | ty::GeneratorWitness(..)
+            | ty::GeneratorWitnessMIR(..)
             | ty::Array(..)
             | ty::Closure(..)
             | ty::Never
@@ -2296,6 +2298,7 @@ impl<'tcx> Ty<'tcx> {
             | ty::Ref(..)
             | ty::Generator(..)
             | ty::GeneratorWitness(..)
+            | ty::GeneratorWitnessMIR(..)
             | ty::Array(..)
             | ty::Closure(..)
             | ty::Never
@@ -2360,7 +2363,7 @@ impl<'tcx> Ty<'tcx> {
             // anything with custom metadata it might be more complicated.
             ty::Ref(_, _, hir::Mutability::Not) | ty::RawPtr(..) => false,
 
-            ty::Generator(..) | ty::GeneratorWitness(..) => false,
+            ty::Generator(..) | ty::GeneratorWitness(..) | ty::GeneratorWitnessMIR(..) => false,
 
             // Might be, but not "trivial" so just giving the safe answer.
             ty::Adt(..) | ty::Closure(..) => false,

compiler/rustc_middle/src/ty/typeck_results.rs

@@ -1,6 +1,7 @@
 use crate::{
     hir::place::Place as HirPlace,
     infer::canonical::Canonical,
+    traits::ObligationCause,
     ty::{
         self, tls, BindingMode, BoundVar, CanonicalPolyFnSig, ClosureSizeProfileData,
         GenericArgKind, InternalSubsts, SubstsRef, Ty, UserSubsts,
@@ -193,6 +194,11 @@ pub struct TypeckResults<'tcx> {
     /// that are live across the yield of this generator (if a generator).
     pub generator_interior_types: ty::Binder<'tcx, Vec<GeneratorInteriorTypeCause<'tcx>>>,
 
+    /// Stores the predicates that apply on generator witness types.
+    /// formatting modified file tests/ui/generator/retain-resume-ref.rs
+    pub generator_interior_predicates:
+        FxHashMap<LocalDefId, Vec<(ty::Predicate<'tcx>, ObligationCause<'tcx>)>>,
+
     /// We sometimes treat byte string literals (which are of type `&[u8; N]`)
     /// as `&[u8]`, depending on the pattern in which they are used.
     /// This hashset records all instances where we behave
@@ -271,6 +277,7 @@ impl<'tcx> TypeckResults<'tcx> {
             closure_fake_reads: Default::default(),
             rvalue_scopes: Default::default(),
             generator_interior_types: ty::Binder::dummy(Default::default()),
+            generator_interior_predicates: Default::default(),
             treat_byte_string_as_slice: Default::default(),
             closure_size_eval: Default::default(),
         }

compiler/rustc_middle/src/ty/util.rs

@@ -615,6 +615,36 @@ impl<'tcx> TyCtxt<'tcx> {
         }
     }
 
+    /// Return the set of types that should be taken into accound when checking
+    /// trait bounds on a generator's internal state.
+    pub fn generator_hidden_types(
+        self,
+        def_id: DefId,
+    ) -> impl Iterator<Item = ty::EarlyBinder<Ty<'tcx>>> {
+        let generator_layout = &self.mir_generator_witnesses(def_id);
+        generator_layout
+            .field_tys
+            .iter()
+            .filter(|decl| !decl.ignore_for_traits)
+            .map(|decl| ty::EarlyBinder(decl.ty))
+    }
+
+    /// Normalizes all opaque types in the given value, replacing them
+    /// with their underlying types.
+    pub fn expand_opaque_types(self, val: Ty<'tcx>) -> Ty<'tcx> {
+        let mut visitor = OpaqueTypeExpander {
+            seen_opaque_tys: FxHashSet::default(),
+            expanded_cache: FxHashMap::default(),
+            primary_def_id: None,
+            found_recursion: false,
+            found_any_recursion: false,
+            check_recursion: false,
+            expand_generators: false,
+            tcx: self,
+        };
+        val.fold_with(&mut visitor)
+    }
+
     /// Expands the given impl trait type, stopping if the type is recursive.
     #[instrument(skip(self), level = "debug", ret)]
     pub fn try_expand_impl_trait_type(
@@ -629,6 +659,7 @@ impl<'tcx> TyCtxt<'tcx> {
             found_recursion: false,
             found_any_recursion: false,
             check_recursion: true,
+            expand_generators: true,
             tcx: self,
         };
 
@@ -741,6 +772,7 @@ struct OpaqueTypeExpander<'tcx> {
     primary_def_id: Option<DefId>,
     found_recursion: bool,
     found_any_recursion: bool,
+    expand_generators: bool,
     /// Whether or not to check for recursive opaque types.
     /// This is `true` when we're explicitly checking for opaque type
     /// recursion, and 'false' otherwise to avoid unnecessary work.
@@ -777,6 +809,37 @@ impl<'tcx> OpaqueTypeExpander<'tcx> {
             None
         }
     }
+
+    fn expand_generator(&mut self, def_id: DefId, substs: SubstsRef<'tcx>) -> Option<Ty<'tcx>> {
+        if self.found_any_recursion {
+            return None;
+        }
+        let substs = substs.fold_with(self);
+        if !self.check_recursion || self.seen_opaque_tys.insert(def_id) {
+            let expanded_ty = match self.expanded_cache.get(&(def_id, substs)) {
+                Some(expanded_ty) => *expanded_ty,
+                None => {
+                    for bty in self.tcx.generator_hidden_types(def_id) {
+                        let hidden_ty = bty.subst(self.tcx, substs);
+                        self.fold_ty(hidden_ty);
+                    }
+                    let expanded_ty = self.tcx.mk_generator_witness_mir(def_id, substs);
+                    self.expanded_cache.insert((def_id, substs), expanded_ty);
+                    expanded_ty
+                }
+            };
+            if self.check_recursion {
+                self.seen_opaque_tys.remove(&def_id);
+            }
+            Some(expanded_ty)
+        } else {
+            // If another opaque type that we contain is recursive, then it
+            // will report the error, so we don't have to.
+            self.found_any_recursion = true;
+            self.found_recursion = def_id == *self.primary_def_id.as_ref().unwrap();
+            None
+        }
+    }
 }
 
 impl<'tcx> TypeFolder<'tcx> for OpaqueTypeExpander<'tcx> {
@@ -785,14 +848,20 @@ impl<'tcx> TypeFolder<'tcx> for OpaqueTypeExpander<'tcx> {
     }
 
     fn fold_ty(&mut self, t: Ty<'tcx>) -> Ty<'tcx> {
-        if let ty::Alias(ty::Opaque, ty::AliasTy { def_id, substs, .. }) = *t.kind() {
+        let mut t = if let ty::Alias(ty::Opaque, ty::AliasTy { def_id, substs, .. }) = *t.kind() {
             self.expand_opaque_ty(def_id, substs).unwrap_or(t)
-        } else if t.has_opaque_types() {
+        } else if t.has_opaque_types() || t.has_generators() {
             t.super_fold_with(self)
         } else {
             t
+        };
+        if self.expand_generators {
+            if let ty::GeneratorWitnessMIR(def_id, substs) = *t.kind() {
+                t = self.expand_generator(def_id, substs).unwrap_or(t);
             }
         }
+        t
+    }
 }
 
 impl<'tcx> Ty<'tcx> {
@@ -896,6 +965,7 @@ impl<'tcx> Ty<'tcx> {
             | ty::Foreign(_)
             | ty::Generator(..)
             | ty::GeneratorWitness(_)
+            | ty::GeneratorWitnessMIR(..)
             | ty::Infer(_)
             | ty::Alias(..)
             | ty::Param(_)
@@ -935,6 +1005,7 @@ impl<'tcx> Ty<'tcx> {
             | ty::Foreign(_)
             | ty::Generator(..)
             | ty::GeneratorWitness(_)
+            | ty::GeneratorWitnessMIR(..)
             | ty::Infer(_)
             | ty::Alias(..)
             | ty::Param(_)
@@ -1062,7 +1133,10 @@ impl<'tcx> Ty<'tcx> {
                 false
             }
 
-            ty::Foreign(_) | ty::GeneratorWitness(..) | ty::Error(_) => false,
+            ty::Foreign(_)
+            | ty::GeneratorWitness(..)
+            | ty::GeneratorWitnessMIR(..)
+            | ty::Error(_) => false,
         }
     }
 
@@ -1158,6 +1232,7 @@ pub fn needs_drop_components<'tcx>(
         | ty::FnPtr(_)
         | ty::Char
         | ty::GeneratorWitness(..)
+        | ty::GeneratorWitnessMIR(..)
         | ty::RawPtr(_)
         | ty::Ref(..)
         | ty::Str => Ok(SmallVec::new()),
@@ -1228,7 +1303,11 @@ pub fn is_trivially_const_drop(ty: Ty<'_>) -> bool {
 
         // Not trivial because they have components, and instead of looking inside,
         // we'll just perform trait selection.
-        ty::Closure(..) | ty::Generator(..) | ty::GeneratorWitness(_) | ty::Adt(..) => false,
+        ty::Closure(..)
+        | ty::Generator(..)
+        | ty::GeneratorWitness(_)
+        | ty::GeneratorWitnessMIR(..)
+        | ty::Adt(..) => false,
 
         ty::Array(ty, _) | ty::Slice(ty) => is_trivially_const_drop(ty),
 
@@ -1289,6 +1368,7 @@ pub fn reveal_opaque_types_in_bounds<'tcx>(
         found_recursion: false,
         found_any_recursion: false,
         check_recursion: false,
+        expand_generators: false,
         tcx,
     };
     val.fold_with(&mut visitor)

compiler/rustc_middle/src/ty/visit.rs

@@ -100,6 +100,9 @@ pub trait TypeVisitable<'tcx>: fmt::Debug + Clone {
     fn has_opaque_types(&self) -> bool {
         self.has_type_flags(TypeFlags::HAS_TY_OPAQUE)
     }
+    fn has_generators(&self) -> bool {
+        self.has_type_flags(TypeFlags::HAS_TY_GENERATOR)
+    }
     fn references_error(&self) -> bool {
         self.has_type_flags(TypeFlags::HAS_ERROR)
     }

compiler/rustc_middle/src/ty/walk.rs

@@ -190,6 +190,7 @@ fn push_inner<'tcx>(stack: &mut TypeWalkerStack<'tcx>, parent: GenericArg<'tcx>)
             ty::Adt(_, substs)
             | ty::Closure(_, substs)
             | ty::Generator(_, substs, _)
+            | ty::GeneratorWitnessMIR(_, substs)
             | ty::FnDef(_, substs) => {
                 stack.extend(substs.iter().rev());
             }

compiler/rustc_mir_build/src/build/matches/mod.rs

@@ -1747,8 +1747,10 @@ impl<'a, 'tcx> Builder<'a, 'tcx> {
                 };
                 let fake_borrow_deref_ty = matched_place.ty(&self.local_decls, tcx).ty;
                 let fake_borrow_ty = tcx.mk_imm_ref(tcx.lifetimes.re_erased, fake_borrow_deref_ty);
-                let fake_borrow_temp =
+                let mut fake_borrow_temp = LocalDecl::new(fake_borrow_ty, temp_span);
-                    self.local_decls.push(LocalDecl::new(fake_borrow_ty, temp_span));
+                fake_borrow_temp.internal = self.local_decls[matched_place.local].internal;
+                fake_borrow_temp.local_info = Some(Box::new(LocalInfo::FakeBorrow));
+                let fake_borrow_temp = self.local_decls.push(fake_borrow_temp);
 
                 (matched_place, fake_borrow_temp)
             })

compiler/rustc_mir_transform/src/generator.rs

@@ -54,7 +54,8 @@ use crate::deref_separator::deref_finder;
 use crate::simplify;
 use crate::util::expand_aggregate;
 use crate::MirPass;
-use rustc_data_structures::fx::FxHashMap;
+use rustc_data_structures::fx::{FxHashMap, FxHashSet};
+use rustc_errors::pluralize;
 use rustc_hir as hir;
 use rustc_hir::lang_items::LangItem;
 use rustc_hir::GeneratorKind;
@@ -70,6 +71,9 @@ use rustc_mir_dataflow::impls::{
 };
 use rustc_mir_dataflow::storage::always_storage_live_locals;
 use rustc_mir_dataflow::{self, Analysis};
+use rustc_span::def_id::DefId;
+use rustc_span::symbol::sym;
+use rustc_span::Span;
 use rustc_target::abi::VariantIdx;
 use rustc_target::spec::PanicStrategy;
 use std::{iter, ops};
@@ -854,7 +858,7 @@ fn sanitize_witness<'tcx>(
     body: &Body<'tcx>,
     witness: Ty<'tcx>,
     upvars: Vec<Ty<'tcx>>,
-    saved_locals: &GeneratorSavedLocals,
+    layout: &GeneratorLayout<'tcx>,
 ) {
     let did = body.source.def_id();
     let param_env = tcx.param_env(did);
@@ -873,31 +877,36 @@ fn sanitize_witness<'tcx>(
         }
     };
 
-    for (local, decl) in body.local_decls.iter_enumerated() {
+    let mut mismatches = Vec::new();
-        // Ignore locals which are internal or not saved between yields.
+    for fty in &layout.field_tys {
-        if !saved_locals.contains(local) || decl.internal {
+        if fty.ignore_for_traits {
             continue;
         }
-        let decl_ty = tcx.normalize_erasing_regions(param_env, decl.ty);
+        let decl_ty = tcx.normalize_erasing_regions(param_env, fty.ty);
 
         // Sanity check that typeck knows about the type of locals which are
         // live across a suspension point
         if !allowed.contains(&decl_ty) && !allowed_upvars.contains(&decl_ty) {
+            mismatches.push(decl_ty);
+        }
+    }
+
+    if !mismatches.is_empty() {
         span_bug!(
             body.span,
-                "Broken MIR: generator contains type {} in MIR, \
+            "Broken MIR: generator contains type {:?} in MIR, \
                        but typeck only knows about {} and {:?}",
-                decl_ty,
+            mismatches,
             allowed,
             allowed_upvars
         );
     }
 }
-}
 
 fn compute_layout<'tcx>(
+    tcx: TyCtxt<'tcx>,
     liveness: LivenessInfo,
-    body: &mut Body<'tcx>,
+    body: &Body<'tcx>,
 ) -> (
     FxHashMap<Local, (Ty<'tcx>, VariantIdx, usize)>,
     GeneratorLayout<'tcx>,
@@ -915,9 +924,33 @@ fn compute_layout<'tcx>(
     let mut locals = IndexVec::<GeneratorSavedLocal, _>::new();
     let mut tys = IndexVec::<GeneratorSavedLocal, _>::new();
     for (saved_local, local) in saved_locals.iter_enumerated() {
-        locals.push(local);
-        tys.push(body.local_decls[local].ty);
         debug!("generator saved local {:?} => {:?}", saved_local, local);
+
+        locals.push(local);
+        let decl = &body.local_decls[local];
+        debug!(?decl);
+
+        let ignore_for_traits = if tcx.sess.opts.unstable_opts.drop_tracking_mir {
+            match decl.local_info {
+                // Do not include raw pointers created from accessing `static` items, as those could
+                // well be re-created by another access to the same static.
+                Some(box LocalInfo::StaticRef { is_thread_local, .. }) => !is_thread_local,
+                // Fake borrows are only read by fake reads, so do not have any reality in
+                // post-analysis MIR.
+                Some(box LocalInfo::FakeBorrow) => true,
+                _ => false,
+            }
+        } else {
+            // FIXME(#105084) HIR-based drop tracking does not account for all the temporaries that
+            // MIR building may introduce. This leads to wrongly ignored types, but this is
+            // necessary for internal consistency and to avoid ICEs.
+            decl.internal
+        };
+        let decl =
+            GeneratorSavedTy { ty: decl.ty, source_info: decl.source_info, ignore_for_traits };
+        debug!(?decl);
+
+        tys.push(decl);
     }
 
     // Leave empty variants for the UNRESUMED, RETURNED, and POISONED states.
@@ -947,7 +980,7 @@ fn compute_layout<'tcx>(
             // just use the first one here. That's fine; fields do not move
             // around inside generators, so it doesn't matter which variant
             // index we access them by.
-            remap.entry(locals[saved_local]).or_insert((tys[saved_local], variant_index, idx));
+            remap.entry(locals[saved_local]).or_insert((tys[saved_local].ty, variant_index, idx));
         }
         variant_fields.push(fields);
         variant_source_info.push(source_info_at_suspension_points[suspension_point_idx]);
@@ -957,6 +990,7 @@ fn compute_layout<'tcx>(
 
     let layout =
         GeneratorLayout { field_tys: tys, variant_fields, variant_source_info, storage_conflicts };
+    debug!(?layout);
 
     (remap, layout, storage_liveness)
 }
@@ -1351,6 +1385,52 @@ fn create_cases<'tcx>(
         .collect()
 }
 
+#[instrument(level = "debug", skip(tcx), ret)]
+pub(crate) fn mir_generator_witnesses<'tcx>(
+    tcx: TyCtxt<'tcx>,
+    def_id: DefId,
+) -> GeneratorLayout<'tcx> {
+    let def_id = def_id.expect_local();
+
+    let (body, _) = tcx.mir_promoted(ty::WithOptConstParam::unknown(def_id));
+    let body = body.borrow();
+    let body = &*body;
+
+    // The first argument is the generator type passed by value
+    let gen_ty = body.local_decls[ty::CAPTURE_STRUCT_LOCAL].ty;
+
+    // Get the interior types and substs which typeck computed
+    let (upvars, interior, movable) = match *gen_ty.kind() {
+        ty::Generator(_, substs, movability) => {
+            let substs = substs.as_generator();
+            (
+                substs.upvar_tys().collect::<Vec<_>>(),
+                substs.witness(),
+                movability == hir::Movability::Movable,
+            )
+        }
+        _ => span_bug!(body.span, "unexpected generator type {}", gen_ty),
+    };
+
+    // When first entering the generator, move the resume argument into its new local.
+    let always_live_locals = always_storage_live_locals(&body);
+
+    let liveness_info = locals_live_across_suspend_points(tcx, body, &always_live_locals, movable);
+
+    // Extract locals which are live across suspension point into `layout`
+    // `remap` gives a mapping from local indices onto generator struct indices
+    // `storage_liveness` tells us which locals have live storage at suspension points
+    let (_, generator_layout, _) = compute_layout(tcx, liveness_info, body);
+
+    if tcx.sess.opts.unstable_opts.drop_tracking_mir {
+        check_suspend_tys(tcx, &generator_layout, &body);
+    } else {
+        sanitize_witness(tcx, body, interior, upvars, &generator_layout);
+    }
+
+    generator_layout
+}
+
 impl<'tcx> MirPass<'tcx> for StateTransform {
     fn run_pass(&self, tcx: TyCtxt<'tcx>, body: &mut Body<'tcx>) {
         let Some(yield_ty) = body.yield_ty() else {
@@ -1363,16 +1443,11 @@ impl<'tcx> MirPass<'tcx> for StateTransform {
         // The first argument is the generator type passed by value
         let gen_ty = body.local_decls.raw[1].ty;
 
-        // Get the interior types and substs which typeck computed
+        // Get the discriminant type and substs which typeck computed
-        let (upvars, interior, discr_ty, movable) = match *gen_ty.kind() {
+        let (discr_ty, movable) = match *gen_ty.kind() {
             ty::Generator(_, substs, movability) => {
                 let substs = substs.as_generator();
-                (
+                (substs.discr_ty(tcx), movability == hir::Movability::Movable)
-                    substs.upvar_tys().collect(),
-                    substs.witness(),
-                    substs.discr_ty(tcx),
-                    movability == hir::Movability::Movable,
-                )
             }
             _ => {
                 tcx.sess
@@ -1434,8 +1509,6 @@ impl<'tcx> MirPass<'tcx> for StateTransform {
         let liveness_info =
             locals_live_across_suspend_points(tcx, body, &always_live_locals, movable);
 
-        sanitize_witness(tcx, body, interior, upvars, &liveness_info.saved_locals);
-
         if tcx.sess.opts.unstable_opts.validate_mir {
             let mut vis = EnsureGeneratorFieldAssignmentsNeverAlias {
                 assigned_local: None,
@@ -1449,7 +1522,7 @@ impl<'tcx> MirPass<'tcx> for StateTransform {
         // Extract locals which are live across suspension point into `layout`
         // `remap` gives a mapping from local indices onto generator struct indices
         // `storage_liveness` tells us which locals have live storage at suspension points
-        let (remap, layout, storage_liveness) = compute_layout(liveness_info, body);
+        let (remap, layout, storage_liveness) = compute_layout(tcx, liveness_info, body);
 
         let can_return = can_return(tcx, body, tcx.param_env(body.source.def_id()));
 
@@ -1631,3 +1704,212 @@ impl<'tcx> Visitor<'tcx> for EnsureGeneratorFieldAssignmentsNeverAlias<'_> {
         }
     }
 }
+
+fn check_suspend_tys<'tcx>(tcx: TyCtxt<'tcx>, layout: &GeneratorLayout<'tcx>, body: &Body<'tcx>) {
+    let mut linted_tys = FxHashSet::default();
+
+    // We want a user-facing param-env.
+    let param_env = tcx.param_env(body.source.def_id());
+
+    for (variant, yield_source_info) in
+        layout.variant_fields.iter().zip(&layout.variant_source_info)
+    {
+        debug!(?variant);
+        for &local in variant {
+            let decl = &layout.field_tys[local];
+            debug!(?decl);
+
+            if !decl.ignore_for_traits && linted_tys.insert(decl.ty) {
+                let Some(hir_id) = decl.source_info.scope.lint_root(&body.source_scopes) else { continue };
+
+                check_must_not_suspend_ty(
+                    tcx,
+                    decl.ty,
+                    hir_id,
+                    param_env,
+                    SuspendCheckData {
+                        source_span: decl.source_info.span,
+                        yield_span: yield_source_info.span,
+                        plural_len: 1,
+                        ..Default::default()
+                    },
+                );
+            }
+        }
+    }
+}
+
+#[derive(Default)]
+struct SuspendCheckData<'a> {
+    source_span: Span,
+    yield_span: Span,
+    descr_pre: &'a str,
+    descr_post: &'a str,
+    plural_len: usize,
+}
+
+// Returns whether it emitted a diagnostic or not
+// Note that this fn and the proceeding one are based on the code
+// for creating must_use diagnostics
+//
+// Note that this technique was chosen over things like a `Suspend` marker trait
+// as it is simpler and has precedent in the compiler
+fn check_must_not_suspend_ty<'tcx>(
+    tcx: TyCtxt<'tcx>,
+    ty: Ty<'tcx>,
+    hir_id: hir::HirId,
+    param_env: ty::ParamEnv<'tcx>,
+    data: SuspendCheckData<'_>,
+) -> bool {
+    if ty.is_unit() {
+        return false;
+    }
+
+    let plural_suffix = pluralize!(data.plural_len);
+
+    debug!("Checking must_not_suspend for {}", ty);
+
+    match *ty.kind() {
+        ty::Adt(..) if ty.is_box() => {
+            let boxed_ty = ty.boxed_ty();
+            let descr_pre = &format!("{}boxed ", data.descr_pre);
+            check_must_not_suspend_ty(
+                tcx,
+                boxed_ty,
+                hir_id,
+                param_env,
+                SuspendCheckData { descr_pre, ..data },
+            )
+        }
+        ty::Adt(def, _) => check_must_not_suspend_def(tcx, def.did(), hir_id, data),
+        // FIXME: support adding the attribute to TAITs
+        ty::Alias(ty::Opaque, ty::AliasTy { def_id: def, .. }) => {
+            let mut has_emitted = false;
+            for &(predicate, _) in tcx.explicit_item_bounds(def) {
+                // We only look at the `DefId`, so it is safe to skip the binder here.
+                if let ty::PredicateKind::Clause(ty::Clause::Trait(ref poly_trait_predicate)) =
+                    predicate.kind().skip_binder()
+                {
+                    let def_id = poly_trait_predicate.trait_ref.def_id;
+                    let descr_pre = &format!("{}implementer{} of ", data.descr_pre, plural_suffix);
+                    if check_must_not_suspend_def(
+                        tcx,
+                        def_id,
+                        hir_id,
+                        SuspendCheckData { descr_pre, ..data },
+                    ) {
+                        has_emitted = true;
+                        break;
+                    }
+                }
+            }
+            has_emitted
+        }
+        ty::Dynamic(binder, _, _) => {
+            let mut has_emitted = false;
+            for predicate in binder.iter() {
+                if let ty::ExistentialPredicate::Trait(ref trait_ref) = predicate.skip_binder() {
+                    let def_id = trait_ref.def_id;
+                    let descr_post = &format!(" trait object{}{}", plural_suffix, data.descr_post);
+                    if check_must_not_suspend_def(
+                        tcx,
+                        def_id,
+                        hir_id,
+                        SuspendCheckData { descr_post, ..data },
+                    ) {
+                        has_emitted = true;
+                        break;
+                    }
+                }
+            }
+            has_emitted
+        }
+        ty::Tuple(fields) => {
+            let mut has_emitted = false;
+            for (i, ty) in fields.iter().enumerate() {
+                let descr_post = &format!(" in tuple element {i}");
+                if check_must_not_suspend_ty(
+                    tcx,
+                    ty,
+                    hir_id,
+                    param_env,
+                    SuspendCheckData { descr_post, ..data },
+                ) {
+                    has_emitted = true;
+                }
+            }
+            has_emitted
+        }
+        ty::Array(ty, len) => {
+            let descr_pre = &format!("{}array{} of ", data.descr_pre, plural_suffix);
+            check_must_not_suspend_ty(
+                tcx,
+                ty,
+                hir_id,
+                param_env,
+                SuspendCheckData {
+                    descr_pre,
+                    plural_len: len.try_eval_usize(tcx, param_env).unwrap_or(0) as usize + 1,
+                    ..data
+                },
+            )
+        }
+        // If drop tracking is enabled, we want to look through references, since the referrent
+        // may not be considered live across the await point.
+        ty::Ref(_region, ty, _mutability) => {
+            let descr_pre = &format!("{}reference{} to ", data.descr_pre, plural_suffix);
+            check_must_not_suspend_ty(
+                tcx,
+                ty,
+                hir_id,
+                param_env,
+                SuspendCheckData { descr_pre, ..data },
+            )
+        }
+        _ => false,
+    }
+}
+
+fn check_must_not_suspend_def(
+    tcx: TyCtxt<'_>,
+    def_id: DefId,
+    hir_id: hir::HirId,
+    data: SuspendCheckData<'_>,
+) -> bool {
+    if let Some(attr) = tcx.get_attr(def_id, sym::must_not_suspend) {
+        let msg = format!(
+            "{}`{}`{} held across a suspend point, but should not be",
+            data.descr_pre,
+            tcx.def_path_str(def_id),
+            data.descr_post,
+        );
+        tcx.struct_span_lint_hir(
+            rustc_session::lint::builtin::MUST_NOT_SUSPEND,
+            hir_id,
+            data.source_span,
+            msg,
+            |lint| {
+                // add span pointing to the offending yield/await
+                lint.span_label(data.yield_span, "the value is held across this suspend point");
+
+                // Add optional reason note
+                if let Some(note) = attr.value_str() {
+                    // FIXME(guswynn): consider formatting this better
+                    lint.span_note(data.source_span, note.as_str());
+                }
+
+                // Add some quick suggestions on what to do
+                // FIXME: can `drop` work as a suggestion here as well?
+                lint.span_help(
+                    data.source_span,
+                    "consider using a block (`{ ... }`) \
+                    to shrink the value's scope, ending before the suspend point",
+                )
+            },
+        );
+
+        true
+    } else {
+        false
+    }
+}

compiler/rustc_mir_transform/src/inline.rs

@@ -947,12 +947,12 @@ impl<'tcx> Visitor<'tcx> for CostChecker<'_, 'tcx> {
                             return;
                         };
 
-                        let Some(&f_ty) = layout.field_tys.get(local) else {
+                        let Some(f_ty) = layout.field_tys.get(local) else {
                             self.validation = Err("malformed MIR");
                             return;
                         };
 
-                        f_ty
+                        f_ty.ty
                     } else {
                         let Some(f_ty) = substs.as_generator().prefix_tys().nth(f.index()) else {
                             self.validation = Err("malformed MIR");

compiler/rustc_mir_transform/src/lib.rs

@@ -123,6 +123,7 @@ pub fn provide(providers: &mut Providers) {
         mir_drops_elaborated_and_const_checked,
         mir_for_ctfe,
         mir_for_ctfe_of_const_arg,
+        mir_generator_witnesses: generator::mir_generator_witnesses,
         optimized_mir,
         is_mir_available,
         is_ctfe_mir_available: |tcx, did| is_mir_available(tcx, did),
@@ -425,6 +426,9 @@ fn mir_drops_elaborated_and_const_checked(
         return tcx.mir_drops_elaborated_and_const_checked(def);
     }
 
+    if tcx.generator_kind(def.did).is_some() {
+        tcx.ensure().mir_generator_witnesses(def.did);
+    }
     let mir_borrowck = tcx.mir_borrowck_opt_const_arg(def);
 
     let is_fn_like = tcx.def_kind(def.did).is_fn_like();

compiler/rustc_privacy/src/lib.rs

@@ -271,7 +271,8 @@ where
             | ty::FnPtr(..)
             | ty::Param(..)
             | ty::Error(_)
-            | ty::GeneratorWitness(..) => {}
+            | ty::GeneratorWitness(..)
+            | ty::GeneratorWitnessMIR(..) => {}
             ty::Bound(..) | ty::Placeholder(..) | ty::Infer(..) => {
                 bug!("unexpected type: {:?}", ty)
             }

compiler/rustc_session/src/options.rs

@@ -1290,6 +1290,8 @@ options! {
         (default: no)"),
     drop_tracking: bool = (false, parse_bool, [TRACKED],
         "enables drop tracking in generators (default: no)"),
+    drop_tracking_mir: bool = (false, parse_bool, [TRACKED],
+        "enables drop tracking on MIR in generators (default: no)"),
     dual_proc_macros: bool = (false, parse_bool, [TRACKED],
         "load proc macros for both target and host, but only link to the target (default: no)"),
     dump_dep_graph: bool = (false, parse_bool, [UNTRACKED],

compiler/rustc_symbol_mangling/src/typeid/typeid_itanium_cxx_abi.rs

@@ -640,6 +640,7 @@ fn encode_ty<'tcx>(
         ty::Bound(..)
         | ty::Error(..)
         | ty::GeneratorWitness(..)
+        | ty::GeneratorWitnessMIR(..)
         | ty::Infer(..)
         | ty::Alias(..)
         | ty::Param(..)
@@ -793,6 +794,7 @@ fn transform_ty<'tcx>(tcx: TyCtxt<'tcx>, ty: Ty<'tcx>, options: TransformTyOptio
         ty::Bound(..)
         | ty::Error(..)
         | ty::GeneratorWitness(..)
+        | ty::GeneratorWitnessMIR(..)
         | ty::Infer(..)
         | ty::Alias(..)
         | ty::Param(..)

compiler/rustc_symbol_mangling/src/v0.rs

@@ -490,6 +490,7 @@ impl<'tcx> Printer<'tcx> for &mut SymbolMangler<'tcx> {
             }
 
             ty::GeneratorWitness(_) => bug!("symbol_names: unexpected `GeneratorWitness`"),
+            ty::GeneratorWitnessMIR(..) => bug!("symbol_names: unexpected `GeneratorWitnessMIR`"),
         }
 
         // Only cache types that do not refer to an enclosing

compiler/rustc_trait_selection/src/solve/assembly.rs

@@ -331,6 +331,7 @@ impl<'tcx> EvalCtxt<'_, 'tcx> {
             | ty::Closure(..)
             | ty::Generator(..)
             | ty::GeneratorWitness(_)
+            | ty::GeneratorWitnessMIR(..)
             | ty::Never
             | ty::Tuple(_)
             | ty::Param(_)
@@ -382,6 +383,7 @@ impl<'tcx> EvalCtxt<'_, 'tcx> {
             | ty::Closure(..)
             | ty::Generator(..)
             | ty::GeneratorWitness(_)
+            | ty::GeneratorWitnessMIR(..)
             | ty::Never
             | ty::Tuple(_)
             | ty::Param(_)

compiler/rustc_trait_selection/src/solve/fulfill.rs

@@ -40,12 +40,7 @@ impl<'tcx> TraitEngine<'tcx> for FulfillmentCtxt<'tcx> {
         self.obligations.push(obligation);
     }
 
-    fn select_all_or_error(&mut self, infcx: &InferCtxt<'tcx>) -> Vec<FulfillmentError<'tcx>> {
+    fn collect_remaining_errors(&mut self) -> Vec<FulfillmentError<'tcx>> {
-        let errors = self.select_where_possible(infcx);
-        if !errors.is_empty() {
-            return errors;
-        }
-
         self.obligations
             .drain(..)
             .map(|obligation| FulfillmentError {
@@ -144,4 +139,11 @@ impl<'tcx> TraitEngine<'tcx> for FulfillmentCtxt<'tcx> {
     fn pending_obligations(&self) -> Vec<PredicateObligation<'tcx>> {
         self.obligations.clone()
     }
+
+    fn drain_unstalled_obligations(
+        &mut self,
+        _: &InferCtxt<'tcx>,
+    ) -> Vec<PredicateObligation<'tcx>> {
+        unimplemented!()
+    }
 }

compiler/rustc_trait_selection/src/solve/project_goals.rs

@@ -414,6 +414,7 @@ impl<'tcx> assembly::GoalKind<'tcx> for ProjectionPredicate<'tcx> {
                 | ty::Infer(ty::IntVar(..) | ty::FloatVar(..))
                 | ty::Generator(..)
                 | ty::GeneratorWitness(..)
+                | ty::GeneratorWitnessMIR(..)
                 | ty::Never
                 | ty::Foreign(..) => tcx.types.unit,
 

compiler/rustc_trait_selection/src/solve/trait_goals/structural_traits.rs

@@ -57,6 +57,8 @@ pub(super) fn instantiate_constituent_tys_for_auto_trait<'tcx>(
             Ok(infcx.replace_bound_vars_with_placeholders(types).to_vec())
         }
 
+        ty::GeneratorWitnessMIR(..) => todo!(),
+
         // For `PhantomData<T>`, we pass `T`.
         ty::Adt(def, substs) if def.is_phantom_data() => Ok(vec![substs.type_at(0)]),
 
@@ -88,6 +90,7 @@ pub(super) fn instantiate_constituent_tys_for_sized_trait<'tcx>(
         | ty::Ref(..)
         | ty::Generator(..)
         | ty::GeneratorWitness(..)
+        | ty::GeneratorWitnessMIR(..)
         | ty::Array(..)
         | ty::Closure(..)
         | ty::Never
@@ -173,6 +176,8 @@ pub(super) fn instantiate_constituent_tys_for_copy_clone_trait<'tcx>(
         ty::GeneratorWitness(types) => {
             Ok(infcx.replace_bound_vars_with_placeholders(types).to_vec())
         }
+
+        ty::GeneratorWitnessMIR(..) => todo!(),
     }
 }
 
@@ -215,6 +220,7 @@ pub(crate) fn extract_tupled_inputs_and_output_from_callable<'tcx>(
         | ty::Dynamic(_, _, _)
         | ty::Generator(_, _, _)
         | ty::GeneratorWitness(_)
+        | ty::GeneratorWitnessMIR(..)
         | ty::Never
         | ty::Tuple(_)
         | ty::Alias(_, _)

compiler/rustc_trait_selection/src/traits/chalk_fulfill.rs

@@ -40,15 +40,7 @@ impl<'tcx> TraitEngine<'tcx> for FulfillmentContext<'tcx> {
         self.obligations.insert(obligation);
     }
 
-    fn select_all_or_error(&mut self, infcx: &InferCtxt<'tcx>) -> Vec<FulfillmentError<'tcx>> {
+    fn collect_remaining_errors(&mut self) -> Vec<FulfillmentError<'tcx>> {
-        {
-            let errors = self.select_where_possible(infcx);
-
-            if !errors.is_empty() {
-                return errors;
-            }
-        }
-
         // any remaining obligations are errors
         self.obligations
             .iter()
@@ -143,6 +135,13 @@ impl<'tcx> TraitEngine<'tcx> for FulfillmentContext<'tcx> {
         errors
     }
 
+    fn drain_unstalled_obligations(
+        &mut self,
+        _: &InferCtxt<'tcx>,
+    ) -> Vec<PredicateObligation<'tcx>> {
+        unimplemented!()
+    }
+
     fn pending_obligations(&self) -> Vec<PredicateObligation<'tcx>> {
         self.obligations.iter().cloned().collect()
     }

compiler/rustc_trait_selection/src/traits/coherence.rs

@@ -696,7 +696,9 @@ impl<'tcx> TypeVisitor<'tcx> for OrphanChecker<'tcx> {
             // This should only be created when checking whether we have to check whether some
             // auto trait impl applies. There will never be multiple impls, so we can just
             // act as if it were a local type here.
-            ty::GeneratorWitness(_) => ControlFlow::Break(OrphanCheckEarlyExit::LocalTy(ty)),
+            ty::GeneratorWitness(_) | ty::GeneratorWitnessMIR(..) => {
+                ControlFlow::Break(OrphanCheckEarlyExit::LocalTy(ty))
+            }
             ty::Alias(ty::Opaque, ..) => {
                 // This merits some explanation.
                 // Normally, opaque types are not involved when performing

compiler/rustc_trait_selection/src/traits/error_reporting/mod.rs

@@ -101,6 +101,18 @@ pub trait InferCtxtExt<'tcx> {
 }
 
 pub trait TypeErrCtxtExt<'tcx> {
+    fn build_overflow_error<T>(
+        &self,
+        predicate: &T,
+        span: Span,
+        suggest_increasing_limit: bool,
+    ) -> DiagnosticBuilder<'tcx, ErrorGuaranteed>
+    where
+        T: fmt::Display
+            + TypeFoldable<'tcx>
+            + Print<'tcx, FmtPrinter<'tcx, 'tcx>, Output = FmtPrinter<'tcx, 'tcx>>,
+        <T as Print<'tcx, FmtPrinter<'tcx, 'tcx>>>::Error: std::fmt::Debug;
+
     fn report_overflow_error<T>(
         &self,
         predicate: &T,
@@ -478,6 +490,26 @@ impl<'tcx> TypeErrCtxtExt<'tcx> for TypeErrCtxt<'_, 'tcx> {
         suggest_increasing_limit: bool,
         mutate: impl FnOnce(&mut Diagnostic),
     ) -> !
+    where
+        T: fmt::Display
+            + TypeFoldable<'tcx>
+            + Print<'tcx, FmtPrinter<'tcx, 'tcx>, Output = FmtPrinter<'tcx, 'tcx>>,
+        <T as Print<'tcx, FmtPrinter<'tcx, 'tcx>>>::Error: std::fmt::Debug,
+    {
+        let mut err = self.build_overflow_error(predicate, span, suggest_increasing_limit);
+        mutate(&mut err);
+        err.emit();
+
+        self.tcx.sess.abort_if_errors();
+        bug!();
+    }
+
+    fn build_overflow_error<T>(
+        &self,
+        predicate: &T,
+        span: Span,
+        suggest_increasing_limit: bool,
+    ) -> DiagnosticBuilder<'tcx, ErrorGuaranteed>
     where
         T: fmt::Display
             + TypeFoldable<'tcx>
@@ -511,11 +543,7 @@ impl<'tcx> TypeErrCtxtExt<'tcx> for TypeErrCtxt<'_, 'tcx> {
             self.suggest_new_overflow_limit(&mut err);
         }
 
-        mutate(&mut err);
+        err
-
-        err.emit();
-        self.tcx.sess.abort_if_errors();
-        bug!();
     }
 
     /// Reports that an overflow has occurred and halts compilation. We
@@ -1891,6 +1919,7 @@ impl<'tcx> InferCtxtPrivExt<'tcx> for TypeErrCtxt<'_, 'tcx> {
                 ty::Generator(..) => Some(16),
                 ty::Foreign(..) => Some(17),
                 ty::GeneratorWitness(..) => Some(18),
+                ty::GeneratorWitnessMIR(..) => Some(19),
                 ty::Placeholder(..) | ty::Bound(..) | ty::Infer(..) | ty::Error(_) => None,
             }
         }

compiler/rustc_trait_selection/src/traits/error_reporting/suggestions.rs

@@ -2226,7 +2226,7 @@ impl<'tcx> TypeErrCtxtExt<'tcx> for TypeErrCtxt<'_, 'tcx> {
                     );
 
                     match *ty.kind() {
-                        ty::Generator(did, ..) => {
+                        ty::Generator(did, ..) | ty::GeneratorWitnessMIR(did, _) => {
                             generator = generator.or(Some(did));
                             outer_generator = Some(did);
                         }
@@ -2256,7 +2256,7 @@ impl<'tcx> TypeErrCtxtExt<'tcx> for TypeErrCtxt<'_, 'tcx> {
                     );
 
                     match *ty.kind() {
-                        ty::Generator(did, ..) => {
+                        ty::Generator(did, ..) | ty::GeneratorWitnessMIR(did, ..) => {
                             generator = generator.or(Some(did));
                             outer_generator = Some(did);
                         }
@@ -2345,6 +2345,11 @@ impl<'tcx> TypeErrCtxtExt<'tcx> for TypeErrCtxt<'_, 'tcx> {
             _ => return false,
         };
 
+        let generator_within_in_progress_typeck = match &self.typeck_results {
+            Some(t) => t.hir_owner.to_def_id() == generator_did_root,
+            _ => false,
+        };
+
         let mut interior_or_upvar_span = None;
 
         let from_awaited_ty = generator_data.get_from_await_ty(visitor, hir, ty_matches);
@@ -2364,6 +2369,35 @@ impl<'tcx> TypeErrCtxtExt<'tcx> for TypeErrCtxt<'_, 'tcx> {
                 *span,
                 Some((*scope_span, *yield_span, *expr, from_awaited_ty)),
             ));
+
+            if interior_or_upvar_span.is_none() && generator_data.is_foreign() {
+                interior_or_upvar_span = Some(GeneratorInteriorOrUpvar::Interior(*span, None));
+            }
+        } else if self.tcx.sess.opts.unstable_opts.drop_tracking_mir
+            // Avoid disclosing internal information to downstream crates.
+            && generator_did.is_local()
+            // Try to avoid cycles.
+            && !generator_within_in_progress_typeck
+        {
+            let generator_info = &self.tcx.mir_generator_witnesses(generator_did);
+            debug!(?generator_info);
+
+            'find_source: for (variant, source_info) in
+                generator_info.variant_fields.iter().zip(&generator_info.variant_source_info)
+            {
+                debug!(?variant);
+                for &local in variant {
+                    let decl = &generator_info.field_tys[local];
+                    debug!(?decl);
+                    if ty_matches(ty::Binder::dummy(decl.ty)) && !decl.ignore_for_traits {
+                        interior_or_upvar_span = Some(GeneratorInteriorOrUpvar::Interior(
+                            decl.source_info.span,
+                            Some((None, source_info.span, None, from_awaited_ty)),
+                        ));
+                        break 'find_source;
+                    }
+                }
+            }
         }
 
         if interior_or_upvar_span.is_none() {
@@ -3012,6 +3046,20 @@ impl<'tcx> TypeErrCtxtExt<'tcx> for TypeErrCtxt<'_, 'tcx> {
                                 }
                                 err.note(msg.trim_end_matches(", "))
                             }
+                            ty::GeneratorWitnessMIR(def_id, substs) => {
+                                use std::fmt::Write;
+
+                                // FIXME: this is kind of an unusual format for rustc, can we make it more clear?
+                                // Maybe we should just remove this note altogether?
+                                // FIXME: only print types which don't meet the trait requirement
+                                let mut msg =
+                                    "required because it captures the following types: ".to_owned();
+                                for bty in tcx.generator_hidden_types(*def_id) {
+                                    let ty = bty.subst(tcx, substs);
+                                    write!(msg, "`{}`, ", ty).unwrap();
+                                }
+                                err.note(msg.trim_end_matches(", "))
+                            }
                             ty::Generator(def_id, _, _) => {
                                 let sp = self.tcx.def_span(def_id);
 

compiler/rustc_trait_selection/src/traits/fulfill.rs

@@ -132,14 +132,7 @@ impl<'tcx> TraitEngine<'tcx> for FulfillmentContext<'tcx> {
             .register_obligation(PendingPredicateObligation { obligation, stalled_on: vec![] });
     }
 
-    fn select_all_or_error(&mut self, infcx: &InferCtxt<'tcx>) -> Vec<FulfillmentError<'tcx>> {
+    fn collect_remaining_errors(&mut self) -> Vec<FulfillmentError<'tcx>> {
-        {
-            let errors = self.select_where_possible(infcx);
-            if !errors.is_empty() {
-                return errors;
-            }
-        }
-
         self.predicates.to_errors(CodeAmbiguity).into_iter().map(to_fulfillment_error).collect()
     }
 
@@ -148,6 +141,55 @@ impl<'tcx> TraitEngine<'tcx> for FulfillmentContext<'tcx> {
         self.select(selcx)
     }
 
+    fn drain_unstalled_obligations(
+        &mut self,
+        infcx: &InferCtxt<'tcx>,
+    ) -> Vec<PredicateObligation<'tcx>> {
+        let mut processor = DrainProcessor { removed_predicates: Vec::new(), infcx };
+        let outcome: Outcome<_, _> = self.predicates.process_obligations(&mut processor);
+        assert!(outcome.errors.is_empty());
+        return processor.removed_predicates;
+
+        struct DrainProcessor<'a, 'tcx> {
+            infcx: &'a InferCtxt<'tcx>,
+            removed_predicates: Vec<PredicateObligation<'tcx>>,
+        }
+
+        impl<'tcx> ObligationProcessor for DrainProcessor<'_, 'tcx> {
+            type Obligation = PendingPredicateObligation<'tcx>;
+            type Error = !;
+            type OUT = Outcome<Self::Obligation, Self::Error>;
+
+            fn needs_process_obligation(&self, pending_obligation: &Self::Obligation) -> bool {
+                pending_obligation
+                    .stalled_on
+                    .iter()
+                    .any(|&var| self.infcx.ty_or_const_infer_var_changed(var))
+            }
+
+            fn process_obligation(
+                &mut self,
+                pending_obligation: &mut PendingPredicateObligation<'tcx>,
+            ) -> ProcessResult<PendingPredicateObligation<'tcx>, !> {
+                assert!(self.needs_process_obligation(pending_obligation));
+                self.removed_predicates.push(pending_obligation.obligation.clone());
+                ProcessResult::Changed(vec![])
+            }
+
+            fn process_backedge<'c, I>(
+                &mut self,
+                cycle: I,
+                _marker: PhantomData<&'c PendingPredicateObligation<'tcx>>,
+            ) -> Result<(), !>
+            where
+                I: Clone + Iterator<Item = &'c PendingPredicateObligation<'tcx>>,
+            {
+                self.removed_predicates.extend(cycle.map(|c| c.obligation.clone()));
+                Ok(())
+            }
+        }
+    }
+
     fn pending_obligations(&self) -> Vec<PredicateObligation<'tcx>> {
         self.predicates.map_pending_obligations(|o| o.obligation.clone())
     }

compiler/rustc_trait_selection/src/traits/project.rs

@@ -1605,6 +1605,7 @@ fn assemble_candidates_from_impls<'cx, 'tcx>(
                         | ty::Closure(..)
                         | ty::Generator(..)
                         | ty::GeneratorWitness(..)
+                        | ty::GeneratorWitnessMIR(..)
                         | ty::Never
                         | ty::Tuple(..)
                         // Integers and floats always have `u8` as their discriminant.
@@ -1654,6 +1655,7 @@ fn assemble_candidates_from_impls<'cx, 'tcx>(
                         | ty::Closure(..)
                         | ty::Generator(..)
                         | ty::GeneratorWitness(..)
+                        | ty::GeneratorWitnessMIR(..)
                         | ty::Never
                         // Extern types have unit metadata, according to RFC 2850
                         | ty::Foreign(_)

compiler/rustc_trait_selection/src/traits/query/dropck_outlives.rs

@@ -31,6 +31,7 @@ pub fn trivial_dropck_outlives<'tcx>(tcx: TyCtxt<'tcx>, ty: Ty<'tcx>) -> bool {
         | ty::FnPtr(_)
         | ty::Char
         | ty::GeneratorWitness(..)
+        | ty::GeneratorWitnessMIR(..)
         | ty::RawPtr(_)
         | ty::Ref(..)
         | ty::Str

compiler/rustc_trait_selection/src/traits/query/normalize.rs

@@ -216,12 +216,16 @@ impl<'cx, 'tcx> FallibleTypeFolder<'tcx> for QueryNormalizer<'cx, 'tcx> {
                         let substs = substs.try_fold_with(self)?;
                         let recursion_limit = self.tcx().recursion_limit();
                         if !recursion_limit.value_within_limit(self.anon_depth) {
-                            self.infcx.err_ctxt().report_overflow_error(
+                            // A closure or generator may have itself as in its upvars.
-                                &ty,
+                            // This should be checked handled by the recursion check for opaque
-                                self.cause.span,
+                            // types, but we may end up here before that check can happen.
-                                true,
+                            // In that case, we delay a bug to mark the trip, and continue without
-                                |_| {},
+                            // revealing the opaque.
-                            );
+                            self.infcx
+                                .err_ctxt()
+                                .build_overflow_error(&ty, self.cause.span, true)
+                                .delay_as_bug();
+                            return ty.try_super_fold_with(self);
                         }
 
                         let generic_ty = self.tcx().bound_type_of(def_id);

compiler/rustc_trait_selection/src/traits/select/candidate_assembly.rs

@@ -765,7 +765,8 @@ impl<'cx, 'tcx> SelectionContext<'cx, 'tcx> {
             | ty::Closure(..)
             | ty::Generator(..)
             | ty::Tuple(_)
-            | ty::GeneratorWitness(_) => {
+            | ty::GeneratorWitness(_)
+            | ty::GeneratorWitnessMIR(..) => {
                 // These are built-in, and cannot have a custom `impl const Destruct`.
                 candidates.vec.push(ConstDestructCandidate(None));
             }
@@ -826,6 +827,7 @@ impl<'cx, 'tcx> SelectionContext<'cx, 'tcx> {
             | ty::Closure(_, _)
             | ty::Generator(_, _, _)
             | ty::GeneratorWitness(_)
+            | ty::GeneratorWitnessMIR(..)
             | ty::Never
             | ty::Alias(..)
             | ty::Param(_)

compiler/rustc_trait_selection/src/traits/select/confirmation.rs

@@ -13,7 +13,7 @@ use rustc_infer::infer::InferOk;
 use rustc_infer::infer::LateBoundRegionConversionTime::HigherRankedType;
 use rustc_middle::ty::{
     self, Binder, GenericArg, GenericArgKind, GenericParamDefKind, InternalSubsts, SubstsRef,
-    ToPolyTraitRef, ToPredicate, TraitRef, Ty, TyCtxt,
+    ToPolyTraitRef, ToPredicate, TraitRef, Ty, TyCtxt, TypeVisitable,
 };
 use rustc_session::config::TraitSolver;
 use rustc_span::def_id::DefId;
@@ -1285,6 +1285,14 @@ impl<'cx, 'tcx> SelectionContext<'cx, 'tcx> {
                 ty::GeneratorWitness(tys) => {
                     stack.extend(tcx.erase_late_bound_regions(tys).to_vec());
                 }
+                ty::GeneratorWitnessMIR(def_id, substs) => {
+                    let tcx = self.tcx();
+                    stack.extend(tcx.generator_hidden_types(def_id).map(|bty| {
+                        let ty = bty.subst(tcx, substs);
+                        debug_assert!(!ty.has_late_bound_regions());
+                        ty
+                    }))
+                }
 
                 // If we have a projection type, make sure to normalize it so we replace it
                 // with a fresh infer variable

compiler/rustc_trait_selection/src/traits/select/mod.rs

@@ -2066,6 +2066,7 @@ impl<'cx, 'tcx> SelectionContext<'cx, 'tcx> {
             | ty::Ref(..)
             | ty::Generator(..)
             | ty::GeneratorWitness(..)
+            | ty::GeneratorWitnessMIR(..)
             | ty::Array(..)
             | ty::Closure(..)
             | ty::Never
@@ -2182,6 +2183,16 @@ impl<'cx, 'tcx> SelectionContext<'cx, 'tcx> {
                 Where(ty::Binder::bind_with_vars(witness_tys.to_vec(), all_vars))
             }
 
+            ty::GeneratorWitnessMIR(def_id, ref substs) => {
+                let hidden_types = bind_generator_hidden_types_above(
+                    self.infcx,
+                    def_id,
+                    substs,
+                    obligation.predicate.bound_vars(),
+                );
+                Where(hidden_types)
+            }
+
             ty::Closure(_, substs) => {
                 // (*) binder moved here
                 let ty = self.infcx.shallow_resolve(substs.as_closure().tupled_upvars_ty());
@@ -2279,6 +2290,10 @@ impl<'cx, 'tcx> SelectionContext<'cx, 'tcx> {
                 types.map_bound(|types| types.to_vec())
             }
 
+            ty::GeneratorWitnessMIR(def_id, ref substs) => {
+                bind_generator_hidden_types_above(self.infcx, def_id, substs, t.bound_vars())
+            }
+
             // For `PhantomData<T>`, we pass `T`.
             ty::Adt(def, substs) if def.is_phantom_data() => t.rebind(substs.types().collect()),
 
@@ -2921,3 +2936,56 @@ pub enum ProjectionMatchesProjection {
     Ambiguous,
     No,
 }
+
+/// Replace all regions inside the generator interior with late bound regions.
+/// Note that each region slot in the types gets a new fresh late bound region, which means that
+/// none of the regions inside relate to any other, even if typeck had previously found constraints
+/// that would cause them to be related.
+#[instrument(level = "trace", skip(infcx), ret)]
+fn bind_generator_hidden_types_above<'tcx>(
+    infcx: &InferCtxt<'tcx>,
+    def_id: DefId,
+    substs: ty::SubstsRef<'tcx>,
+    bound_vars: &ty::List<ty::BoundVariableKind>,
+) -> ty::Binder<'tcx, Vec<Ty<'tcx>>> {
+    let tcx = infcx.tcx;
+    let mut seen_tys = FxHashSet::default();
+
+    let considering_regions = infcx.considering_regions;
+
+    let num_bound_variables = bound_vars.len() as u32;
+    let mut counter = num_bound_variables;
+
+    let hidden_types: Vec<_> = tcx
+        .generator_hidden_types(def_id)
+        // Deduplicate tys to avoid repeated work.
+        .filter(|bty| seen_tys.insert(*bty))
+        .map(|bty| {
+            let mut ty = bty.subst(tcx, substs);
+
+            // Only remap erased regions if we use them.
+            if considering_regions {
+                ty = tcx.fold_regions(ty, |mut r, current_depth| {
+                    if let ty::ReErased = r.kind() {
+                        let br = ty::BoundRegion {
+                            var: ty::BoundVar::from_u32(counter),
+                            kind: ty::BrAnon(counter, None),
+                        };
+                        counter += 1;
+                        r = tcx.mk_region(ty::ReLateBound(current_depth, br));
+                    }
+                    r
+                })
+            }
+
+            ty
+        })
+        .collect();
+    if considering_regions {
+        debug_assert!(!hidden_types.has_erased_regions());
+    }
+    let bound_vars = tcx.mk_bound_variable_kinds(bound_vars.iter().chain(
+        (num_bound_variables..counter).map(|i| ty::BoundVariableKind::Region(ty::BrAnon(i, None))),
+    ));
+    ty::Binder::bind_with_vars(hidden_types, bound_vars)
+}

compiler/rustc_trait_selection/src/traits/structural_match.rs

@@ -101,7 +101,7 @@ impl<'tcx> TypeVisitor<'tcx> for Search<'tcx> {
             ty::Closure(..) => {
                 return ControlFlow::Break(ty);
             }
-            ty::Generator(..) | ty::GeneratorWitness(..) => {
+            ty::Generator(..) | ty::GeneratorWitness(..) | ty::GeneratorWitnessMIR(..) => {
                 return ControlFlow::Break(ty);
             }
             ty::FnDef(..) => {

compiler/rustc_trait_selection/src/traits/wf.rs

@@ -551,6 +551,7 @@ impl<'tcx> WfPredicates<'tcx> {
                 | ty::Error(_)
                 | ty::Str
                 | ty::GeneratorWitness(..)
+                | ty::GeneratorWitnessMIR(..)
                 | ty::Never
                 | ty::Param(_)
                 | ty::Bound(..)

compiler/rustc_traits/src/chalk/lowering.rs

@@ -343,6 +343,7 @@ impl<'tcx> LowerInto<'tcx, chalk_ir::Ty<RustInterner<'tcx>>> for Ty<'tcx> {
                 substs.lower_into(interner),
             ),
             ty::GeneratorWitness(_) => unimplemented!(),
+            ty::GeneratorWitnessMIR(..) => unimplemented!(),
             ty::Never => chalk_ir::TyKind::Never,
             ty::Tuple(types) => {
                 chalk_ir::TyKind::Tuple(types.len(), types.as_substs().lower_into(interner))

compiler/rustc_traits/src/codegen.rs

@@ -4,7 +4,7 @@
 // general routines.
 
 use rustc_infer::infer::{DefiningAnchor, TyCtxtInferExt};
-use rustc_infer::traits::FulfillmentErrorCode;
+use rustc_infer::traits::{FulfillmentErrorCode, TraitEngineExt as _};
 use rustc_middle::traits::CodegenObligationError;
 use rustc_middle::ty::{self, TyCtxt};
 use rustc_trait_selection::traits::error_reporting::TypeErrCtxtExt;

compiler/rustc_traits/src/dropck_outlives.rs

@@ -164,7 +164,8 @@ fn dtorck_constraint_for_ty<'tcx>(
         | ty::Ref(..)
         | ty::FnDef(..)
         | ty::FnPtr(_)
-        | ty::GeneratorWitness(..) => {
+        | ty::GeneratorWitness(..)
+        | ty::GeneratorWitnessMIR(..) => {
             // these types never have a destructor
         }
 

compiler/rustc_ty_utils/src/layout.rs

@@ -470,7 +470,10 @@ fn layout_of_uncached<'tcx>(
             return Err(LayoutError::Unknown(ty));
         }
 
-        ty::Placeholder(..) | ty::GeneratorWitness(..) | ty::Infer(_) => {
+        ty::Placeholder(..)
+        | ty::GeneratorWitness(..)
+        | ty::GeneratorWitnessMIR(..)
+        | ty::Infer(_) => {
             bug!("Layout::compute: unexpected type `{}`", ty)
         }
 
@@ -640,7 +643,7 @@ fn generator_layout<'tcx>(
 
     let promoted_layouts = ineligible_locals
         .iter()
-        .map(|local| subst_field(info.field_tys[local]))
+        .map(|local| subst_field(info.field_tys[local].ty))
         .map(|ty| tcx.mk_maybe_uninit(ty))
         .map(|ty| cx.layout_of(ty));
     let prefix_layouts = substs
@@ -710,7 +713,7 @@ fn generator_layout<'tcx>(
                     Assigned(_) => bug!("assignment does not match variant"),
                     Ineligible(_) => false,
                 })
-                .map(|local| subst_field(info.field_tys[*local]));
+                .map(|local| subst_field(info.field_tys[*local].ty));
 
             let mut variant = univariant_uninterned(
                 cx,

compiler/rustc_ty_utils/src/needs_drop.rs

@@ -109,6 +109,13 @@ where
 
             for component in components {
                 match *component.kind() {
+                    // The information required to determine whether a generator has drop is
+                    // computed on MIR, while this very method is used to build MIR.
+                    // To avoid cycles, we consider that generators always require drop.
+                    ty::Generator(..) if tcx.sess.opts.unstable_opts.drop_tracking_mir => {
+                        return Some(Err(AlwaysRequiresDrop));
+                    }
+
                     _ if component.is_copy_modulo_regions(tcx, self.param_env) => (),
 
                     ty::Closure(_, substs) => {

compiler/rustc_ty_utils/src/ty.rs

@@ -16,7 +16,13 @@ fn sized_constraint_for_ty<'tcx>(
         Bool | Char | Int(..) | Uint(..) | Float(..) | RawPtr(..) | Ref(..) | FnDef(..)
         | FnPtr(_) | Array(..) | Closure(..) | Generator(..) | Never => vec![],
 
-        Str | Dynamic(..) | Slice(_) | Foreign(..) | Error(_) | GeneratorWitness(..) => {
+        Str
+        | Dynamic(..)
+        | Slice(_)
+        | Foreign(..)
+        | Error(_)
+        | GeneratorWitness(..)
+        | GeneratorWitnessMIR(..) => {
             // these are never sized - return the target type
             vec![ty]
         }

compiler/rustc_type_ir/src/lib.rs

@@ -265,6 +265,9 @@ bitflags! {
 
         /// Does this value have `InferConst::Fresh`?
         const HAS_CT_FRESH                = 1 << 21;
+
+        /// Does this have `Generator` or `GeneratorWitness`?
+        const HAS_TY_GENERATOR            = 1 << 22;
     }
 }
 

compiler/rustc_type_ir/src/sty.rs

@@ -160,6 +160,32 @@ pub enum TyKind<I: Interner> {
     /// ```
     GeneratorWitness(I::BinderListTy),
 
+    /// A type representing the types stored inside a generator.
+    /// This should only appear as part of the `GeneratorSubsts`.
+    ///
+    /// Unlike upvars, the witness can reference lifetimes from
+    /// inside of the generator itself. To deal with them in
+    /// the type of the generator, we convert them to higher ranked
+    /// lifetimes bound by the witness itself.
+    ///
+    /// This variant is only using when `drop_tracking_mir` is set.
+    /// This contains the `DefId` and the `SubstRef` of the generator.
+    /// The actual witness types are computed on MIR by the `mir_generator_witnesses` query.
+    ///
+    /// Looking at the following example, the witness for this generator
+    /// may end up as something like `for<'a> [Vec<i32>, &'a Vec<i32>]`:
+    ///
+    /// ```ignore UNSOLVED (ask @compiler-errors, should this error? can we just swap the yields?)
+    /// #![feature(generators)]
+    /// |a| {
+    ///     let x = &vec![3];
+    ///     yield a;
+    ///     yield x[0];
+    /// }
+    /// # ;
+    /// ```
+    GeneratorWitnessMIR(I::DefId, I::SubstsRef),
+
     /// The never type `!`.
     Never,
 
@@ -241,6 +267,7 @@ const fn tykind_discriminant<I: Interner>(value: &TyKind<I>) -> usize {
         Placeholder(_) => 23,
         Infer(_) => 24,
         Error(_) => 25,
+        GeneratorWitnessMIR(_, _) => 26,
     }
 }
 
@@ -266,6 +293,7 @@ impl<I: Interner> Clone for TyKind<I> {
             Closure(d, s) => Closure(d.clone(), s.clone()),
             Generator(d, s, m) => Generator(d.clone(), s.clone(), m.clone()),
             GeneratorWitness(g) => GeneratorWitness(g.clone()),
+            GeneratorWitnessMIR(d, s) => GeneratorWitnessMIR(d.clone(), s.clone()),
             Never => Never,
             Tuple(t) => Tuple(t.clone()),
             Alias(k, p) => Alias(*k, p.clone()),
@@ -303,6 +331,10 @@ impl<I: Interner> PartialEq for TyKind<I> {
                     a_d == b_d && a_s == b_s && a_m == b_m
                 }
                 (GeneratorWitness(a_g), GeneratorWitness(b_g)) => a_g == b_g,
+                (
+                    &GeneratorWitnessMIR(ref a_d, ref a_s),
+                    &GeneratorWitnessMIR(ref b_d, ref b_s),
+                ) => a_d == b_d && a_s == b_s,
                 (Tuple(a_t), Tuple(b_t)) => a_t == b_t,
                 (Alias(a_i, a_p), Alias(b_i, b_p)) => a_i == b_i && a_p == b_p,
                 (Param(a_p), Param(b_p)) => a_p == b_p,
@@ -360,6 +392,13 @@ impl<I: Interner> Ord for TyKind<I> {
                     a_d.cmp(b_d).then_with(|| a_s.cmp(b_s).then_with(|| a_m.cmp(b_m)))
                 }
                 (GeneratorWitness(a_g), GeneratorWitness(b_g)) => a_g.cmp(b_g),
+                (
+                    &GeneratorWitnessMIR(ref a_d, ref a_s),
+                    &GeneratorWitnessMIR(ref b_d, ref b_s),
+                ) => match Ord::cmp(a_d, b_d) {
+                    Ordering::Equal => Ord::cmp(a_s, b_s),
+                    cmp => cmp,
+                },
                 (Tuple(a_t), Tuple(b_t)) => a_t.cmp(b_t),
                 (Alias(a_i, a_p), Alias(b_i, b_p)) => a_i.cmp(b_i).then_with(|| a_p.cmp(b_p)),
                 (Param(a_p), Param(b_p)) => a_p.cmp(b_p),
@@ -421,6 +460,10 @@ impl<I: Interner> hash::Hash for TyKind<I> {
                 m.hash(state)
             }
             GeneratorWitness(g) => g.hash(state),
+            GeneratorWitnessMIR(d, s) => {
+                d.hash(state);
+                s.hash(state);
+            }
             Tuple(t) => t.hash(state),
             Alias(i, p) => {
                 i.hash(state);
@@ -461,6 +504,7 @@ impl<I: Interner> fmt::Debug for TyKind<I> {
             Closure(d, s) => f.debug_tuple_field2_finish("Closure", d, s),
             Generator(d, s, m) => f.debug_tuple_field3_finish("Generator", d, s, m),
             GeneratorWitness(g) => f.debug_tuple_field1_finish("GeneratorWitness", g),
+            GeneratorWitnessMIR(d, s) => f.debug_tuple_field2_finish("GeneratorWitnessMIR", d, s),
             Never => f.write_str("Never"),
             Tuple(t) => f.debug_tuple_field1_finish("Tuple", t),
             Alias(i, a) => f.debug_tuple_field2_finish("Alias", i, a),
@@ -559,6 +603,10 @@ where
             GeneratorWitness(b) => e.emit_enum_variant(disc, |e| {
                 b.encode(e);
             }),
+            GeneratorWitnessMIR(def_id, substs) => e.emit_enum_variant(disc, |e| {
+                def_id.encode(e);
+                substs.encode(e);
+            }),
             Never => e.emit_enum_variant(disc, |_| {}),
             Tuple(substs) => e.emit_enum_variant(disc, |e| {
                 substs.encode(e);
@@ -641,6 +689,7 @@ where
             23 => Placeholder(Decodable::decode(d)),
             24 => Infer(Decodable::decode(d)),
             25 => Error(Decodable::decode(d)),
+            26 => GeneratorWitnessMIR(Decodable::decode(d), Decodable::decode(d)),
             _ => panic!(
                 "{}",
                 format!(
@@ -742,6 +791,10 @@ where
             GeneratorWitness(b) => {
                 b.hash_stable(__hcx, __hasher);
             }
+            GeneratorWitnessMIR(def_id, substs) => {
+                def_id.hash_stable(__hcx, __hasher);
+                substs.hash_stable(__hcx, __hasher);
+            }
             Never => {}
             Tuple(substs) => {
                 substs.hash_stable(__hcx, __hasher);

src/librustdoc/clean/mod.rs

@@ -1855,6 +1855,7 @@ pub(crate) fn clean_middle_ty<'tcx>(
         ty::Bound(..) => panic!("Bound"),
         ty::Placeholder(..) => panic!("Placeholder"),
         ty::GeneratorWitness(..) => panic!("GeneratorWitness"),
+        ty::GeneratorWitnessMIR(..) => panic!("GeneratorWitnessMIR"),
         ty::Infer(..) => panic!("Infer"),
         ty::Error(_) => rustc_errors::FatalError.raise(),
     }

src/librustdoc/passes/collect_intra_doc_links.rs

@@ -542,6 +542,7 @@ impl<'a, 'tcx> LinkCollector<'a, 'tcx> {
             | ty::Closure(..)
             | ty::Generator(..)
             | ty::GeneratorWitness(_)
+            | ty::GeneratorWitnessMIR(..)
             | ty::Dynamic(..)
             | ty::Param(_)
             | ty::Bound(..)

src/tools/clippy/clippy_lints/src/dereference.rs

@@ -1419,6 +1419,7 @@ fn ty_auto_deref_stability<'tcx>(cx: &LateContext<'tcx>, ty: Ty<'tcx>, precedenc
             | ty::FnDef(..)
             | ty::Generator(..)
             | ty::GeneratorWitness(..)
+            | ty::GeneratorWitnessMIR(..)
             | ty::Closure(..)
             | ty::Never
             | ty::Tuple(_)

tests/mir-opt/building/async_await.b-{closure#0}.generator_resume.0.mir

@@ -1,8 +1,22 @@
 // MIR for `b::{closure#0}` 0 generator_resume
 /* generator_layout = GeneratorLayout {
     field_tys: {
-        _0: impl std::future::Future<Output = ()>,
+        _0: GeneratorSavedTy {
-        _1: impl std::future::Future<Output = ()>,
+            ty: impl std::future::Future<Output = ()>,
+            source_info: SourceInfo {
+                span: $DIR/async_await.rs:15:8: 15:14 (#9),
+                scope: scope[0],
+            },
+            ignore_for_traits: false,
+        },
+        _1: GeneratorSavedTy {
+            ty: impl std::future::Future<Output = ()>,
+            source_info: SourceInfo {
+                span: $DIR/async_await.rs:16:8: 16:14 (#12),
+                scope: scope[0],
+            },
+            ignore_for_traits: false,
+        },
     },
     variant_fields: {
         Unresumed(0): [],

tests/mir-opt/generator_drop_cleanup.main-{closure#0}.generator_drop.0.mir

@@ -1,7 +1,14 @@
 // MIR for `main::{closure#0}` 0 generator_drop
 /* generator_layout = GeneratorLayout {
     field_tys: {
-        _0: std::string::String,
+        _0: GeneratorSavedTy {
+            ty: std::string::String,
+            source_info: SourceInfo {
+                span: $DIR/generator_drop_cleanup.rs:11:13: 11:15 (#0),
+                scope: scope[0],
+            },
+            ignore_for_traits: false,
+        },
     },
     variant_fields: {
         Unresumed(0): [],

tests/mir-opt/generator_tiny.main-{closure#0}.generator_resume.0.mir

@@ -1,7 +1,14 @@
 // MIR for `main::{closure#0}` 0 generator_resume
 /* generator_layout = GeneratorLayout {
     field_tys: {
-        _0: HasDrop,
+        _0: GeneratorSavedTy {
+            ty: HasDrop,
+            source_info: SourceInfo {
+                span: $DIR/generator_tiny.rs:20:13: 20:15 (#0),
+                scope: scope[0],
+            },
+            ignore_for_traits: false,
+        },
     },
     variant_fields: {
         Unresumed(0): [],

tests/rustdoc-ui/z-help.stdout

@@ -20,6 +20,7 @@
     -Z                                 dlltool=val -- import library generation tool (windows-gnu only)
     -Z                 dont-buffer-diagnostics=val -- emit diagnostics rather than buffering (breaks NLL error downgrading, sorting) (default: no)
     -Z                           drop-tracking=val -- enables drop tracking in generators (default: no)
+    -Z                       drop-tracking-mir=val -- enables drop tracking on MIR in generators (default: no)
     -Z                        dual-proc-macros=val -- load proc macros for both target and host, but only link to the target (default: no)
     -Z                          dump-dep-graph=val -- dump the dependency graph to $RUST_DEP_GRAPH (default: /tmp/dep_graph.gv) (default: no)
     -Z                  dump-drop-tracking-cfg=val -- dump drop-tracking control-flow graph as a `.dot` file (default: no)

tests/ui-fulldeps/internal-lints/ty_tykind_usage.rs

@@ -31,6 +31,7 @@ fn main() {
         TyKind::Closure(..) => (),          //~ ERROR usage of `ty::TyKind::<kind>`
         TyKind::Generator(..) => (),        //~ ERROR usage of `ty::TyKind::<kind>`
         TyKind::GeneratorWitness(..) => (), //~ ERROR usage of `ty::TyKind::<kind>`
+        TyKind::GeneratorWitnessMIR(..) => (), //~ ERROR usage of `ty::TyKind::<kind>`
         TyKind::Never => (),                //~ ERROR usage of `ty::TyKind::<kind>`
         TyKind::Tuple(..) => (),            //~ ERROR usage of `ty::TyKind::<kind>`
         TyKind::Alias(..) => (),            //~ ERROR usage of `ty::TyKind::<kind>`

tests/ui-fulldeps/internal-lints/ty_tykind_usage.stderr

@@ -121,59 +121,65 @@ LL |         TyKind::GeneratorWitness(..) => (),
 error: usage of `ty::TyKind::<kind>`
   --> $DIR/ty_tykind_usage.rs:34:9
    |
-LL |         TyKind::Never => (),
+LL |         TyKind::GeneratorWitnessMIR(..) => (),
    |         ^^^^^^ help: try using `ty::<kind>` directly: `ty`
 
 error: usage of `ty::TyKind::<kind>`
   --> $DIR/ty_tykind_usage.rs:35:9
    |
-LL |         TyKind::Tuple(..) => (),
+LL |         TyKind::Never => (),
    |         ^^^^^^ help: try using `ty::<kind>` directly: `ty`
 
 error: usage of `ty::TyKind::<kind>`
   --> $DIR/ty_tykind_usage.rs:36:9
    |
-LL |         TyKind::Alias(..) => (),
+LL |         TyKind::Tuple(..) => (),
    |         ^^^^^^ help: try using `ty::<kind>` directly: `ty`
 
 error: usage of `ty::TyKind::<kind>`
   --> $DIR/ty_tykind_usage.rs:37:9
    |
-LL |         TyKind::Param(..) => (),
+LL |         TyKind::Alias(..) => (),
    |         ^^^^^^ help: try using `ty::<kind>` directly: `ty`
 
 error: usage of `ty::TyKind::<kind>`
   --> $DIR/ty_tykind_usage.rs:38:9
    |
-LL |         TyKind::Bound(..) => (),
+LL |         TyKind::Param(..) => (),
    |         ^^^^^^ help: try using `ty::<kind>` directly: `ty`
 
 error: usage of `ty::TyKind::<kind>`
   --> $DIR/ty_tykind_usage.rs:39:9
    |
-LL |         TyKind::Placeholder(..) => (),
+LL |         TyKind::Bound(..) => (),
    |         ^^^^^^ help: try using `ty::<kind>` directly: `ty`
 
 error: usage of `ty::TyKind::<kind>`
   --> $DIR/ty_tykind_usage.rs:40:9
    |
-LL |         TyKind::Infer(..) => (),
+LL |         TyKind::Placeholder(..) => (),
    |         ^^^^^^ help: try using `ty::<kind>` directly: `ty`
 
 error: usage of `ty::TyKind::<kind>`
   --> $DIR/ty_tykind_usage.rs:41:9
    |
+LL |         TyKind::Infer(..) => (),
+   |         ^^^^^^ help: try using `ty::<kind>` directly: `ty`
+
+error: usage of `ty::TyKind::<kind>`
+  --> $DIR/ty_tykind_usage.rs:42:9
+   |
 LL |         TyKind::Error(_) => (),
    |         ^^^^^^ help: try using `ty::<kind>` directly: `ty`
 
 error: usage of `ty::TyKind::<kind>`
-  --> $DIR/ty_tykind_usage.rs:46:12
+  --> $DIR/ty_tykind_usage.rs:47:12
    |
 LL |     if let TyKind::Int(int_ty) = kind {}
    |            ^^^^^^ help: try using `ty::<kind>` directly: `ty`
 
 error: usage of `ty::TyKind`
-  --> $DIR/ty_tykind_usage.rs:48:24
+  --> $DIR/ty_tykind_usage.rs:49:24
    |
 LL |     fn ty_kind(ty_bad: TyKind<'_>, ty_good: Ty<'_>) {}
    |                        ^^^^^^^^^^
@@ -181,7 +187,7 @@ LL |     fn ty_kind(ty_bad: TyKind<'_>, ty_good: Ty<'_>) {}
    = help: try using `Ty` instead
 
 error: usage of `ty::TyKind`
-  --> $DIR/ty_tykind_usage.rs:50:37
+  --> $DIR/ty_tykind_usage.rs:51:37
    |
 LL |     fn ir_ty_kind<I: Interner>(bad: IrTyKind<I>) -> IrTyKind<I> {
    |                                     ^^^^^^^^^^^
@@ -189,7 +195,7 @@ LL |     fn ir_ty_kind<I: Interner>(bad: IrTyKind<I>) -> IrTyKind<I> {
    = help: try using `Ty` instead
 
 error: usage of `ty::TyKind`
-  --> $DIR/ty_tykind_usage.rs:50:53
+  --> $DIR/ty_tykind_usage.rs:51:53
    |
 LL |     fn ir_ty_kind<I: Interner>(bad: IrTyKind<I>) -> IrTyKind<I> {
    |                                                     ^^^^^^^^^^^
@@ -197,12 +203,12 @@ LL |     fn ir_ty_kind<I: Interner>(bad: IrTyKind<I>) -> IrTyKind<I> {
    = help: try using `Ty` instead
 
 error: usage of `ty::TyKind::<kind>`
-  --> $DIR/ty_tykind_usage.rs:53:9
+  --> $DIR/ty_tykind_usage.rs:54:9
    |
 LL |         IrTyKind::Bool
    |         --------^^^^^^
    |         |
    |         help: try using `ty::<kind>` directly: `ty`
 
-error: aborting due to 32 previous errors
+error: aborting due to 33 previous errors
 

tests/ui/async-await/async-await-let-else.drop_tracking.stderr

@@ -0,0 +1,106 @@
+error: future cannot be sent between threads safely
+  --> $DIR/async-await-let-else.rs:48:13
+   |
+LL |     is_send(foo(Some(true)));
+   |             ^^^^^^^^^^^^^^^ future returned by `foo` is not `Send`
+   |
+   = help: within `impl Future<Output = ()>`, the trait `Send` is not implemented for `Rc<()>`
+note: future is not `Send` as this value is used across an await
+  --> $DIR/async-await-let-else.rs:11:14
+   |
+LL |         let r = Rc::new(());
+   |             - has type `Rc<()>` which is not `Send`
+LL |         bar().await
+   |              ^^^^^^ await occurs here, with `r` maybe used later
+LL |     };
+   |     - `r` is later dropped here
+note: required by a bound in `is_send`
+  --> $DIR/async-await-let-else.rs:19:15
+   |
+LL | fn is_send<T: Send>(_: T) {}
+   |               ^^^^ required by this bound in `is_send`
+
+error[E0277]: `Rc<()>` cannot be sent between threads safely
+  --> $DIR/async-await-let-else.rs:50:13
+   |
+LL | async fn foo2(x: Option<bool>) {
+   |                                - within this `impl Future<Output = ()>`
+...
+LL |     is_send(foo2(Some(true)));
+   |     ------- ^^^^^^^^^^^^^^^^ `Rc<()>` cannot be sent between threads safely
+   |     |
+   |     required by a bound introduced by this call
+   |
+   = help: within `impl Future<Output = ()>`, the trait `Send` is not implemented for `Rc<()>`
+note: required because it's used within this `async fn` body
+  --> $DIR/async-await-let-else.rs:27:29
+   |
+LL |   async fn bar2<T>(_: T) -> ! {
+   |  _____________________________^
+LL | |     panic!()
+LL | | }
+   | |_^
+   = note: required because it captures the following types: `ResumeTy`, `Option<bool>`, `impl Future<Output = !>`, `()`
+note: required because it's used within this `async fn` body
+  --> $DIR/async-await-let-else.rs:21:32
+   |
+LL |   async fn foo2(x: Option<bool>) {
+   |  ________________________________^
+LL | |     let Some(_) = x else {
+LL | |         bar2(Rc::new(())).await
+LL | |     };
+LL | | }
+   | |_^
+note: required by a bound in `is_send`
+  --> $DIR/async-await-let-else.rs:19:15
+   |
+LL | fn is_send<T: Send>(_: T) {}
+   |               ^^^^ required by this bound in `is_send`
+
+error: future cannot be sent between threads safely
+  --> $DIR/async-await-let-else.rs:52:13
+   |
+LL |     is_send(foo3(Some(true)));
+   |             ^^^^^^^^^^^^^^^^ future returned by `foo3` is not `Send`
+   |
+   = help: within `impl Future<Output = ()>`, the trait `Send` is not implemented for `Rc<()>`
+note: future is not `Send` as this value is used across an await
+  --> $DIR/async-await-let-else.rs:33:28
+   |
+LL |         (Rc::new(()), bar().await);
+   |          -----------       ^^^^^^ - `Rc::new(())` is later dropped here
+   |          |                 |
+   |          |                 await occurs here, with `Rc::new(())` maybe used later
+   |          has type `Rc<()>` which is not `Send`
+note: required by a bound in `is_send`
+  --> $DIR/async-await-let-else.rs:19:15
+   |
+LL | fn is_send<T: Send>(_: T) {}
+   |               ^^^^ required by this bound in `is_send`
+
+error: future cannot be sent between threads safely
+  --> $DIR/async-await-let-else.rs:54:13
+   |
+LL |     is_send(foo4(Some(true)));
+   |             ^^^^^^^^^^^^^^^^ future returned by `foo4` is not `Send`
+   |
+   = help: within `impl Future<Output = ()>`, the trait `Send` is not implemented for `Rc<()>`
+note: future is not `Send` as this value is used across an await
+  --> $DIR/async-await-let-else.rs:41:14
+   |
+LL |         let r = Rc::new(());
+   |             - has type `Rc<()>` which is not `Send`
+LL |         bar().await;
+   |              ^^^^^^ await occurs here, with `r` maybe used later
+...
+LL |     };
+   |     - `r` is later dropped here
+note: required by a bound in `is_send`
+  --> $DIR/async-await-let-else.rs:19:15
+   |
+LL | fn is_send<T: Send>(_: T) {}
+   |               ^^^^ required by this bound in `is_send`
+
+error: aborting due to 4 previous errors
+
+For more information about this error, try `rustc --explain E0277`.

tests/ui/async-await/async-await-let-else.drop_tracking_mir.stderr

@@ -0,0 +1,100 @@
+error: future cannot be sent between threads safely
+  --> $DIR/async-await-let-else.rs:48:13
+   |
+LL |     is_send(foo(Some(true)));
+   |             ^^^^^^^^^^^^^^^ future returned by `foo` is not `Send`
+   |
+   = help: within `impl Future<Output = ()>`, the trait `Send` is not implemented for `Rc<()>`
+note: future is not `Send` as this value is used across an await
+  --> $DIR/async-await-let-else.rs:11:14
+   |
+LL |         let r = Rc::new(());
+   |             - has type `Rc<()>` which is not `Send`
+LL |         bar().await
+   |              ^^^^^^ await occurs here, with `r` maybe used later
+note: required by a bound in `is_send`
+  --> $DIR/async-await-let-else.rs:19:15
+   |
+LL | fn is_send<T: Send>(_: T) {}
+   |               ^^^^ required by this bound in `is_send`
+
+error[E0277]: `Rc<()>` cannot be sent between threads safely
+  --> $DIR/async-await-let-else.rs:50:13
+   |
+LL | async fn foo2(x: Option<bool>) {
+   |                                - within this `impl Future<Output = ()>`
+...
+LL |     is_send(foo2(Some(true)));
+   |     ------- ^^^^^^^^^^^^^^^^ `Rc<()>` cannot be sent between threads safely
+   |     |
+   |     required by a bound introduced by this call
+   |
+   = help: within `impl Future<Output = ()>`, the trait `Send` is not implemented for `Rc<()>`
+note: required because it's used within this `async fn` body
+  --> $DIR/async-await-let-else.rs:27:29
+   |
+LL |   async fn bar2<T>(_: T) -> ! {
+   |  _____________________________^
+LL | |     panic!()
+LL | | }
+   | |_^
+   = note: required because it captures the following types: `impl Future<Output = !>`
+note: required because it's used within this `async fn` body
+  --> $DIR/async-await-let-else.rs:21:32
+   |
+LL |   async fn foo2(x: Option<bool>) {
+   |  ________________________________^
+LL | |     let Some(_) = x else {
+LL | |         bar2(Rc::new(())).await
+LL | |     };
+LL | | }
+   | |_^
+note: required by a bound in `is_send`
+  --> $DIR/async-await-let-else.rs:19:15
+   |
+LL | fn is_send<T: Send>(_: T) {}
+   |               ^^^^ required by this bound in `is_send`
+
+error: future cannot be sent between threads safely
+  --> $DIR/async-await-let-else.rs:52:13
+   |
+LL |     is_send(foo3(Some(true)));
+   |             ^^^^^^^^^^^^^^^^ future returned by `foo3` is not `Send`
+   |
+   = help: within `impl Future<Output = ()>`, the trait `Send` is not implemented for `Rc<()>`
+note: future is not `Send` as this value is used across an await
+  --> $DIR/async-await-let-else.rs:33:28
+   |
+LL |         (Rc::new(()), bar().await);
+   |          -----------       ^^^^^^ await occurs here, with `Rc::new(())` maybe used later
+   |          |
+   |          has type `Rc<()>` which is not `Send`
+note: required by a bound in `is_send`
+  --> $DIR/async-await-let-else.rs:19:15
+   |
+LL | fn is_send<T: Send>(_: T) {}
+   |               ^^^^ required by this bound in `is_send`
+
+error: future cannot be sent between threads safely
+  --> $DIR/async-await-let-else.rs:54:13
+   |
+LL |     is_send(foo4(Some(true)));
+   |             ^^^^^^^^^^^^^^^^ future returned by `foo4` is not `Send`
+   |
+   = help: within `impl Future<Output = ()>`, the trait `Send` is not implemented for `Rc<()>`
+note: future is not `Send` as this value is used across an await
+  --> $DIR/async-await-let-else.rs:41:14
+   |
+LL |         let r = Rc::new(());
+   |             - has type `Rc<()>` which is not `Send`
+LL |         bar().await;
+   |              ^^^^^^ await occurs here, with `r` maybe used later
+note: required by a bound in `is_send`
+  --> $DIR/async-await-let-else.rs:19:15
+   |
+LL | fn is_send<T: Send>(_: T) {}
+   |               ^^^^ required by this bound in `is_send`
+
+error: aborting due to 4 previous errors
+
+For more information about this error, try `rustc --explain E0277`.

tests/ui/async-await/async-await-let-else.no_drop_tracking.stderr

@@ -0,0 +1,90 @@
+error: future cannot be sent between threads safely
+  --> $DIR/async-await-let-else.rs:48:13
+   |
+LL |     is_send(foo(Some(true)));
+   |             ^^^^^^^^^^^^^^^ future returned by `foo` is not `Send`
+   |
+   = help: within `impl Future<Output = ()>`, the trait `Send` is not implemented for `Rc<()>`
+note: future is not `Send` as this value is used across an await
+  --> $DIR/async-await-let-else.rs:11:14
+   |
+LL |         let r = Rc::new(());
+   |             - has type `Rc<()>` which is not `Send`
+LL |         bar().await
+   |              ^^^^^^ await occurs here, with `r` maybe used later
+LL |     };
+   |     - `r` is later dropped here
+note: required by a bound in `is_send`
+  --> $DIR/async-await-let-else.rs:19:15
+   |
+LL | fn is_send<T: Send>(_: T) {}
+   |               ^^^^ required by this bound in `is_send`
+
+error: future cannot be sent between threads safely
+  --> $DIR/async-await-let-else.rs:50:13
+   |
+LL |     is_send(foo2(Some(true)));
+   |             ^^^^^^^^^^^^^^^^ future returned by `foo2` is not `Send`
+   |
+   = help: within `impl Future<Output = ()>`, the trait `Send` is not implemented for `Rc<()>`
+note: future is not `Send` as this value is used across an await
+  --> $DIR/async-await-let-else.rs:23:26
+   |
+LL |         bar2(Rc::new(())).await
+   |              ----------- ^^^^^^ await occurs here, with `Rc::new(())` maybe used later
+   |              |
+   |              has type `Rc<()>` which is not `Send`
+LL |     };
+   |     - `Rc::new(())` is later dropped here
+note: required by a bound in `is_send`
+  --> $DIR/async-await-let-else.rs:19:15
+   |
+LL | fn is_send<T: Send>(_: T) {}
+   |               ^^^^ required by this bound in `is_send`
+
+error: future cannot be sent between threads safely
+  --> $DIR/async-await-let-else.rs:52:13
+   |
+LL |     is_send(foo3(Some(true)));
+   |             ^^^^^^^^^^^^^^^^ future returned by `foo3` is not `Send`
+   |
+   = help: within `impl Future<Output = ()>`, the trait `Send` is not implemented for `Rc<()>`
+note: future is not `Send` as this value is used across an await
+  --> $DIR/async-await-let-else.rs:33:28
+   |
+LL |         (Rc::new(()), bar().await);
+   |          -----------       ^^^^^^ - `Rc::new(())` is later dropped here
+   |          |                 |
+   |          |                 await occurs here, with `Rc::new(())` maybe used later
+   |          has type `Rc<()>` which is not `Send`
+note: required by a bound in `is_send`
+  --> $DIR/async-await-let-else.rs:19:15
+   |
+LL | fn is_send<T: Send>(_: T) {}
+   |               ^^^^ required by this bound in `is_send`
+
+error: future cannot be sent between threads safely
+  --> $DIR/async-await-let-else.rs:54:13
+   |
+LL |     is_send(foo4(Some(true)));
+   |             ^^^^^^^^^^^^^^^^ future returned by `foo4` is not `Send`
+   |
+   = help: within `impl Future<Output = ()>`, the trait `Send` is not implemented for `Rc<()>`
+note: future is not `Send` as this value is used across an await
+  --> $DIR/async-await-let-else.rs:41:14
+   |
+LL |         let r = Rc::new(());
+   |             - has type `Rc<()>` which is not `Send`
+LL |         bar().await;
+   |              ^^^^^^ await occurs here, with `r` maybe used later
+...
+LL |     };
+   |     - `r` is later dropped here
+note: required by a bound in `is_send`
+  --> $DIR/async-await-let-else.rs:19:15
+   |
+LL | fn is_send<T: Send>(_: T) {}
+   |               ^^^^ required by this bound in `is_send`
+
+error: aborting due to 4 previous errors
+

tests/ui/async-await/async-await-let-else.rs

@@ -1,7 +1,7 @@
 // edition:2021
-// revisions: drop-tracking no-drop-tracking
+// revisions: no_drop_tracking drop_tracking drop_tracking_mir
-// [drop-tracking] compile-flags: -Zdrop-tracking=yes
+// [drop_tracking] compile-flags: -Zdrop-tracking
-// [no-drop-tracking] compile-flags: -Zdrop-tracking=no
+// [drop_tracking_mir] compile-flags: -Zdrop-tracking-mir
 
 use std::rc::Rc;
 

tests/ui/async-await/async-error-span.drop_tracking.stderr

@@ -1,5 +1,5 @@
 error[E0277]: `()` is not a future
-  --> $DIR/async-error-span.rs:7:20
+  --> $DIR/async-error-span.rs:10:20
    |
 LL | fn get_future() -> impl Future<Output = ()> {
    |                    ^^^^^^^^^^^^^^^^^^^^^^^^ `()` is not a future
@@ -8,13 +8,13 @@ LL | fn get_future() -> impl Future<Output = ()> {
    = note: () must be a future or must implement `IntoFuture` to be awaited
 
 error[E0698]: type inside `async fn` body must be known in this context
-  --> $DIR/async-error-span.rs:13:9
+  --> $DIR/async-error-span.rs:16:9
    |
 LL |     let a;
    |         ^ cannot infer type
    |
 note: the type is part of the `async fn` body because of this `await`
-  --> $DIR/async-error-span.rs:14:17
+  --> $DIR/async-error-span.rs:19:17
    |
 LL |     get_future().await;
    |                 ^^^^^^

tests/ui/async-await/async-error-span.drop_tracking_mir.stderr

@@ -0,0 +1,24 @@
+error[E0277]: `()` is not a future
+  --> $DIR/async-error-span.rs:10:20
+   |
+LL | fn get_future() -> impl Future<Output = ()> {
+   |                    ^^^^^^^^^^^^^^^^^^^^^^^^ `()` is not a future
+   |
+   = help: the trait `Future` is not implemented for `()`
+   = note: () must be a future or must implement `IntoFuture` to be awaited
+
+error[E0282]: type annotations needed
+  --> $DIR/async-error-span.rs:16:9
+   |
+LL |     let a;
+   |         ^
+   |
+help: consider giving `a` an explicit type
+   |
+LL |     let a: /* Type */;
+   |          ++++++++++++
+
+error: aborting due to 2 previous errors
+
+Some errors have detailed explanations: E0277, E0282.
+For more information about an error, try `rustc --explain E0277`.

tests/ui/async-await/async-error-span.no_drop_tracking.stderr

@@ -0,0 +1,25 @@
+error[E0277]: `()` is not a future
+  --> $DIR/async-error-span.rs:10:20
+   |
+LL | fn get_future() -> impl Future<Output = ()> {
+   |                    ^^^^^^^^^^^^^^^^^^^^^^^^ `()` is not a future
+   |
+   = help: the trait `Future` is not implemented for `()`
+   = note: () must be a future or must implement `IntoFuture` to be awaited
+
+error[E0698]: type inside `async fn` body must be known in this context
+  --> $DIR/async-error-span.rs:16:9
+   |
+LL |     let a;
+   |         ^ cannot infer type
+   |
+note: the type is part of the `async fn` body because of this `await`
+  --> $DIR/async-error-span.rs:19:17
+   |
+LL |     get_future().await;
+   |                 ^^^^^^
+
+error: aborting due to 2 previous errors
+
+Some errors have detailed explanations: E0277, E0698.
+For more information about an error, try `rustc --explain E0277`.

tests/ui/async-await/async-error-span.rs

@@ -1,3 +1,6 @@
+// revisions: no_drop_tracking drop_tracking drop_tracking_mir
+// [drop_tracking] compile-flags: -Zdrop-tracking
+// [drop_tracking_mir] compile-flags: -Zdrop-tracking-mir
 // edition:2018
 
 // Regression test for issue #62382.
@@ -10,7 +13,9 @@ fn get_future() -> impl Future<Output = ()> {
 }
 
 async fn foo() {
-    let a; //~ ERROR type inside `async fn` body must be known in this context
+    let a;
+    //[no_drop_tracking,drop_tracking]~^ ERROR type inside `async fn` body must be known in this context
+    //[drop_tracking_mir]~^^ ERROR type annotations needed
     get_future().await;
 }
 

tests/ui/async-await/async-fn-nonsend.drop_tracking.stderr

@@ -0,0 +1,49 @@
+error: future cannot be sent between threads safely
+  --> $DIR/async-fn-nonsend.rs:72:17
+   |
+LL |     assert_send(non_send_temporary_in_match());
+   |                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ future returned by `non_send_temporary_in_match` is not `Send`
+   |
+   = help: within `impl Future<Output = ()>`, the trait `Send` is not implemented for `Rc<()>`
+note: future is not `Send` as this value is used across an await
+  --> $DIR/async-fn-nonsend.rs:36:25
+   |
+LL |     match Some(non_send()) {
+   |           ---------------- has type `Option<impl Debug>` which is not `Send`
+LL |         Some(_) => fut().await,
+   |                         ^^^^^^ await occurs here, with `Some(non_send())` maybe used later
+...
+LL | }
+   | - `Some(non_send())` is later dropped here
+note: required by a bound in `assert_send`
+  --> $DIR/async-fn-nonsend.rs:67:24
+   |
+LL | fn assert_send(_: impl Send) {}
+   |                        ^^^^ required by this bound in `assert_send`
+
+error: future cannot be sent between threads safely
+  --> $DIR/async-fn-nonsend.rs:74:17
+   |
+LL |     assert_send(non_sync_with_method_call());
+   |                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^ future returned by `non_sync_with_method_call` is not `Send`
+   |
+   = help: within `impl Future<Output = ()>`, the trait `Send` is not implemented for `dyn std::fmt::Write`
+note: future is not `Send` as this value is used across an await
+  --> $DIR/async-fn-nonsend.rs:49:14
+   |
+LL |     let f: &mut std::fmt::Formatter = &mut get_formatter();
+   |                                            --------------- has type `Formatter<'_>` which is not `Send`
+...
+LL |         fut().await;
+   |              ^^^^^^ await occurs here, with `get_formatter()` maybe used later
+LL |     }
+LL | }
+   | - `get_formatter()` is later dropped here
+note: required by a bound in `assert_send`
+  --> $DIR/async-fn-nonsend.rs:67:24
+   |
+LL | fn assert_send(_: impl Send) {}
+   |                        ^^^^ required by this bound in `assert_send`
+
+error: aborting due to 2 previous errors
+