Auto merge of #87835 - xFrednet:rfc-2383-expect-attribute-with-ids, r=wesleywiser

Implementation of the `expect` attribute (RFC 2383)

This is an implementation of the `expect` attribute as described in [RFC-2383](https://rust-lang.github.io/rfcs/2383-lint-reasons.html). The attribute allows the suppression of lint message by expecting them. Unfulfilled lint expectations (meaning no expected lint was caught) will emit the `unfulfilled_lint_expectations` lint at the `expect` attribute.

### Example
#### input
```rs
// required feature flag
#![feature(lint_reasons)]

#[expect(unused_mut)] // Will warn about an unfulfilled expectation
#[expect(unused_variables)] // Will be fulfilled by x
fn main() {
    let x = 0;
}
```

#### output

```txt
warning: this lint expectation is unfulfilled
  --> $DIR/trigger_lint.rs:3:1
   |
LL | #[expect(unused_mut)] // Will warn about an unfulfilled expectation
   |          ^^^^^^^^^^
   |
   = note: `#[warn(unfulfilled_lint_expectations)]` on by default
```

### Implementation

This implementation introduces `Expect` as a new lint level for diagnostics, which have been expected. All lint expectations marked via the `expect` attribute are collected in the [`LintLevelsBuilder`] and assigned an ID that is stored in the new lint level. The `LintLevelsBuilder` stores all found expectations and the data needed to emit the `unfulfilled_lint_expectations` in the [`LintLevelsMap`] which is the result of the [`lint_levels()`] query.

The [`rustc_errors::HandlerInner`] is the central error handler in rustc and handles the emission of all diagnostics. Lint message with the level `Expect` are suppressed during this emission, while the expectation ID is stored in a set which marks them as fulfilled. The last step is then so simply check if all expectations collected by the [`LintLevelsBuilder`] in the [`LintLevelsMap`] have been marked as fulfilled in the [`rustc_errors::HandlerInner`]. Otherwise, a new lint message will be emitted.

The implementation of the `LintExpectationId` required some special handling to make it stable between sessions. Lints can be emitted during [`EarlyLintPass`]es. At this stage, it's not possible to create a stable identifier. The level instead stores an unstable identifier, which is later converted to a stable `LintExpectationId`.

### Followup TO-DOs
All open TO-DOs have been marked with `FIXME` comments in the code. This is the combined list of them:

* [ ] The current implementation doesn't cover cases where the `unfulfilled_lint_expectations` lint is actually expected by another `expect` attribute.
   * This should be easily possible, but I wanted to get some feedback before putting more work into this.
   * This could also be done in a new PR to not add to much more code to this one
* [ ] Update unstable documentation to reflect this change.
* [ ] Update unstable expectation ids in [`HandlerInner::stashed_diagnostics`](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_errors/struct.HandlerInner.html#structfield.stashed_diagnostics)

### Open questions
I also have a few open questions where I would like to get feedback on:
1. The RFC discussion included a suggestion to change the `expect` attribute to something else. (Initiated by `@Ixrec` [here](https://github.com/rust-lang/rfcs/pull/2383#issuecomment-378424091), suggestion from `@scottmcm` to use `#[should_lint(...)]` [here](https://github.com/rust-lang/rfcs/pull/2383#issuecomment-378648877)). No real conclusion was drawn on that point from my understanding. Is this still open for discussion, or was this discarded with the merge of the RFC?
2. How should the expect attribute deal with the new `force-warn` lint level?

---

This approach was inspired by a discussion with `@LeSeulArtichaut.`

RFC tracking issue: #54503

Mentoring/Implementation issue: #85549

[`LintLevelsBuilder`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_lint/levels/struct.LintLevelsBuilder.html
[`LintLevelsMap`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/lint/struct.LintLevelMap.html
[`lint_levels()`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/context/struct.TyCtxt.html#method.lint_levels
[`rustc_errors::HandlerInner`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_errors/struct.HandlerInner.html
[`EarlyLintPass`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_lint/trait.EarlyLintPass.html

compiler/rustc_lint/src/context.rs

@@ -109,6 +109,7 @@ struct LintGroup {
     depr: Option<LintAlias>,
 }
 
+#[derive(Debug)]
 pub enum CheckLintNameResult<'a> {
     Ok(&'a [LintId]),
     /// Lint doesn't exist. Potentially contains a suggestion for a correct lint name.
@@ -377,6 +378,9 @@ impl LintStore {
                     Level::ForceWarn => "--force-warn",
                     Level::Deny => "-D",
                     Level::Forbid => "-F",
+                    Level::Expect(_) => {
+                        unreachable!("lints with the level of `expect` should not run this code");
+                    }
                 },
                 lint_name
             );

compiler/rustc_lint/src/early.rs

@@ -59,7 +59,8 @@ impl<'a, T: EarlyLintPass> EarlyContextAndPass<'a, T> {
         F: FnOnce(&mut Self),
     {
         let is_crate_node = id == ast::CRATE_NODE_ID;
-        let push = self.context.builder.push(attrs, is_crate_node);
+        let push = self.context.builder.push(attrs, is_crate_node, None);
+
         self.check_id(id);
         self.enter_attrs(attrs);
         f(self);

compiler/rustc_lint/src/expect.rs

@@ -0,0 +1,49 @@
+use crate::builtin;
+use rustc_hir::HirId;
+use rustc_middle::{lint::LintExpectation, ty::TyCtxt};
+use rustc_session::lint::LintExpectationId;
+use rustc_span::symbol::sym;
+
+pub fn check_expectations(tcx: TyCtxt<'_>) {
+    if !tcx.sess.features_untracked().enabled(sym::lint_reasons) {
+        return;
+    }
+
+    let fulfilled_expectations = tcx.sess.diagnostic().steal_fulfilled_expectation_ids();
+    let lint_expectations = &tcx.lint_levels(()).lint_expectations;
+
+    for (id, expectation) in lint_expectations {
+        if !fulfilled_expectations.contains(id) {
+            // This check will always be true, since `lint_expectations` only
+            // holds stable ids
+            if let LintExpectationId::Stable { hir_id, .. } = id {
+                emit_unfulfilled_expectation_lint(tcx, *hir_id, expectation);
+            } else {
+                unreachable!("at this stage all `LintExpectationId`s are stable");
+            }
+        }
+    }
+}
+
+fn emit_unfulfilled_expectation_lint(
+    tcx: TyCtxt<'_>,
+    hir_id: HirId,
+    expectation: &LintExpectation,
+) {
+    // FIXME: The current implementation doesn't cover cases where the
+    // `unfulfilled_lint_expectations` is actually expected by another lint
+    // expectation. This can be added here by checking the lint level and
+    // retrieving the `LintExpectationId` if it was expected.
+    tcx.struct_span_lint_hir(
+        builtin::UNFULFILLED_LINT_EXPECTATIONS,
+        hir_id,
+        expectation.emission_span,
+        |diag| {
+            let mut diag = diag.build("this lint expectation is unfulfilled");
+            if let Some(rationale) = expectation.reason {
+                diag.note(&rationale.as_str());
+            }
+            diag.emit();
+        },
+    );
+}

compiler/rustc_lint/src/late.rs

@@ -503,4 +503,7 @@ pub fn check_crate<'tcx, T: LateLintPass<'tcx>>(
             });
         },
     );
+
+    // This check has to be run after all lints are done processing for this crate
+    tcx.sess.time("check_lint_expectations", || crate::expect::check_expectations(tcx));
 }

compiler/rustc_lint/src/levels.rs

@@ -7,17 +7,15 @@ use rustc_errors::{struct_span_err, Applicability, Diagnostic};
 use rustc_hir as hir;
 use rustc_hir::{intravisit, HirId};
 use rustc_middle::hir::nested_filter;
-use rustc_middle::lint::LevelAndSource;
-use rustc_middle::lint::LintDiagnosticBuilder;
 use rustc_middle::lint::{
-    struct_lint_level, LintLevelMap, LintLevelSets, LintLevelSource, LintSet, LintStackIndex,
-    COMMAND_LINE,
+    struct_lint_level, LevelAndSource, LintDiagnosticBuilder, LintExpectation, LintLevelMap,
+    LintLevelSets, LintLevelSource, LintSet, LintStackIndex, COMMAND_LINE,
 };
 use rustc_middle::ty::query::Providers;
 use rustc_middle::ty::{RegisteredTools, TyCtxt};
 use rustc_session::lint::{
     builtin::{self, FORBIDDEN_LINT_GROUPS},
-    Level, Lint, LintId,
+    Level, Lint, LintExpectationId, LintId,
 };
 use rustc_session::parse::feature_err;
 use rustc_session::Session;
@@ -34,16 +32,23 @@ fn lint_levels(tcx: TyCtxt<'_>, (): ()) -> LintLevelMap {
 
     builder.levels.id_to_set.reserve(krate.owners.len() + 1);
 
-    let push = builder.levels.push(tcx.hir().attrs(hir::CRATE_HIR_ID), true);
+    let push =
+        builder.levels.push(tcx.hir().attrs(hir::CRATE_HIR_ID), true, Some(hir::CRATE_HIR_ID));
+
     builder.levels.register_id(hir::CRATE_HIR_ID);
     tcx.hir().walk_toplevel_module(&mut builder);
     builder.levels.pop(push);
 
+    builder.levels.update_unstable_expectation_ids();
     builder.levels.build_map()
 }
 
 pub struct LintLevelsBuilder<'s> {
     sess: &'s Session,
+    lint_expectations: Vec<(LintExpectationId, LintExpectation)>,
+    /// Each expectation has a stable and an unstable identifier. This map
+    /// is used to map from unstable to stable [`LintExpectationId`]s.
+    expectation_id_map: FxHashMap<LintExpectationId, LintExpectationId>,
     sets: LintLevelSets,
     id_to_set: FxHashMap<HirId, LintStackIndex>,
     cur: LintStackIndex,
@@ -66,6 +71,8 @@ impl<'s> LintLevelsBuilder<'s> {
     ) -> Self {
         let mut builder = LintLevelsBuilder {
             sess,
+            lint_expectations: Default::default(),
+            expectation_id_map: Default::default(),
             sets: LintLevelSets::new(),
             cur: COMMAND_LINE,
             id_to_set: Default::default(),
@@ -226,13 +233,24 @@ impl<'s> LintLevelsBuilder<'s> {
     ///   `#[allow]`
     ///
     /// Don't forget to call `pop`!
-    pub(crate) fn push(&mut self, attrs: &[ast::Attribute], is_crate_node: bool) -> BuilderPush {
+    pub(crate) fn push(
+        &mut self,
+        attrs: &[ast::Attribute],
+        is_crate_node: bool,
+        source_hir_id: Option<HirId>,
+    ) -> BuilderPush {
         let mut specs = FxHashMap::default();
         let sess = self.sess;
         let bad_attr = |span| struct_span_err!(sess, span, E0452, "malformed lint attribute input");
-        for attr in attrs {
-            let Some(level) = Level::from_symbol(attr.name_or_empty()) else {
-                continue
+        for (attr_index, attr) in attrs.iter().enumerate() {
+            let level = match Level::from_attr(attr) {
+                None => continue,
+                Some(Level::Expect(unstable_id)) if let Some(hir_id) = source_hir_id => {
+                    let stable_id = self.create_stable_id(unstable_id, hir_id, attr_index);
+
+                    Level::Expect(stable_id)
+                }
+                Some(lvl) => lvl,
             };
 
             let Some(mut metas) = attr.meta_item_list() else {
@@ -285,9 +303,17 @@ impl<'s> LintLevelsBuilder<'s> {
                 }
             }
 
-            for li in metas {
+            for (lint_index, li) in metas.iter_mut().enumerate() {
+                let level = match level {
+                    Level::Expect(mut id) => {
+                        id.set_lint_index(Some(lint_index as u16));
+                        Level::Expect(id)
+                    }
+                    level => level,
+                };
+
                 let sp = li.span();
-                let mut meta_item = match li {
+                let meta_item = match li {
                     ast::NestedMetaItem::MetaItem(meta_item) if meta_item.is_word() => meta_item,
                     _ => {
                         let mut err = bad_attr(sp);
@@ -327,6 +353,10 @@ impl<'s> LintLevelsBuilder<'s> {
                             self.check_gated_lint(id, attr.span);
                             self.insert_spec(&mut specs, id, (level, src));
                         }
+                        if let Level::Expect(expect_id) = level {
+                            self.lint_expectations
+                                .push((expect_id, LintExpectation::new(reason, sp)));
+                        }
                     }
 
                     CheckLintNameResult::Tool(result) => {
@@ -342,6 +372,10 @@ impl<'s> LintLevelsBuilder<'s> {
                                 for id in ids {
                                     self.insert_spec(&mut specs, *id, (level, src));
                                 }
+                                if let Level::Expect(expect_id) = level {
+                                    self.lint_expectations
+                                        .push((expect_id, LintExpectation::new(reason, sp)));
+                                }
                             }
                             Err((Some(ids), ref new_lint_name)) => {
                                 let lint = builtin::RENAMED_AND_REMOVED_LINTS;
@@ -378,6 +412,10 @@ impl<'s> LintLevelsBuilder<'s> {
                                 for id in ids {
                                     self.insert_spec(&mut specs, *id, (level, src));
                                 }
+                                if let Level::Expect(expect_id) = level {
+                                    self.lint_expectations
+                                        .push((expect_id, LintExpectation::new(reason, sp)));
+                                }
                             }
                             Err((None, _)) => {
                                 // If Tool(Err(None, _)) is returned, then either the lint does not
@@ -471,6 +509,10 @@ impl<'s> LintLevelsBuilder<'s> {
                             self.check_gated_lint(id, attr.span);
                             self.insert_spec(&mut specs, id, (level, src));
                         }
+                        if let Level::Expect(expect_id) = level {
+                            self.lint_expectations
+                                .push((expect_id, LintExpectation::new(reason, sp)));
+                        }
                     } else {
                         panic!("renamed lint does not exist: {}", new_name);
                     }
@@ -519,6 +561,20 @@ impl<'s> LintLevelsBuilder<'s> {
         BuilderPush { prev, changed: prev != self.cur }
     }
 
+    fn create_stable_id(
+        &mut self,
+        unstable_id: LintExpectationId,
+        hir_id: HirId,
+        attr_index: usize,
+    ) -> LintExpectationId {
+        let stable_id =
+            LintExpectationId::Stable { hir_id, attr_index: attr_index as u16, lint_index: None };
+
+        self.expectation_id_map.insert(unstable_id, stable_id);
+
+        stable_id
+    }
+
     /// Checks if the lint is gated on a feature that is not enabled.
     fn check_gated_lint(&self, lint_id: LintId, span: Span) {
         if let Some(feature) = lint_id.lint.feature_gate {
@@ -562,8 +618,16 @@ impl<'s> LintLevelsBuilder<'s> {
         self.id_to_set.insert(id, self.cur);
     }
 
+    fn update_unstable_expectation_ids(&self) {
+        self.sess.diagnostic().update_unstable_expectation_id(&self.expectation_id_map);
+    }
+
     pub fn build_map(self) -> LintLevelMap {
-        LintLevelMap { sets: self.sets, id_to_set: self.id_to_set }
+        LintLevelMap {
+            sets: self.sets,
+            id_to_set: self.id_to_set,
+            lint_expectations: self.lint_expectations,
+        }
     }
 }
 
@@ -579,7 +643,8 @@ impl LintLevelMapBuilder<'_> {
     {
         let is_crate_hir = id == hir::CRATE_HIR_ID;
         let attrs = self.tcx.hir().attrs(id);
-        let push = self.levels.push(attrs, is_crate_hir);
+        let push = self.levels.push(attrs, is_crate_hir, Some(id));
+
         if push.changed {
             self.levels.register_id(id);
         }

compiler/rustc_lint/src/lib.rs

@@ -51,6 +51,7 @@ pub mod builtin;
 mod context;
 mod early;
 mod enum_intrinsics_non_enums;
+mod expect;
 pub mod hidden_unicode_codepoints;
 mod internal;
 mod late;