bjoernager/rust
bjoernager/rust
1
Fork 0
Code Activity

Rollup merge of #102623 - davidtwco:translation-eager, r=compiler-errors

Browse source 
translation: eager translation

Part of #100717. See [Zulip thread](https://rust-lang.zulipchat.com/#narrow/stream/336883-i18n/topic/.23100717.20lists!/near/295010720) for additional context.

- **Store diagnostic arguments in a `HashMap`**: Eager translation will enable subdiagnostics to be translated multiple times with different arguments - this requires the ability to replace the value of one argument with a new value, which is better suited to a `HashMap` than the previous storage, a `Vec`.
- **Add `AddToDiagnostic::add_to_diagnostic_with`**: `AddToDiagnostic::add_to_diagnostic_with` is similar to the previous `AddToDiagnostic::add_to_diagnostic` but takes a function that can be used by the caller to modify diagnostic messages originating from the subdiagnostic (such as performing translation eagerly). `add_to_diagnostic` now just calls `add_to_diagnostic_with` with an empty closure.
- **Add `DiagnosticMessage::Eager`**: Add variant of `DiagnosticMessage` for eagerly translated messages
(messages in the target language which don't need translated by the emitter during emission). Also adds `eager_subdiagnostic` function which is intended to be invoked by the diagnostic derive for subdiagnostic fields which are marked as needing eager translation.
- **Support `#[subdiagnostic(eager)]`**: Add support for `eager` argument to the `subdiagnostic` attribute which generates a call to `eager_subdiagnostic`.
- **Finish migrating `rustc_query_system`**: Using eager translation, migrate the remaining repeated cycle stack diagnostic.
- **Split formatting initialization and use in diagnostic derives**: Diagnostic derives have previously had to take special care when ordering the generated code so that fields were not used after a move.

  This is unlikely for most fields because a field is either annotated with a subdiagnostic attribute and is thus likely a `Span` and copiable, or is a argument, in which case it is only used once by `set_arg`
anyway.

  However, format strings for code in suggestions can result in fields being used after being moved if not ordered carefully. As a result, the derive currently puts `set_arg` calls last (just before emission), such as:

      let diag = { /* create diagnostic */ };

      diag.span_suggestion_with_style(
          span,
          fluent::crate::slug,
          format!("{}", __binding_0),
          Applicability::Unknown,
          SuggestionStyle::ShowAlways
      );
      /* + other subdiagnostic additions */

      diag.set_arg("foo", __binding_0);
      /* + other `set_arg` calls */

      diag.emit();

  For eager translation, this doesn't work, as the message being translated eagerly can assume that all arguments are available - so arguments _must_ be set first.

  Format strings for suggestion code are now separated into two parts - an initialization line that performs the formatting into a variable, and a usage in the subdiagnostic addition.

  By separating these parts, the initialization can happen before arguments are set, preserving the desired order so that code compiles, while still enabling arguments to be set before subdiagnostics are added.

      let diag = { /* create diagnostic */ };

      let __code_0 = format!("{}", __binding_0);
      /* + other formatting */

      diag.set_arg("foo", __binding_0);
      /* + other `set_arg` calls */

      diag.span_suggestion_with_style(
          span,
          fluent::crate::slug,
          __code_0,
          Applicability::Unknown,
          SuggestionStyle::ShowAlways
      );
      /* + other subdiagnostic additions */

      diag.emit();

- **Remove field ordering logic in diagnostic derive:** Following the approach taken in earlier commits to separate formatting initialization from use in the subdiagnostic derive, simplify the diagnostic derive by removing the field-ordering logic that previously solved this problem.

r? ```@compiler-errors```
This commit is contained in:
Dylan DPC 2022-10-12 22:13:23 +05:30 • committed by GitHub
commit dc9f6f3243
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
26 changed files with 540 additions and 236 deletions
Download patch file Download diff file Expand all files Collapse all files

4
compiler/rustc_errors/src/annotate_snippet_emitter_writer.rs
View file

@ -7,7 +7,7 @@
use crate::emitter::FileWithAnnotatedLines;
use crate::snippet::Line;
use crate::translation::Translate;
use crate::translation::{to_fluent_args, Translate};
use crate::{
CodeSuggestion, Diagnostic, DiagnosticId, DiagnosticMessage, Emitter, FluentBundle,
LazyFallbackBundle, Level, MultiSpan, Style, SubDiagnostic,
@ -46,7 +46,7 @@ impl Translate for AnnotateSnippetEmitterWriter {
impl Emitter for AnnotateSnippetEmitterWriter {
/// The entry point for the diagnostics generation
fn emit_diagnostic(&mut self, diag: &Diagnostic) {
let fluent_args = self.to_fluent_args(diag.args());
let fluent_args = to_fluent_args(diag.args());
let mut children = diag.children.clone();
let (mut primary_span, suggestions) = self.primary_span_formatted(&diag, &fluent_args);

57
compiler/rustc_errors/src/diagnostic.rs
View file

@ -27,7 +27,11 @@ pub struct SuggestionsDisabled;
/// Simplified version of `FluentArg` that can implement `Encodable` and `Decodable`. Collection of
/// `DiagnosticArg` are converted to `FluentArgs` (consuming the collection) at the start of
/// diagnostic emission.
pub type DiagnosticArg<'source> = (Cow<'source, str>, DiagnosticArgValue<'source>);
pub type DiagnosticArg<'iter, 'source> =
(&'iter DiagnosticArgName<'source>, &'iter DiagnosticArgValue<'source>);
/// Name of a diagnostic argument.
pub type DiagnosticArgName<'source> = Cow<'source, str>;
/// Simplified version of `FluentValue` that can implement `Encodable` and `Decodable`. Converted
/// to a `FluentValue` by the emitter to be used in diagnostic translation.
@ -199,9 +203,20 @@ impl IntoDiagnosticArg for ast::token::TokenKind {
/// `#[derive(Subdiagnostic)]` -- see [rustc_macros::Subdiagnostic].
#[cfg_attr(bootstrap, rustc_diagnostic_item = "AddSubdiagnostic")]
#[cfg_attr(not(bootstrap), rustc_diagnostic_item = "AddToDiagnostic")]
pub trait AddToDiagnostic {
pub trait AddToDiagnostic
where
Self: Sized,
{
/// Add a subdiagnostic to an existing diagnostic.
fn add_to_diagnostic(self, diag: &mut Diagnostic);
fn add_to_diagnostic(self, diag: &mut Diagnostic) {
self.add_to_diagnostic_with(diag, |_, m| m);
}
/// Add a subdiagnostic to an existing diagnostic where `f` is invoked on every message used
/// (to optionally perform eager translation).
fn add_to_diagnostic_with<F>(self, diag: &mut Diagnostic, f: F)
where
F: Fn(&mut Diagnostic, SubdiagnosticMessage) -> SubdiagnosticMessage;
}
/// Trait implemented by lint types. This should not be implemented manually. Instead, use
@ -229,7 +244,7 @@ pub struct Diagnostic {
pub span: MultiSpan,
pub children: Vec<SubDiagnostic>,
pub suggestions: Result<Vec<CodeSuggestion>, SuggestionsDisabled>,
args: Vec<DiagnosticArg<'static>>,
args: FxHashMap<DiagnosticArgName<'static>, DiagnosticArgValue<'static>>,
/// This is not used for highlighting or rendering any error message. Rather, it can be used
/// as a sort key to sort a buffer of diagnostics. By default, it is the primary span of
@ -321,7 +336,7 @@ impl Diagnostic {
span: MultiSpan::new(),
children: vec![],
suggestions: Ok(vec![]),
args: vec![],
args: Default::default(),
sort_span: DUMMY_SP,
is_lint: false,
}
@ -917,13 +932,30 @@ impl Diagnostic {
self
}
/// Add a subdiagnostic from a type that implements `Subdiagnostic` - see
/// [rustc_macros::Subdiagnostic].
/// Add a subdiagnostic from a type that implements `Subdiagnostic` (see
/// [rustc_macros::Subdiagnostic]).
pub fn subdiagnostic(&mut self, subdiagnostic: impl AddToDiagnostic) -> &mut Self {
subdiagnostic.add_to_diagnostic(self);
self
}
/// Add a subdiagnostic from a type that implements `Subdiagnostic` (see
/// [rustc_macros::Subdiagnostic]). Performs eager translation of any translatable messages
/// used in the subdiagnostic, so suitable for use with repeated messages (i.e. re-use of
/// interpolated variables).
pub fn eager_subdiagnostic(
&mut self,
handler: &crate::Handler,
subdiagnostic: impl AddToDiagnostic,
) -> &mut Self {
subdiagnostic.add_to_diagnostic_with(self, |diag, msg| {
let args = diag.args();
let msg = diag.subdiagnostic_message_to_diagnostic_message(msg);
handler.eagerly_translate(msg, args)
});
self
}
pub fn set_span<S: Into<MultiSpan>>(&mut self, sp: S) -> &mut Self {
self.span = sp.into();
if let Some(span) = self.span.primary_span() {
@ -956,8 +988,11 @@ impl Diagnostic {
self
}
pub fn args(&self) -> &[DiagnosticArg<'static>] {
&self.args
// Exact iteration order of diagnostic arguments shouldn't make a difference to output because
// they're only used in interpolation.
#[allow(rustc::potential_query_instability)]
pub fn args<'a>(&'a self) -> impl Iterator<Item = DiagnosticArg<'a, 'static>> {
self.args.iter()
}
pub fn set_arg(
@ -965,7 +1000,7 @@ impl Diagnostic {
name: impl Into<Cow<'static, str>>,
arg: impl IntoDiagnosticArg,
) -> &mut Self {
self.args.push((name.into(), arg.into_diagnostic_arg()));
self.args.insert(name.into(), arg.into_diagnostic_arg());
self
}
@ -976,7 +1011,7 @@ impl Diagnostic {
/// Helper function that takes a `SubdiagnosticMessage` and returns a `DiagnosticMessage` by
/// combining it with the primary message of the diagnostic (if translatable, otherwise it just
/// passes the user's string along).
fn subdiagnostic_message_to_diagnostic_message(
pub(crate) fn subdiagnostic_message_to_diagnostic_message(
&self,
attr: impl Into<SubdiagnosticMessage>,
) -> DiagnosticMessage {

4
compiler/rustc_errors/src/emitter.rs
View file

@ -14,7 +14,7 @@ use rustc_span::{FileLines, SourceFile, Span};
use crate::snippet::{Annotation, AnnotationType, Line, MultilineAnnotation, Style, StyledString};
use crate::styled_buffer::StyledBuffer;
use crate::translation::Translate;
use crate::translation::{to_fluent_args, Translate};
use crate::{
CodeSuggestion, Diagnostic, DiagnosticId, DiagnosticMessage, FluentBundle, Handler,
LazyFallbackBundle, Level, MultiSpan, SubDiagnostic, SubstitutionHighlight, SuggestionStyle,
@ -535,7 +535,7 @@ impl Emitter for EmitterWriter {
}
fn emit_diagnostic(&mut self, diag: &Diagnostic) {
let fluent_args = self.to_fluent_args(diag.args());
let fluent_args = to_fluent_args(diag.args());
let mut children = diag.children.clone();
let (mut primary_span, suggestions) = self.primary_span_formatted(&diag, &fluent_args);

4
compiler/rustc_errors/src/json.rs
View file

@ -13,7 +13,7 @@ use rustc_span::source_map::{FilePathMapping, SourceMap};
use crate::emitter::{Emitter, HumanReadableErrorType};
use crate::registry::Registry;
use crate::translation::Translate;
use crate::translation::{to_fluent_args, Translate};
use crate::DiagnosticId;
use crate::{
CodeSuggestion, FluentBundle, LazyFallbackBundle, MultiSpan, SpanLabel, SubDiagnostic,
@ -312,7 +312,7 @@ struct UnusedExterns<'a, 'b, 'c> {
impl Diagnostic {
fn from_errors_diagnostic(diag: &crate::Diagnostic, je: &JsonEmitter) -> Diagnostic {
let args = je.to_fluent_args(diag.args());
let args = to_fluent_args(diag.args());
let sugg = diag.suggestions.iter().flatten().map(|sugg| {
let translated_message = je.translate_message(&sugg.msg, &args);
Diagnostic {

11
compiler/rustc_errors/src/lib.rs
View file

@ -598,6 +598,17 @@ impl Handler {
}
}
/// Translate `message` eagerly with `args`.
pub fn eagerly_translate<'a>(
&self,
message: DiagnosticMessage,
args: impl Iterator<Item = DiagnosticArg<'a, 'static>>,
) -> SubdiagnosticMessage {
let inner = self.inner.borrow();
let args = crate::translation::to_fluent_args(args);
SubdiagnosticMessage::Eager(inner.emitter.translate_message(&message, &args).to_string())
}
// This is here to not allow mutation of flags;
// as of this writing it's only used in tests in librustc_middle.
pub fn can_emit_warnings(&self) -> bool {

34
compiler/rustc_errors/src/translation.rs
View file

@ -4,6 +4,27 @@ use rustc_data_structures::sync::Lrc;
use rustc_error_messages::FluentArgs;
use std::borrow::Cow;
/// Convert diagnostic arguments (a rustc internal type that exists to implement
/// `Encodable`/`Decodable`) into `FluentArgs` which is necessary to perform translation.
///
/// Typically performed once for each diagnostic at the start of `emit_diagnostic` and then
/// passed around as a reference thereafter.
pub fn to_fluent_args<'iter, 'arg: 'iter>(
iter: impl Iterator<Item = DiagnosticArg<'iter, 'arg>>,
) -> FluentArgs<'arg> {
let mut args = if let Some(size) = iter.size_hint().1 {
FluentArgs::with_capacity(size)
} else {
FluentArgs::new()
};
for (k, v) in iter {
args.set(k.clone(), v.clone());
}
args
}
pub trait Translate {
/// Return `FluentBundle` with localized diagnostics for the locale requested by the user. If no
/// language was requested by the user then this will be `None` and `fallback_fluent_bundle`
@ -15,15 +36,6 @@ pub trait Translate {
/// unavailable for the requested locale.
fn fallback_fluent_bundle(&self) -> &FluentBundle;
/// Convert diagnostic arguments (a rustc internal type that exists to implement
/// `Encodable`/`Decodable`) into `FluentArgs` which is necessary to perform translation.
///
/// Typically performed once for each diagnostic at the start of `emit_diagnostic` and then
/// passed around as a reference thereafter.
fn to_fluent_args<'arg>(&self, args: &[DiagnosticArg<'arg>]) -> FluentArgs<'arg> {
FromIterator::from_iter(args.iter().cloned())
}
/// Convert `DiagnosticMessage`s to a string, performing translation if necessary.
fn translate_messages(
&self,
@ -43,7 +55,9 @@ pub trait Translate {
) -> Cow<'_, str> {
trace!(?message, ?args);
let (identifier, attr) = match message {
DiagnosticMessage::Str(msg) => return Cow::Borrowed(&msg),
DiagnosticMessage::Str(msg) | DiagnosticMessage::Eager(msg) => {
return Cow::Borrowed(&msg);
}
DiagnosticMessage::FluentIdentifier(identifier, attr) => (identifier, attr),
};