Auto merge of #115898 - onur-ozkan:config-change-tracking, r=Mark-Simulacrum

bootstrap major change detection implementation

The use of `changelog-seen` and `bootstrap/CHANGELOG.md` has not been functional in any way for many years. We often do major/breaking changes but never update the changelog file or the `changelog-seen`. This is an alternative method for tracking major or breaking changes and informing developers when such changes occur.

Example output when bootstrap detects a major change:
![image](ee802dfa-a02b-488b-a433-f853ce079b8a)

compiler/rustc_codegen_cranelift/scripts/setup_rust_fork.sh

@@ -31,7 +31,7 @@ index d95b5b7f17f..00b6f0e3635 100644
 EOF
 
 cat > config.toml <<EOF
-changelog-seen = 2
+change-id = 115898
 
 [llvm]
 ninja = false

compiler/rustc_codegen_gcc/test.sh

@@ -214,7 +214,7 @@ function setup_rustc() {
     rm config.toml || true
 
     cat > config.toml <<EOF
-changelog-seen = 2
+change-id = 115898
 
 [rust]
 codegen-backends = []

config.example.toml

@@ -19,11 +19,18 @@
 # Note that this has no default value (x.py uses the defaults in `config.example.toml`).
 #profile = <none>
 
-# Keeps track of the last version of `x.py` used.
+# Keeps track of major changes made to this configuration.
-# If `changelog-seen` does not match the version that is currently running,
+#
-# `x.py` will prompt you to update it and to read the changelog.
+# This value also represents ID of the PR that caused major changes. Meaning,
-# See `src/bootstrap/CHANGELOG.md` for more information.
+# you can visit github.com/rust-lang/rust/pull/{change-id} to check for more details.
-changelog-seen = 2
+#
+# A 'major change' includes any of the following
+#  - A new option
+#  - A change in the default values
+#
+# If `change-id` does not match the version that is currently running,
+# `x.py` will prompt you to update it and check the related PR for more details.
+change-id = 115898
 
 # =============================================================================
 # Tweaking how LLVM is compiled

src/bootstrap/CHANGELOG.md

@@ -1,71 +0,0 @@
-# Changelog
-
-All notable changes to bootstrap will be documented in this file.
-
-The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
-
-
-## [Changes since the last major version]
-
-- Vendoring is no longer done automatically when building from git sources. To use vendoring, run `cargo vendor` manually, or use the pre-vendored `rustc-src` tarball.
-- `llvm-libunwind` now accepts `in-tree` (formerly true), `system` or `no` (formerly false) [#77703](https://github.com/rust-lang/rust/pull/77703)
-- The options `infodir`, `localstatedir`, and `gpg-password-file` are no longer allowed in config.toml. Previously, they were ignored without warning. Note that `infodir` and `localstatedir` are still accepted by `./configure`, with a warning. [#82451](https://github.com/rust-lang/rust/pull/82451)
-- Change the names for `dist` commands to match the component they generate. [#90684](https://github.com/rust-lang/rust/pull/90684)
-- The `build.fast-submodules` option has been removed. Fast submodule checkouts are enabled unconditionally. Automatic submodule handling can still be disabled with `build.submodules = false`.
-- Several unsupported `./configure` options have been removed: `optimize`, `parallel-compiler`. These can still be enabled with `--set`, although it isn't recommended.
-- `remote-test-server`'s `verbose` argument has been removed in favor of the `--verbose` flag
-- `remote-test-server`'s `remote` argument has been removed in favor of the `--bind` flag. Use `--bind 0.0.0.0:12345` to replicate the behavior of the `remote` argument.
-- `x.py fmt` now formats only files modified between the merge-base of HEAD and the last commit in the master branch of the rust-lang repository and the current working directory. To restore old behaviour, use `x.py fmt .`. The check mode is not affected by this change. [#105702](https://github.com/rust-lang/rust/pull/105702)
-- The `llvm.version-check` config option has been removed. Older versions were never supported. If you still need to support older versions (e.g. you are applying custom patches), patch `check_llvm_version` in bootstrap to change the minimum version. [#108619](https://github.com/rust-lang/rust/pull/108619)
-- The `rust.ignore-git` option has been renamed to `rust.omit-git-hash`. [#110059](https://github.com/rust-lang/rust/pull/110059)
-- `--exclude` no longer accepts a `Kind` as part of a Step; instead it uses the top-level Kind of the subcommand. If this matches how you were already using --exclude (e.g. `x test --exclude test::std`), simply remove the kind: `--exclude std`. If you were using a kind that did not match the top-level subcommand, please open an issue explaining why you wanted this feature.
-
-### Non-breaking changes
-
-- `x.py check` needs opt-in to check tests (--all-targets) [#77473](https://github.com/rust-lang/rust/pull/77473)
-- The default bootstrap profiles are now located at `bootstrap/defaults/config.$PROFILE.toml` (previously they were located at `bootstrap/defaults/config.toml.$PROFILE`) [#77558](https://github.com/rust-lang/rust/pull/77558)
-- If you have Rust already installed, `x.py` will now infer the host target
-  from the default rust toolchain. [#78513](https://github.com/rust-lang/rust/pull/78513)
-- Add options for enabling overflow checks, one for std (`overflow-checks-std`) and one for everything else (`overflow-checks`). Both default to false.
-- Add llvm option `enable-warnings` to have control on llvm compilation warnings. Default to false.
-- Add `rpath` option in `target` section to support set rpath option for each target independently. [#111242](https://github.com/rust-lang/rust/pull/111242)
-
-
-## [Version 2] - 2020-09-25
-
-- `host` now defaults to the value of `build` in all cases
-  + Previously `host` defaulted to an empty list when `target` was overridden, and to `build` otherwise
-
-### Non-breaking changes
-
-- Add `x.py setup` [#76631](https://github.com/rust-lang/rust/pull/76631)
-- Add a changelog for x.py [#76626](https://github.com/rust-lang/rust/pull/76626)
-- Optionally, download LLVM from CI on Linux and NixOS. This can be enabled with `download-ci-llvm = true` under `[llvm]`.
-  + [#76439](https://github.com/rust-lang/rust/pull/76349)
-  + [#76667](https://github.com/rust-lang/rust/pull/76667)
-  + [#76708](https://github.com/rust-lang/rust/pull/76708)
-- Distribute rustc sources as part of `rustc-dev` [#76856](https://github.com/rust-lang/rust/pull/76856)
-- Make the default stage for x.py configurable [#76625](https://github.com/rust-lang/rust/pull/76625). This can be enabled with `build-stage = N`, `doc-stage = N`, etc.
-- Add a dedicated debug-logging option [#76588](https://github.com/rust-lang/rust/pull/76588). Previously, `debug-logging` could only be set with `debug-assertions`, slowing down the compiler more than necessary.
-- Add sample defaults for x.py [#76628](https://github.com/rust-lang/rust/pull/76628)
-- Add `--keep-stage-std`, which behaves like `keep-stage` but allows the stage
-  0 compiler artifacts (i.e., stage1/bin/rustc) to be rebuilt if changed
-  [#77120](https://github.com/rust-lang/rust/pull/77120).
-- File locking is now used to avoid collisions between multiple running instances of `x.py` (e.g. when using `rust-analyzer` and `x.py` at the same time). Note that Solaris and possibly other non Unix and non Windows systems don't support it [#108607](https://github.com/rust-lang/rust/pull/108607). This might possibly lead to build data corruption.
-
-
-## [Version 1] - 2020-09-11
-
-This is the first changelog entry, and it does not attempt to be an exhaustive list of features in x.py.
-Instead, this documents the changes to bootstrap in the past 2 months.
-
-- Improve defaults in `x.py` [#73964](https://github.com/rust-lang/rust/pull/73964)
-  (see [blog post] for details)
-- Set `ninja = true` by default [#74922](https://github.com/rust-lang/rust/pull/74922)
-- Avoid trying to inversely cross-compile for build triple from host triples [#76415](https://github.com/rust-lang/rust/pull/76415)
-- Allow blessing expect-tests in tools [#75975](https://github.com/rust-lang/rust/pull/75975)
-- `x.py check` checks tests/examples/benches [#76258](https://github.com/rust-lang/rust/pull/76258)
-- Fix `rust.use-lld` when linker is not set [#76326](https://github.com/rust-lang/rust/pull/76326)
-- Build tests with LLD if `use-lld = true` was passed [#76378](https://github.com/rust-lang/rust/pull/76378)
-
-[blog post]: https://blog.rust-lang.org/inside-rust/2020/08/30/changes-to-x-py-defaults.html

src/bootstrap/README.md

@@ -181,11 +181,10 @@ Some general areas that you may be interested in modifying are:
   `Config` struct.
 * Adding a sanity check? Take a look at `bootstrap/sanity.rs`.
 
-If you make a major change, please remember to:
+If you make a major change on bootstrap configuration, please remember to:
 
-+ Update `VERSION` in `src/bootstrap/main.rs`.
++ Update `CONFIG_CHANGE_HISTORY` in `src/bootstrap/lib.rs`.
-* Update `changelog-seen = N` in `config.example.toml`.
+* Update `change-id = {pull-request-id}` in `config.example.toml`.
-* Add an entry in `src/bootstrap/CHANGELOG.md`.
 
 A 'major change' includes
 
@@ -193,7 +192,7 @@ A 'major change' includes
 * A change in the default options.
 
 Changes that do not affect contributors to the compiler or users
-building rustc from source don't need an update to `VERSION`.
+building rustc from source don't need an update to `CONFIG_CHANGE_HISTORY`.
 
 If you have any questions, feel free to reach out on the `#t-infra/bootstrap` channel
 at [Rust Bootstrap Zulip server][rust-bootstrap-zulip]. When you encounter bugs,
@@ -201,3 +200,8 @@ please file issues on the [Rust issue tracker][rust-issue-tracker].
 
 [rust-bootstrap-zulip]: https://rust-lang.zulipchat.com/#narrow/stream/t-infra.2Fbootstrap
 [rust-issue-tracker]: https://github.com/rust-lang/rust/issues
+
+## Changelog
+
+Because we do not release bootstrap with versions, we also do not maintain CHANGELOG files. To
+review the changes made to bootstrap, simply run `git log --no-merges --oneline -- src/bootstrap`.

src/bootstrap/bin/main.rs

@@ -13,7 +13,7 @@ use std::{env, fs};
 
 #[cfg(all(any(unix, windows), not(target_os = "solaris")))]
 use bootstrap::t;
-use bootstrap::{Build, Config, Subcommand, VERSION};
+use bootstrap::{find_recent_config_change_ids, Build, Config, Subcommand, CONFIG_CHANGE_HISTORY};
 
 fn main() {
     let args = env::args().skip(1).collect::<Vec<_>>();
@@ -42,7 +42,7 @@ fn main() {
             }
             err => {
                 drop(err);
-                println!("warning: build directory locked by process {pid}, waiting for lock");
+                println!("WARNING: build directory locked by process {pid}, waiting for lock");
                 let mut lock = t!(build_lock.write());
                 t!(lock.write(&process::id().to_string().as_ref()));
                 lock
@@ -51,7 +51,7 @@ fn main() {
     }
 
     #[cfg(any(not(any(unix, windows)), target_os = "solaris"))]
-    println!("warning: file locking not supported for target, not locking build directory");
+    println!("WARNING: file locking not supported for target, not locking build directory");
 
     // check_version warnings are not printed during setup
     let changelog_suggestion =
@@ -61,7 +61,7 @@ fn main() {
     // changelog warning, not the `x.py setup` message.
     let suggest_setup = config.config.is_none() && !matches!(config.cmd, Subcommand::Setup { .. });
     if suggest_setup {
-        println!("warning: you have not made a `config.toml`");
+        println!("WARNING: you have not made a `config.toml`");
         println!(
             "help: consider running `./x.py setup` or copying `config.example.toml` by running \
             `cp config.example.toml config.toml`"
@@ -74,7 +74,7 @@ fn main() {
     Build::new(config).build();
 
     if suggest_setup {
-        println!("warning: you have not made a `config.toml`");
+        println!("WARNING: you have not made a `config.toml`");
         println!(
             "help: consider running `./x.py setup` or copying `config.example.toml` by running \
             `cp config.example.toml config.toml`"
@@ -91,7 +91,7 @@ fn main() {
         contents.contains("https://github.com/rust-lang/rust/issues/77620#issuecomment-705144570")
     }) {
         println!(
-            "warning: You have the pre-push script installed to .git/hooks/pre-commit. \
+            "WARNING: You have the pre-push script installed to .git/hooks/pre-commit. \
                   Consider moving it to .git/hooks/pre-push instead, which runs less often."
         );
     }
@@ -104,19 +104,34 @@ fn main() {
 fn check_version(config: &Config) -> Option<String> {
     let mut msg = String::new();
 
-    let suggestion = if let Some(seen) = config.changelog_seen {
+    if config.changelog_seen.is_some() {
-        if seen != VERSION {
+        msg.push_str("WARNING: The use of `changelog-seen` is deprecated. Please refer to `change-id` option in `config.example.toml` instead.\n");
-            msg.push_str("warning: there have been changes to x.py since you last updated.\n");
+    }
-            format!("update `config.toml` to use `changelog-seen = {VERSION}` instead")
+
+    let latest_config_id = CONFIG_CHANGE_HISTORY.last().unwrap();
+    let suggestion = if let Some(id) = config.change_id {
+        if &id != latest_config_id {
+            msg.push_str("WARNING: there have been changes to x.py since you last updated.\n");
+            let change_links: Vec<String> = find_recent_config_change_ids(id)
+                .iter()
+                .map(|id| format!("https://github.com/rust-lang/rust/pull/{id}"))
+                .collect();
+            if !change_links.is_empty() {
+                msg.push_str("To see more detail about these changes, visit the following PRs:\n");
+                for link in change_links {
+                    msg.push_str(&format!("  - {link}\n"));
+                }
+            }
+            msg.push_str("WARNING: there have been changes to x.py since you last updated.\n");
+            format!("update `config.toml` to use `change-id = {latest_config_id}` instead")
         } else {
             return None;
         }
     } else {
-        msg.push_str("warning: x.py has made several changes recently you may want to look at\n");
+        msg.push_str("WARNING: The `change-id` is missing in the `config.toml`. This means that you will not be able to track the major changes made to the bootstrap configurations.\n");
-        format!("add `changelog-seen = {VERSION}` at the top of `config.toml`")
+        format!("add `change-id = {latest_config_id}` at the top of `config.toml`")
     };
 
-    msg.push_str("help: consider looking at the changes in `src/bootstrap/CHANGELOG.md`\n");
     msg.push_str("note: to silence this warning, ");
     msg.push_str(&suggestion);
 

src/bootstrap/bootstrap_test.py

@@ -103,7 +103,7 @@ class GenerateAndParseConfig(unittest.TestCase):
     """Test that we can serialize and deserialize a config.toml file"""
     def test_no_args(self):
         build = serialize_and_parse([])
-        self.assertEqual(build.get_toml("changelog-seen"), '2')
+        self.assertEqual(build.get_toml("change-id"), '115898')
         self.assertEqual(build.get_toml("profile"), 'dist')
         self.assertIsNone(build.get_toml("llvm.download-ci-llvm"))
 

src/bootstrap/config.rs

@@ -112,7 +112,8 @@ impl Display for DebuginfoLevel {
 /// `config.example.toml`.
 #[derive(Default, Clone)]
 pub struct Config {
-    pub changelog_seen: Option<usize>,
+    pub changelog_seen: Option<usize>, // FIXME: Deprecated field. Remove it at 2024.
+    pub change_id: Option<usize>,
     pub ccache: Option<String>,
     /// Call Build::ninja() instead of this.
     pub ninja_in_file: bool,
@@ -546,7 +547,8 @@ impl Target {
 #[derive(Deserialize, Default)]
 #[serde(deny_unknown_fields, rename_all = "kebab-case")]
 struct TomlConfig {
-    changelog_seen: Option<usize>,
+    changelog_seen: Option<usize>, // FIXME: Deprecated field. Remove it at 2024.
+    change_id: Option<usize>,
     build: Option<Build>,
     install: Option<Install>,
     llvm: Option<Llvm>,
@@ -574,7 +576,17 @@ trait Merge {
 impl Merge for TomlConfig {
     fn merge(
         &mut self,
-        TomlConfig { build, install, llvm, rust, dist, target, profile: _, changelog_seen }: Self,
+        TomlConfig {
+            build,
+            install,
+            llvm,
+            rust,
+            dist,
+            target,
+            profile: _,
+            changelog_seen,
+            change_id,
+        }: Self,
         replace: ReplaceOpt,
     ) {
         fn do_merge<T: Merge>(x: &mut Option<T>, y: Option<T>, replace: ReplaceOpt) {
@@ -587,6 +599,7 @@ impl Merge for TomlConfig {
             }
         }
         self.changelog_seen.merge(changelog_seen, replace);
+        self.change_id.merge(change_id, replace);
         do_merge(&mut self.build, build, replace);
         do_merge(&mut self.install, install, replace);
         do_merge(&mut self.llvm, llvm, replace);
@@ -1242,6 +1255,7 @@ impl Config {
         toml.merge(override_toml, ReplaceOpt::Override);
 
         config.changelog_seen = toml.changelog_seen;
+        config.change_id = toml.change_id;
 
         let build = toml.build.unwrap_or_default();
         if let Some(file_build) = build.build {

src/bootstrap/config/tests.rs

@@ -102,7 +102,7 @@ fn override_toml() {
         &[
             "check".to_owned(),
             "--config=/does/not/exist".to_owned(),
-            "--set=changelog-seen=1".to_owned(),
+            "--set=change-id=1".to_owned(),
             "--set=rust.lto=fat".to_owned(),
             "--set=rust.deny-warnings=false".to_owned(),
             "--set=build.gdb=\"bar\"".to_owned(),
@@ -112,7 +112,7 @@ fn override_toml() {
         |&_| {
             toml::from_str(
                 r#"
-changelog-seen = 0
+change-id = 0
 [rust]
 lto = "off"
 deny-warnings = true
@@ -129,7 +129,7 @@ build-config = {}
             .unwrap()
         },
     );
-    assert_eq!(config.changelog_seen, Some(1), "setting top-level value");
+    assert_eq!(config.change_id, Some(1), "setting top-level value");
     assert_eq!(
         config.rust_lto,
         crate::config::RustcLto::Fat,
@@ -156,10 +156,10 @@ fn override_toml_duplicate() {
         &[
             "check".to_owned(),
             "--config=/does/not/exist".to_owned(),
-            "--set=changelog-seen=1".to_owned(),
+            "--set=change-id=1".to_owned(),
-            "--set=changelog-seen=2".to_owned(),
+            "--set=change-id=2".to_owned(),
         ],
-        |&_| toml::from_str("changelog-seen = 0").unwrap(),
+        |&_| toml::from_str("change-id = 0").unwrap(),
     );
 }
 

src/bootstrap/lib.rs

@@ -112,7 +112,15 @@ const LLVM_TOOLS: &[&str] = &[
 /// LLD file names for all flavors.
 const LLD_FILE_NAMES: &[&str] = &["ld.lld", "ld64.lld", "lld-link", "wasm-ld"];
 
-pub const VERSION: usize = 2;
+/// Keeps track of major changes made to the bootstrap configuration.
+///
+/// These values also represent the IDs of the PRs that caused major changes.
+/// You can visit `https://github.com/rust-lang/rust/pull/{any-id-from-the-list}` to
+/// check for more details regarding each change.
+///
+/// If you make any major changes (such as adding new values or changing default values), please
+/// ensure that the associated PR ID is added to the end of this list.
+pub const CONFIG_CHANGE_HISTORY: &[usize] = &[115898];
 
 /// Extra --check-cfg to add when building
 /// (Mode restriction, config name, config values (if any))
@@ -1844,3 +1852,16 @@ fn envify(s: &str) -> String {
         .flat_map(|c| c.to_uppercase())
         .collect()
 }
+
+pub fn find_recent_config_change_ids(current_id: usize) -> Vec<usize> {
+    let index = CONFIG_CHANGE_HISTORY
+        .iter()
+        .position(|&id| id == current_id)
+        .expect(&format!("Value `{}` was not found in `CONFIG_CHANGE_HISTORY`.", current_id));
+
+    CONFIG_CHANGE_HISTORY
+        .iter()
+        .skip(index + 1) // Skip the current_id and IDs before it
+        .cloned()
+        .collect()
+}

src/bootstrap/setup.rs

@@ -1,6 +1,6 @@
 use crate::builder::{Builder, RunConfig, ShouldRun, Step};
 use crate::Config;
-use crate::{t, VERSION};
+use crate::{t, CONFIG_CHANGE_HISTORY};
 use sha2::Digest;
 use std::env::consts::EXE_SUFFIX;
 use std::fmt::Write as _;
@@ -208,10 +208,11 @@ fn setup_config_toml(path: &PathBuf, profile: Profile, config: &Config) {
         crate::exit!(1);
     }
 
+    let latest_change_id = CONFIG_CHANGE_HISTORY.last().unwrap();
     let settings = format!(
         "# Includes one of the default files in src/bootstrap/defaults\n\
     profile = \"{profile}\"\n\
-    changelog-seen = {VERSION}\n"
+    change-id = {latest_change_id}\n"
     );
 
     t!(fs::write(path, settings));

src/doc/rustc/src/platform-support/aarch64-unknown-teeos.md

@@ -58,7 +58,7 @@ To build a rust toolchain, create a `config.toml` with the following contents:
 
 ```toml
 profile = "compiler"
-changelog-seen = 2
+change-id = 115898
 
 [build]
 sanitizers = true

src/doc/rustc/src/platform-support/nto-qnx.md

@@ -98,7 +98,7 @@ Example content:
 
 ```toml
 profile = "compiler"
-changelog-seen = 2
+change-id = 115898
 ```
 
 2. Compile the Rust toolchain for an `x86_64-unknown-linux-gnu` host (for both `aarch64` and `x86_64` targets)

src/doc/rustc/src/platform-support/openharmony.md

@@ -101,7 +101,7 @@ To build a rust toolchain, create a `config.toml` with the following contents:
 
 ```toml
 profile = "compiler"
-changelog-seen = 2
+change-id = 115898
 
 [build]
 sanitizers = true

src/tools/opt-dist/src/tests.rs

@@ -61,7 +61,7 @@ pub fn run_tests(env: &Environment) -> anyhow::Result<()> {
 
     let config_content = format!(
         r#"profile = "user"
-changelog-seen = 2
+change-id = 115898
 
 [build]
 rustc = "{rustc}"

triagebot.toml

@@ -570,6 +570,11 @@ message = "`src/tools/x` was changed. Bump version of Cargo.toml in `src/tools/x
 message = "Third-party dependency whitelist may have been modified! You must ensure that any new dependencies have compatible licenses before merging."
 cc = ["@davidtwco", "@wesleywiser"]
 
+[mentions."src/bootstrap/config.rs"]
+message = "This PR changes `src/bootstrap/config.rs`. If appropriate, please also update `CONFIG_CHANGE_HISTORY` in `src/bootstrap/lib.rs` and `change-id` in `config.example.toml`."
+[mentions."config.example.toml"]
+message = "This PR changes `config.example.toml`. If appropriate, please also update `CONFIG_CHANGE_HISTORY` in `src/bootstrap/lib.rs` and `change-id` in `config.example.toml`."
+
 [mentions."src/bootstrap/defaults/config.compiler.toml"]
 message = "This PR changes src/bootstrap/defaults/config.compiler.toml. If appropriate, please also update `config.codegen.toml` so the defaults are in sync."
 [mentions."src/bootstrap/defaults/config.codegen.toml"]