Rollup merge of #123409 - ZhuUx:master, r=oli-obk

Implement Modified Condition/Decision  Coverage

This is an implementation based on llvm backend support (>= 18) by `@evodius96` and branch coverage support by `@Zalathar.`

### Major changes:

* Add -Zcoverage-options=mcdc as switch. Now coverage options accept either `no-branch`, `branch`, or `mcdc`. `mcdc` also enables `branch` because it is essential to work.
* Add coverage mapping for MCDCBranch and MCDCDecision. Note that MCDCParameter evolves from  llvm 18 to llvm 19. The mapping in rust side mainly references to 19 and is casted to 18 types in llvm wrapper.
* Add wrapper for mcdc instrinc functions from llvm. And inject associated statements to mir.
* Add BcbMappingKind::Decision, I'm not sure is it proper but can't find a better way temporarily.
* Let coverage-dump support parsing MCDCBranch and MCDCDecision from llvm ir.
* Add simple tests to check whether mcdc works.
* Same as clang, currently rustc does not generate instrument for decision with more than 6 condtions or only 1 condition due to considerations of resource.

### Implementation Details

1. To get information about conditions and decisions, `MCDCState` in `BranchInfoBuilder` is used during hir lowering to mir. For expressions with logical op we call `Builder::visit_coverage_branch_operation` to record its sub conditions, generate condition ids for them and save their spans (to construct the span of whole decision). This process mainly references to the implementation in clang and is described in comments over `MCDCState::record_conditions`. Also true marks and false marks introduced by branch coverage are used to detect where the decision evaluation ends: the next id  of the condition == 0.
2. Once the `MCDCState::decision_stack` popped all recorded conditions, we can ensure that the decision is checked over and push it into `decision_spans`. We do not manually insert decision span to avoid complexity from then_else_break in nested if scopes.
3. When constructing CoverageSpans, add condition info to BcbMappingKind::Branch and decision info to BcbMappingKind::Decision. If the branch mapping has non-zero condition id it will be transformed to MCDCBranch mapping and insert `CondBitmapUpdate` statements to its evaluated blocks. While decision bcb mapping will insert `TestVectorBitmapUpdate` in all its end blocks.

### Usage
```bash
 echo "[build]\nprofiler=true" >> config.toml
./x build --stage 1
./x test tests/coverage/mcdc_if.rs
```
to build the compiler and run tests.

```shell
export PATH=path/to/llvm-build:$PATH
rustup toolchain link mcdc build/host/stage1
cargo +mcdc rustc --bin foo -- -Cinstrument-coverage -Zcoverage-options=mcdc
cd target/debug
LLVM_PROFILE_FILE="foo.profraw" ./foo
llvm-profdata merge -sparse foo.profraw -o foo.profdata
llvm-cov show ./foo -instr-profile=foo.profdata --show-mcdc
```
to check "foo" code.

### Problems to solve

For now decision mapping will insert statements to its all end blocks, which may be optimized by inserting a final block of the decision. To do this we must also trace the evaluated value at each end of the decision and join them separately.

This implementation is not heavily tested so there should be some unrevealed issues. We are going to check our rust products in the next.  Please let me know if you had any suggestions or comments.

compiler/rustc_mir_transform/src/coverage/mod.rs

@@ -100,9 +100,12 @@ fn instrument_function_for_coverage<'tcx>(tcx: TyCtxt<'tcx>, mir_body: &mut mir:
         &coverage_counters,
     );
 
+    inject_mcdc_statements(mir_body, &basic_coverage_blocks, &coverage_spans);
+
     mir_body.function_coverage_info = Some(Box::new(FunctionCoverageInfo {
         function_source_hash: hir_info.function_source_hash,
         num_counters: coverage_counters.num_counters(),
+        mcdc_bitmap_bytes: coverage_spans.test_vector_bitmap_bytes(),
         expressions: coverage_counters.into_expressions(),
         mappings,
     }));
@@ -136,20 +139,33 @@ fn create_mappings<'tcx>(
             .as_term()
     };
 
-    coverage_spans
-        .all_bcb_mappings()
-        .filter_map(|&BcbMapping { kind: bcb_mapping_kind, span }| {
-            let kind = match bcb_mapping_kind {
+    let mut mappings = Vec::new();
+
+    mappings.extend(coverage_spans.all_bcb_mappings().filter_map(
+        |BcbMapping { kind: bcb_mapping_kind, span }| {
+            let kind = match *bcb_mapping_kind {
                 BcbMappingKind::Code(bcb) => MappingKind::Code(term_for_bcb(bcb)),
                 BcbMappingKind::Branch { true_bcb, false_bcb } => MappingKind::Branch {
                     true_term: term_for_bcb(true_bcb),
                     false_term: term_for_bcb(false_bcb),
                 },
+                BcbMappingKind::MCDCBranch { true_bcb, false_bcb, condition_info } => {
+                    MappingKind::MCDCBranch {
+                        true_term: term_for_bcb(true_bcb),
+                        false_term: term_for_bcb(false_bcb),
+                        mcdc_params: condition_info,
+                    }
+                }
+                BcbMappingKind::MCDCDecision { bitmap_idx, conditions_num, .. } => {
+                    MappingKind::MCDCDecision(DecisionInfo { bitmap_idx, conditions_num })
+                }
             };
-            let code_region = make_code_region(source_map, file_name, span, body_span)?;
+            let code_region = make_code_region(source_map, file_name, *span, body_span)?;
             Some(Mapping { kind, code_region })
-        })
-        .collect::<Vec<_>>()
+        },
+    ));
+
+    mappings
 }
 
 /// For each BCB node or BCB edge that has an associated coverage counter,
@@ -204,6 +220,55 @@ fn inject_coverage_statements<'tcx>(
     }
 }
 
+/// For each conditions inject statements to update condition bitmap after it has been evaluated.
+/// For each decision inject statements to update test vector bitmap after it has been evaluated.
+fn inject_mcdc_statements<'tcx>(
+    mir_body: &mut mir::Body<'tcx>,
+    basic_coverage_blocks: &CoverageGraph,
+    coverage_spans: &CoverageSpans,
+) {
+    if coverage_spans.test_vector_bitmap_bytes() == 0 {
+        return;
+    }
+
+    // Inject test vector update first because `inject_statement` always insert new statement at head.
+    for (end_bcbs, bitmap_idx) in
+        coverage_spans.all_bcb_mappings().filter_map(|mapping| match &mapping.kind {
+            BcbMappingKind::MCDCDecision { end_bcbs, bitmap_idx, .. } => {
+                Some((end_bcbs, *bitmap_idx))
+            }
+            _ => None,
+        })
+    {
+        for end in end_bcbs {
+            let end_bb = basic_coverage_blocks[*end].leader_bb();
+            inject_statement(mir_body, CoverageKind::TestVectorBitmapUpdate { bitmap_idx }, end_bb);
+        }
+    }
+
+    for (true_bcb, false_bcb, condition_id) in
+        coverage_spans.all_bcb_mappings().filter_map(|mapping| match mapping.kind {
+            BcbMappingKind::MCDCBranch { true_bcb, false_bcb, condition_info } => {
+                Some((true_bcb, false_bcb, condition_info.condition_id))
+            }
+            _ => None,
+        })
+    {
+        let true_bb = basic_coverage_blocks[true_bcb].leader_bb();
+        inject_statement(
+            mir_body,
+            CoverageKind::CondBitmapUpdate { id: condition_id, value: true },
+            true_bb,
+        );
+        let false_bb = basic_coverage_blocks[false_bcb].leader_bb();
+        inject_statement(
+            mir_body,
+            CoverageKind::CondBitmapUpdate { id: condition_id, value: false },
+            false_bb,
+        );
+    }
+}
+
 /// Given two basic blocks that have a control-flow edge between them, creates
 /// and returns a new block that sits between those blocks.
 fn inject_edge_counter_basic_block(

compiler/rustc_mir_transform/src/coverage/query.rs

@@ -61,7 +61,17 @@ fn coverage_ids_info<'tcx>(
         .max()
         .unwrap_or(CounterId::ZERO);
 
-    CoverageIdsInfo { max_counter_id }
+    let mcdc_bitmap_bytes = mir_body
+        .coverage_branch_info
+        .as_deref()
+        .map(|info| {
+            info.mcdc_decision_spans
+                .iter()
+                .fold(0, |acc, decision| acc + (1_u32 << decision.conditions_num).div_ceil(8))
+        })
+        .unwrap_or_default();
+
+    CoverageIdsInfo { max_counter_id, mcdc_bitmap_bytes }
 }
 
 fn all_coverage_in_mir_body<'a, 'tcx>(

compiler/rustc_mir_transform/src/coverage/spans.rs

@@ -1,7 +1,9 @@
 use rustc_data_structures::graph::DirectedGraph;
 use rustc_index::bit_set::BitSet;
 use rustc_middle::mir;
+use rustc_middle::mir::coverage::ConditionInfo;
 use rustc_span::{BytePos, Span};
+use std::collections::BTreeSet;
 
 use crate::coverage::graph::{BasicCoverageBlock, CoverageGraph, START_BCB};
 use crate::coverage::spans::from_mir::SpanFromMir;
@@ -9,12 +11,20 @@ use crate::coverage::ExtractedHirInfo;
 
 mod from_mir;
 
-#[derive(Clone, Copy, Debug)]
+#[derive(Clone, Debug)]
 pub(super) enum BcbMappingKind {
     /// Associates an ordinary executable code span with its corresponding BCB.
     Code(BasicCoverageBlock),
     /// Associates a branch span with BCBs for its true and false arms.
     Branch { true_bcb: BasicCoverageBlock, false_bcb: BasicCoverageBlock },
+    /// Associates a mcdc branch span with condition info besides fields for normal branch.
+    MCDCBranch {
+        true_bcb: BasicCoverageBlock,
+        false_bcb: BasicCoverageBlock,
+        condition_info: ConditionInfo,
+    },
+    /// Associates a mcdc decision with its join BCB.
+    MCDCDecision { end_bcbs: BTreeSet<BasicCoverageBlock>, bitmap_idx: u32, conditions_num: u16 },
 }
 
 #[derive(Debug)]
@@ -26,6 +36,7 @@ pub(super) struct BcbMapping {
 pub(super) struct CoverageSpans {
     bcb_has_mappings: BitSet<BasicCoverageBlock>,
     mappings: Vec<BcbMapping>,
+    test_vector_bitmap_bytes: u32,
 }
 
 impl CoverageSpans {
@@ -36,6 +47,10 @@ impl CoverageSpans {
     pub(super) fn all_bcb_mappings(&self) -> impl Iterator<Item = &BcbMapping> {
         self.mappings.iter()
     }
+
+    pub(super) fn test_vector_bitmap_bytes(&self) -> u32 {
+        self.test_vector_bitmap_bytes
+    }
 }
 
 /// Extracts coverage-relevant spans from MIR, and associates them with
@@ -85,17 +100,26 @@ pub(super) fn generate_coverage_spans(
     let mut insert = |bcb| {
         bcb_has_mappings.insert(bcb);
     };
-    for &BcbMapping { kind, span: _ } in &mappings {
-        match kind {
+    let mut test_vector_bitmap_bytes = 0;
+    for BcbMapping { kind, span: _ } in &mappings {
+        match *kind {
             BcbMappingKind::Code(bcb) => insert(bcb),
-            BcbMappingKind::Branch { true_bcb, false_bcb } => {
+            BcbMappingKind::Branch { true_bcb, false_bcb }
+            | BcbMappingKind::MCDCBranch { true_bcb, false_bcb, .. } => {
                 insert(true_bcb);
                 insert(false_bcb);
             }
+            BcbMappingKind::MCDCDecision { bitmap_idx, conditions_num, .. } => {
+                // `bcb_has_mappings` is used for inject coverage counters
+                // but they are not needed for decision BCBs.
+                // While the length of test vector bitmap should be calculated here.
+                test_vector_bitmap_bytes = test_vector_bitmap_bytes
+                    .max(bitmap_idx + (1_u32 << conditions_num as u32).div_ceil(8));
+            }
         }
     }
 
-    Some(CoverageSpans { bcb_has_mappings, mappings })
+    Some(CoverageSpans { bcb_has_mappings, mappings, test_vector_bitmap_bytes })
 }
 
 #[derive(Debug)]

compiler/rustc_mir_transform/src/coverage/spans/from_mir.rs

@@ -1,7 +1,9 @@
 use rustc_data_structures::captures::Captures;
 use rustc_data_structures::fx::FxHashSet;
 use rustc_index::IndexVec;
-use rustc_middle::mir::coverage::{BlockMarkerId, BranchSpan, CoverageKind};
+use rustc_middle::mir::coverage::{
+    BlockMarkerId, BranchSpan, CoverageKind, MCDCBranchSpan, MCDCDecisionSpan,
+};
 use rustc_middle::mir::{
     self, AggregateKind, BasicBlock, FakeReadCause, Rvalue, Statement, StatementKind, Terminator,
     TerminatorKind,
@@ -227,7 +229,10 @@ fn filtered_statement_span(statement: &Statement<'_>) -> Option<Span> {
 
         // These coverage statements should not exist prior to coverage instrumentation.
         StatementKind::Coverage(
-            CoverageKind::CounterIncrement { .. } | CoverageKind::ExpressionUsed { .. },
+            CoverageKind::CounterIncrement { .. }
+            | CoverageKind::ExpressionUsed { .. }
+            | CoverageKind::CondBitmapUpdate { .. }
+            | CoverageKind::TestVectorBitmapUpdate { .. },
         ) => bug!(
             "Unexpected coverage statement found during coverage instrumentation: {statement:?}"
         ),
@@ -384,10 +389,11 @@ pub(super) fn extract_branch_mappings(
         }
     }
 
-    branch_info
-        .branch_spans
-        .iter()
-        .filter_map(|&BranchSpan { span: raw_span, true_marker, false_marker }| {
+    let bcb_from_marker =
+        |marker: BlockMarkerId| basic_coverage_blocks.bcb_from_bb(block_markers[marker]?);
+
+    let check_branch_bcb =
+        |raw_span: Span, true_marker: BlockMarkerId, false_marker: BlockMarkerId| {
             // For now, ignore any branch span that was introduced by
             // expansion. This makes things like assert macros less noisy.
             if !raw_span.ctxt().outer_expn_data().is_root() {
@@ -395,13 +401,56 @@ pub(super) fn extract_branch_mappings(
             }
             let (span, _) = unexpand_into_body_span_with_visible_macro(raw_span, body_span)?;
 
-            let bcb_from_marker =
-                |marker: BlockMarkerId| basic_coverage_blocks.bcb_from_bb(block_markers[marker]?);
-
             let true_bcb = bcb_from_marker(true_marker)?;
             let false_bcb = bcb_from_marker(false_marker)?;
+            Some((span, true_bcb, false_bcb))
+        };
 
-            Some(BcbMapping { kind: BcbMappingKind::Branch { true_bcb, false_bcb }, span })
+    let branch_filter_map = |&BranchSpan { span: raw_span, true_marker, false_marker }| {
+        check_branch_bcb(raw_span, true_marker, false_marker).map(|(span, true_bcb, false_bcb)| {
+            BcbMapping { kind: BcbMappingKind::Branch { true_bcb, false_bcb }, span }
         })
+    };
+
+    let mcdc_branch_filter_map =
+        |&MCDCBranchSpan { span: raw_span, true_marker, false_marker, condition_info }| {
+            check_branch_bcb(raw_span, true_marker, false_marker).map(
+                |(span, true_bcb, false_bcb)| BcbMapping {
+                    kind: BcbMappingKind::MCDCBranch { true_bcb, false_bcb, condition_info },
+                    span,
+                },
+            )
+        };
+
+    let mut next_bitmap_idx = 0;
+
+    let decision_filter_map = |decision: &MCDCDecisionSpan| {
+        let (span, _) = unexpand_into_body_span_with_visible_macro(decision.span, body_span)?;
+
+        let end_bcbs = decision
+            .end_markers
+            .iter()
+            .map(|&marker| bcb_from_marker(marker))
+            .collect::<Option<_>>()?;
+
+        let bitmap_idx = next_bitmap_idx;
+        next_bitmap_idx += (1_u32 << decision.conditions_num).div_ceil(8);
+
+        Some(BcbMapping {
+            kind: BcbMappingKind::MCDCDecision {
+                end_bcbs,
+                bitmap_idx,
+                conditions_num: decision.conditions_num as u16,
+            },
+            span,
+        })
+    };
+
+    branch_info
+        .branch_spans
+        .iter()
+        .filter_map(branch_filter_map)
+        .chain(branch_info.mcdc_branch_spans.iter().filter_map(mcdc_branch_filter_map))
+        .chain(branch_info.mcdc_decision_spans.iter().filter_map(decision_filter_map))
         .collect::<Vec<_>>()
 }