Rollup merge of #41625 - nikomatsakis:incr-comp-dep-tracking-cell-mir, r=eddyb

rework the queries for the MIR pipeline

This PR refashions the MIR pipeline. There are a number of changes:

* We no longer have "MIR passes" and the pass manager is completely reworked. Unless we are doing the interprocedural optimization (meaning, right now, the inline pass), we will process a single MIR from beginning to finish in a completely on-demand fashion; i.e., when you request `optimized_mir(D)`, that will trigger the MIR for `D` to actually be built and optimized, but no other functions are built or touched.
* We no longer use `&'tcx RefCell<Mir<'tcx>>` as the result of queries, since that spoils the view of queries as "pure functions". To avoid however copying the MIR, we use a `&'tcx Steal<Mir<'tcx>>` -- this is something like a ref-cell, in that you can use `borrow()` to read it, but it has no `borrow_mut()`. Instead, it has `steal()`, which will take the contents and then panic if any further read attempt occurs.
* We now support `[multi]` queries, which can optionally yield not just one result but a sequence of (K, V) pairs. This is used for the inlining pass. If inlining is enabled, then when it is invoked on **any** def-id D, it will go and read the results for **all** def-ids and transform them, and then return the results for all of them at once. This isn't ideal, and we'll probably want to rework this further, but it seems ok for now (note that MIR inlining is not enabled by default).

**Tips for the reviewer:** The commits here are meant to build individually, but the path is a *bit* meandering. In some cases, for example, I introduce a trait in one commit, and then tweak it in a later commit as I actually try to put it to use. You may want to read the README in the final commit to get a sense of where the overall design is headed.

@eddyb I did not wind up adding support for queries that produce more than one *kind* of result. Instead, I decided to just insert judicious use of the `force()` command. In other words, we had talked about e.g. having a query that produced not only the MIR but also the `const_qualif` result for the MIR in one sweep. I realized you can also have the same effect by having a kind of meta-query that forces the const-qualif pass and then reads the result. See the README for a description. (We can still do these "multi-query results" later if we want, I'm not sure though if it is necessary.)

r? @eddyb

cc @michaelwoerister @matthewhammer @arielb1, who participated in the IRC discussion.

src/librustc/dep_graph/dep_node.rs

@@ -76,6 +76,7 @@ pub enum DepNode<D: Clone + Debug> {
     BorrowCheck(D),
     RvalueCheck(D),
     Reachability,
+    MirKeys,
     LateLintCheck,
     TransCrateItem(D),
     TransInlinedItem(D),
@@ -202,6 +203,7 @@ impl<D: Clone + Debug> DepNode<D> {
             Variance => Some(Variance),
             PrivacyAccessLevels(k) => Some(PrivacyAccessLevels(k)),
             Reachability => Some(Reachability),
+            MirKeys => Some(MirKeys),
             LateLintCheck => Some(LateLintCheck),
             TransWriteMetadata => Some(TransWriteMetadata),
 

src/librustc/dep_graph/mod.rs

@@ -28,6 +28,5 @@ pub use self::graph::WorkProduct;
 pub use self::query::DepGraphQuery;
 pub use self::safe::AssertDepGraphSafe;
 pub use self::safe::DepGraphSafe;
-pub use self::visit::visit_all_bodies_in_krate;
 pub use self::visit::visit_all_item_likes_in_krate;
 pub use self::raii::DepTask;

src/librustc/dep_graph/safe.rs

@@ -50,6 +50,12 @@ impl<A, B> DepGraphSafe for (A, B)
 {
 }
 
+/// Shared ref to dep-graph-safe stuff should still be dep-graph-safe.
+impl<'a, A> DepGraphSafe for &'a A
+    where A: DepGraphSafe,
+{
+}
+
 /// No data here! :)
 impl DepGraphSafe for () {
 }

src/librustc/dep_graph/visit.rs

@@ -75,15 +75,3 @@ pub fn visit_all_item_likes_in_krate<'a, 'tcx, V, F>(tcx: TyCtxt<'a, 'tcx, 'tcx>
     krate.visit_all_item_likes(&mut tracking_visitor)
 }
 
-pub fn visit_all_bodies_in_krate<'a, 'tcx, C>(tcx: TyCtxt<'a, 'tcx, 'tcx>, callback: C)
-    where C: Fn(/* body_owner */
-                DefId,
-                /* body id */
-                hir::BodyId)
-{
-    let krate = tcx.hir.krate();
-    for &body_id in &krate.body_ids {
-        let body_owner_def_id = tcx.hir.body_owner_def_id(body_id);
-        callback(body_owner_def_id, body_id);
-    }
-}

src/librustc/hir/map/mod.rs

@@ -455,7 +455,7 @@ impl<'hir> Map<'hir> {
                 if let EntryExpr(_, expr) = entry {
                     BodyId { node_id: expr.id }
                 } else {
-                    span_bug!(self.span(id), "id `{}` has no associated body", id);
+                    span_bug!(self.span(id), "id `{}` has no associated body: {:?}", id, entry);
                 }
             }
         } else {

src/librustc/lib.rs

@@ -43,6 +43,9 @@
 #![feature(unboxed_closures)]
 #![feature(discriminant_value)]
 #![feature(sort_unstable)]
+#![feature(trace_macros)]
+
+#![recursion_limit="128"]
 
 extern crate arena;
 extern crate core;

src/librustc/mir/README.md

@@ -0,0 +1,90 @@
+# MIR definition and pass system
+
+This file contains the definition of the MIR datatypes along with the
+various types for the "MIR Pass" system, which lets you easily
+register and define new MIR transformations and analyses.
+
+Most of the code that operates on MIR can be found in the
+`librustc_mir` crate or other crates. The code found here in
+`librustc` is just the datatype definitions, alonging the functions
+which operate on MIR to be placed everywhere else.
+
+## MIR Data Types and visitor
+
+The main MIR data type is `rustc::mir::Mir`, defined in `mod.rs`.
+There is also the MIR visitor (in `visit.rs`) which allows you to walk
+the MIR and override what actions will be taken at various points (you
+can visit in either shared or mutable mode; the latter allows changing
+the MIR in place). Finally `traverse.rs` contains various traversal
+routines for visiting the MIR CFG in [different standard orders][traversal]
+(e.g. pre-order, reverse post-order, and so forth).
+
+[traversal]: https://en.wikipedia.org/wiki/Tree_traversal
+
+## MIR pass suites and their integration into the query system
+
+As a MIR *consumer*, you are expected to use one of the queries that
+returns a "final MIR". As of the time of this writing, there is only
+one: `optimized_mir(def_id)`, but more are expected to come in the
+future. For foreign def-ids, we simply read the MIR from the other
+crate's metadata. But for local query, this query will construct the
+MIR and then iteratively optimize it by putting it through various
+pipeline stages. This section describes those pipeline stages and how
+you can extend them.
+
+To produce the `optimized_mir(D)` for a given def-id `D`, the MIR
+passes through several suites of optimizations, each represented by a
+query. Each suite consists of multiple optimizations and
+transformations. These suites represent useful intermediate points
+where we want to access the MIR for type checking or other purposes:
+
+- `mir_build(D)` -- not a query, but this constructs the initial MIR
+- `mir_const(D)` -- applies some simple transformations to make MIR ready for constant evaluation;
+- `mir_validated(D)` -- applies some more transformations, making MIR ready for borrow checking;
+- `optimized_mir(D)` -- the final state, after all optimizations have been performed.
+
+### Stealing
+
+The intermediate queries `mir_const()` and `mir_validated()` yield up
+a `&'tcx Steal<Mir<'tcx>>`, allocated using
+`tcx.alloc_steal_mir()`. This indicates that the result may be
+**stolen** by the next suite of optimizations -- this is an
+optimization to avoid cloning the MIR. Attempting to use a stolen
+result will cause a panic in the compiler. Therefore, it is important
+that you not read directly from these intermediate queries except as
+part of the MIR processing pipeline.
+
+Because of this stealing mechanism, some care must also be taken to
+ensure that, before the MIR at a particular phase in the processing
+pipeline is stolen, anyone who may want to read from it has already
+done so. Concretely, this means that if you have some query `foo(D)`
+that wants to access the result of `mir_const(D)` or
+`mir_validated(D)`, you need to have the successor pass either "force"
+`foo(D)` using `ty::queries::foo::force(...)`. This will force a query
+to execute even though you don't directly require its result.
+
+As an example, consider MIR const qualification. It wants to read the
+result produced by the `mir_const()` suite. However, that result will
+be **stolen** by the `mir_validated()` suite. If nothing was done,
+then `mir_const_qualif(D)` would succeed if it came before
+`mir_validated(D)`, but fail otherwise. Therefore, `mir_validated(D)`
+will **force** `mir_const_qualif` before it actually steals, thus
+ensuring that the reads have already happened:
+
+```
+mir_const(D) --read-by--> mir_const_qualif(D)
+     |                       ^
+  stolen-by                  |
+     |                    (forces)
+     v                       |
+mir_validated(D) ------------+
+```
+
+### Implementing and registering a pass
+
+To create a new MIR pass, you simply implement the `MirPass` trait for
+some fresh singleton type `Foo`. Once you have implemented a trait for
+your type `Foo`, you then have to insert `Foo` into one of the suites;
+this is done in `librustc_driver/driver.rs` by invoking `push_pass(S,
+Foo)` with the appropriate suite substituted for `S`.
+

src/librustc/mir/mod.rs

@@ -8,6 +8,8 @@
 // option. This file may not be copied, modified, or distributed
 // except according to those terms.
 
+//! MIR datatypes and passes. See [the README](README.md) for details.
+
 use graphviz::IntoCow;
 use middle::const_val::ConstVal;
 use rustc_const_math::{ConstUsize, ConstInt, ConstMathErr};

src/librustc/mir/transform.rs

@@ -8,16 +8,17 @@
 // option. This file may not be copied, modified, or distributed
 // except according to those terms.
 
-use dep_graph::DepNode;
+//! See [the README](README.md) for details on writing your own pass.
+
 use hir;
+use hir::def_id::DefId;
 use hir::map::DefPathData;
 use mir::{Mir, Promoted};
 use ty::TyCtxt;
+use std::rc::Rc;
 use syntax::ast::NodeId;
-use util::common::time;
 
 use std::borrow::Cow;
-use std::fmt;
 
 /// Where a specific Mir comes from.
 #[derive(Debug, Copy, Clone)]
@@ -36,6 +37,11 @@ pub enum MirSource {
 }
 
 impl<'a, 'tcx> MirSource {
+    pub fn from_local_def_id(tcx: TyCtxt<'a, 'tcx, 'tcx>, def_id: DefId) -> MirSource {
+        let id = tcx.hir.as_local_node_id(def_id).expect("mir source requires local def-id");
+        Self::from_node(tcx, id)
+    }
+
     pub fn from_node(tcx: TyCtxt<'a, 'tcx, 'tcx>, id: NodeId) -> MirSource {
         use hir::*;
 
@@ -70,124 +76,111 @@ impl<'a, 'tcx> MirSource {
     }
 }
 
-/// Various information about pass.
-pub trait Pass {
-    // fn should_run(Session) to check if pass should run?
-    fn name<'a>(&self) -> Cow<'static, str> {
-        let name = unsafe { ::std::intrinsics::type_name::<Self>() };
-        if let Some(tail) = name.rfind(":") {
-            Cow::from(&name[tail+1..])
-        } else {
-            Cow::from(name)
-        }
+/// Generates a default name for the pass based on the name of the
+/// type `T`.
+pub fn default_name<T: ?Sized>() -> Cow<'static, str> {
+    let name = unsafe { ::std::intrinsics::type_name::<T>() };
+    if let Some(tail) = name.rfind(":") {
+        Cow::from(&name[tail+1..])
+    } else {
+        Cow::from(name)
     }
-    fn disambiguator<'a>(&'a self) -> Option<Box<fmt::Display+'a>> { None }
 }
 
-/// A pass which inspects the whole Mir map.
-pub trait MirMapPass<'tcx>: Pass {
-    fn run_pass<'a>(
-        &mut self,
-        tcx: TyCtxt<'a, 'tcx, 'tcx>,
-        hooks: &mut [Box<for<'s> MirPassHook<'s>>]);
+#[derive(Copy, Clone, Debug, PartialEq, Eq, Hash, PartialOrd, Ord)]
+pub struct MirSuite(pub usize);
+
+#[derive(Copy, Clone, Debug, PartialEq, Eq, Hash, PartialOrd, Ord)]
+pub struct MirPassIndex(pub usize);
+
+/// A pass hook is invoked both before and after each pass executes.
+/// This is primarily used to dump MIR for debugging.
+///
+/// You can tell whether this is before or after by inspecting the
+/// `mir` parameter -- before the pass executes, it will be `None` (in
+/// which case you can inspect the MIR from previous pass by executing
+/// `mir_cx.read_previous_mir()`); after the pass executes, it will be
+/// `Some()` with the result of the pass (in which case the output
+/// from the previous pass is most likely stolen, so you would not
+/// want to try and access it). If the pass is interprocedural, then
+/// the hook will be invoked once per output.
+pub trait PassHook {
+    fn on_mir_pass<'a, 'tcx: 'a>(&self,
+                                 tcx: TyCtxt<'a, 'tcx, 'tcx>,
+                                 suite: MirSuite,
+                                 pass_num: MirPassIndex,
+                                 pass_name: &str,
+                                 source: MirSource,
+                                 mir: &Mir<'tcx>,
+                                 is_after: bool);
 }
 
-pub trait MirPassHook<'tcx>: Pass {
-    fn on_mir_pass<'a>(
-        &mut self,
-        tcx: TyCtxt<'a, 'tcx, 'tcx>,
-        src: MirSource,
-        mir: &Mir<'tcx>,
-        pass: &Pass,
-        is_after: bool
-    );
-}
+/// The full suite of types that identifies a particular
+/// application of a pass to a def-id.
+pub type PassId = (MirSuite, MirPassIndex, DefId);
 
-/// A pass which inspects Mir of functions in isolation.
-pub trait MirPass<'tcx>: Pass {
-    fn run_pass<'a>(&mut self, tcx: TyCtxt<'a, 'tcx, 'tcx>,
-                    src: MirSource, mir: &mut Mir<'tcx>);
-}
-
-impl<'tcx, T: MirPass<'tcx>> MirMapPass<'tcx> for T {
-    fn run_pass<'a>(&mut self,
-                    tcx: TyCtxt<'a, 'tcx, 'tcx>,
-                    hooks: &mut [Box<for<'s> MirPassHook<'s>>])
-    {
-        let def_ids = tcx.maps.mir.borrow().keys();
-        for def_id in def_ids {
-            if !def_id.is_local() {
-                continue;
-            }
-
-            let _task = tcx.dep_graph.in_task(DepNode::Mir(def_id));
-            let mir = &mut tcx.maps.mir.borrow()[&def_id].borrow_mut();
-            tcx.dep_graph.write(DepNode::Mir(def_id));
-
-            let id = tcx.hir.as_local_node_id(def_id).unwrap();
-            let src = MirSource::from_node(tcx, id);
-
-            for hook in &mut *hooks {
-                hook.on_mir_pass(tcx, src, mir, self, false);
-            }
-            MirPass::run_pass(self, tcx, src, mir);
-            for hook in &mut *hooks {
-                hook.on_mir_pass(tcx, src, mir, self, true);
-            }
-
-            for (i, mir) in mir.promoted.iter_enumerated_mut() {
-                let src = MirSource::Promoted(id, i);
-                for hook in &mut *hooks {
-                    hook.on_mir_pass(tcx, src, mir, self, false);
-                }
-                MirPass::run_pass(self, tcx, src, mir);
-                for hook in &mut *hooks {
-                    hook.on_mir_pass(tcx, src, mir, self, true);
-                }
-            }
-        }
+/// A streamlined trait that you can implement to create a pass; the
+/// pass will be named after the type, and it will consist of a main
+/// loop that goes over each available MIR and applies `run_pass`.
+pub trait MirPass {
+    fn name<'a>(&'a self) -> Cow<'a, str> {
+        default_name::<Self>()
     }
+
+    fn run_pass<'a, 'tcx>(&self,
+                          tcx: TyCtxt<'a, 'tcx, 'tcx>,
+                          source: MirSource,
+                          mir: &mut Mir<'tcx>);
 }
 
 /// A manager for MIR passes.
+///
+/// FIXME(#41712) -- it is unclear whether we should have this struct.
+#[derive(Clone)]
 pub struct Passes {
-    passes: Vec<Box<for<'tcx> MirMapPass<'tcx>>>,
-    pass_hooks: Vec<Box<for<'tcx> MirPassHook<'tcx>>>,
-    plugin_passes: Vec<Box<for<'tcx> MirMapPass<'tcx>>>
+    pass_hooks: Vec<Rc<PassHook>>,
+    suites: Vec<Vec<Rc<MirPass>>>,
 }
 
+/// The number of "pass suites" that we have:
+///
+/// - ready for constant evaluation
+/// - unopt
+/// - optimized
+pub const MIR_SUITES: usize = 3;
+
+/// Run the passes we need to do constant qualification and evaluation.
+pub const MIR_CONST: MirSuite = MirSuite(0);
+
+/// Run the passes we need to consider the MIR validated and ready for borrowck etc.
+pub const MIR_VALIDATED: MirSuite = MirSuite(1);
+
+/// Run the passes we need to consider the MIR *optimized*.
+pub const MIR_OPTIMIZED: MirSuite = MirSuite(2);
+
 impl<'a, 'tcx> Passes {
     pub fn new() -> Passes {
-        let passes = Passes {
-            passes: Vec::new(),
+        Passes {
             pass_hooks: Vec::new(),
-            plugin_passes: Vec::new()
-        };
-        passes
-    }
-
-    pub fn run_passes(&mut self, tcx: TyCtxt<'a, 'tcx, 'tcx>) {
-        let Passes { ref mut passes, ref mut plugin_passes, ref mut pass_hooks } = *self;
-        for pass in plugin_passes.iter_mut().chain(passes.iter_mut()) {
-            time(tcx.sess.time_passes(), &*pass.name(),
-                 || pass.run_pass(tcx, pass_hooks));
+            suites: (0..MIR_SUITES).map(|_| Vec::new()).collect(),
         }
     }
 
     /// Pushes a built-in pass.
-    pub fn push_pass(&mut self, pass: Box<for<'b> MirMapPass<'b>>) {
-        self.passes.push(pass);
+    pub fn push_pass<T: MirPass + 'static>(&mut self, suite: MirSuite, pass: T) {
+        self.suites[suite.0].push(Rc::new(pass));
     }
 
     /// Pushes a pass hook.
-    pub fn push_hook(&mut self, hook: Box<for<'b> MirPassHook<'b>>) {
-        self.pass_hooks.push(hook);
+    pub fn push_hook<T: PassHook + 'static>(&mut self, hook: T) {
+        self.pass_hooks.push(Rc::new(hook));
     }
-}
 
-/// Copies the plugin passes.
-impl ::std::iter::Extend<Box<for<'a> MirMapPass<'a>>> for Passes {
-    fn extend<I: IntoIterator<Item=Box<for <'a> MirMapPass<'a>>>>(&mut self, it: I) {
-        self.plugin_passes.extend(it);
+    pub fn passes(&self, suite: MirSuite) -> &[Rc<MirPass>] {
+        &self.suites[suite.0]
+    }
+
+    pub fn hooks(&self) -> &[Rc<PassHook>] {
+        &self.pass_hooks
     }
 }

src/librustc/session/config.rs

@@ -1003,6 +1003,8 @@ options! {DebuggingOptions, DebuggingSetter, basic_debugging_options,
           "dump MIR state at various points in translation"),
     dump_mir_dir: Option<String> = (None, parse_opt_string, [UNTRACKED],
           "the directory the MIR is dumped into"),
+    dump_mir_exclude_pass_number: bool = (false, parse_bool, [UNTRACKED],
+          "if set, exclude the pass number when dumping MIR (used in tests)"),
     perf_stats: bool = (false, parse_bool, [UNTRACKED],
           "print some performance-related statistics"),
     hir_stats: bool = (false, parse_bool, [UNTRACKED],

src/librustc/session/mod.rs

@@ -21,7 +21,6 @@ use session::config::DebugInfoLevel;
 use ty::tls;
 use util::nodemap::{FxHashMap, FxHashSet};
 use util::common::duration_to_secs_str;
-use mir::transform as mir_pass;
 
 use syntax::ast::NodeId;
 use errors::{self, DiagnosticBuilder};
@@ -85,7 +84,6 @@ pub struct Session {
     /// redundantly verbose output (Issue #24690).
     pub one_time_diagnostics: RefCell<FxHashSet<(lint::LintId, Span, String)>>,
     pub plugin_llvm_passes: RefCell<Vec<String>>,
-    pub mir_passes: RefCell<mir_pass::Passes>,
     pub plugin_attributes: RefCell<Vec<(String, AttributeType)>>,
     pub crate_types: RefCell<Vec<config::CrateType>>,
     pub dependency_formats: RefCell<dependency_format::Dependencies>,
@@ -670,7 +668,6 @@ pub fn build_session_(sopts: config::Options,
         lints: RefCell::new(lint::LintTable::new()),
         one_time_diagnostics: RefCell::new(FxHashSet()),
         plugin_llvm_passes: RefCell::new(Vec::new()),
-        mir_passes: RefCell::new(mir_pass::Passes::new()),
         plugin_attributes: RefCell::new(Vec::new()),
         crate_types: RefCell::new(Vec::new()),
         dependency_formats: RefCell::new(FxHashMap()),

src/librustc/ty/context.rs

@@ -25,6 +25,7 @@ use middle::region::{CodeExtent, CodeExtentData};
 use middle::resolve_lifetime;
 use middle::stability;
 use mir::Mir;
+use mir::transform::Passes;
 use ty::subst::{Kind, Substs};
 use ty::ReprOptions;
 use traits;
@@ -39,6 +40,7 @@ use ty::TypeVariants::*;
 use ty::layout::{Layout, TargetDataLayout};
 use ty::inhabitedness::DefIdForest;
 use ty::maps;
+use ty::steal::Steal;
 use util::nodemap::{NodeMap, NodeSet, DefIdMap, DefIdSet};
 use util::nodemap::{FxHashMap, FxHashSet};
 use rustc_data_structures::accumulate_vec::AccumulateVec;
@@ -47,11 +49,12 @@ use arena::{TypedArena, DroplessArena};
 use rustc_data_structures::indexed_vec::IndexVec;
 use std::borrow::Borrow;
 use std::cell::{Cell, RefCell};
+use std::cmp::Ordering;
 use std::hash::{Hash, Hasher};
 use std::mem;
 use std::ops::Deref;
 use std::iter;
-use std::cmp::Ordering;
+use std::rc::Rc;
 use syntax::abi;
 use syntax::ast::{self, Name, NodeId};
 use syntax::attr;
@@ -68,7 +71,8 @@ pub struct GlobalArenas<'tcx> {
     generics: TypedArena<ty::Generics>,
     trait_def: TypedArena<ty::TraitDef>,
     adt_def: TypedArena<ty::AdtDef>,
-    mir: TypedArena<RefCell<Mir<'tcx>>>,
+    steal_mir: TypedArena<Steal<Mir<'tcx>>>,
+    mir: TypedArena<Mir<'tcx>>,
     tables: TypedArena<ty::TypeckTables<'tcx>>,
 }
 
@@ -79,6 +83,7 @@ impl<'tcx> GlobalArenas<'tcx> {
             generics: TypedArena::new(),
             trait_def: TypedArena::new(),
             adt_def: TypedArena::new(),
+            steal_mir: TypedArena::new(),
             mir: TypedArena::new(),
             tables: TypedArena::new(),
         }
@@ -443,8 +448,11 @@ pub struct GlobalCtxt<'tcx> {
     pub named_region_map: resolve_lifetime::NamedRegionMap,
 
     pub hir: hir_map::Map<'tcx>,
+
     pub maps: maps::Maps<'tcx>,
 
+    pub mir_passes: Rc<Passes>,
+
     // Records the free variables refrenced by every closure
     // expression. Do not track deps for this, just recompute it from
     // scratch every time.
@@ -619,8 +627,12 @@ impl<'a, 'gcx, 'tcx> TyCtxt<'a, 'gcx, 'tcx> {
         self.global_arenas.generics.alloc(generics)
     }
 
-    pub fn alloc_mir(self, mir: Mir<'gcx>) -> &'gcx RefCell<Mir<'gcx>> {
-        self.global_arenas.mir.alloc(RefCell::new(mir))
+    pub fn alloc_steal_mir(self, mir: Mir<'gcx>) -> &'gcx Steal<Mir<'gcx>> {
+        self.global_arenas.steal_mir.alloc(Steal::new(mir))
+    }
+
+    pub fn alloc_mir(self, mir: Mir<'gcx>) -> &'gcx Mir<'gcx> {
+        self.global_arenas.mir.alloc(mir)
     }
 
     pub fn alloc_tables(self, tables: ty::TypeckTables<'gcx>) -> &'gcx ty::TypeckTables<'gcx> {
@@ -714,6 +726,7 @@ impl<'a, 'gcx, 'tcx> TyCtxt<'a, 'gcx, 'tcx> {
     pub fn create_and_enter<F, R>(s: &'tcx Session,
                                   local_providers: ty::maps::Providers<'tcx>,
                                   extern_providers: ty::maps::Providers<'tcx>,
+                                  mir_passes: Rc<Passes>,
                                   arenas: &'tcx GlobalArenas<'tcx>,
                                   arena: &'tcx DroplessArena,
                                   resolutions: ty::Resolutions,
@@ -748,6 +761,7 @@ impl<'a, 'gcx, 'tcx> TyCtxt<'a, 'gcx, 'tcx> {
             fulfilled_predicates: RefCell::new(fulfilled_predicates),
             hir: hir,
             maps: maps::Maps::new(dep_graph, providers),
+            mir_passes,
             freevars: RefCell::new(resolutions.freevars),
             maybe_unused_trait_imports: resolutions.maybe_unused_trait_imports,
             rcache: RefCell::new(FxHashMap()),

src/librustc/ty/maps.rs

@@ -16,14 +16,18 @@ use middle::const_val;
 use middle::privacy::AccessLevels;
 use middle::region::RegionMaps;
 use mir;
+use mir::transform::{MirSuite, MirPassIndex};
 use session::CompileResult;
 use ty::{self, CrateInherentImpls, Ty, TyCtxt};
 use ty::item_path;
+use ty::steal::Steal;
 use ty::subst::Substs;
-use util::nodemap::NodeSet;
+use util::nodemap::{DefIdSet, NodeSet};
 
 use rustc_data_structures::indexed_vec::IndexVec;
 use std::cell::{RefCell, RefMut};
+use std::fmt::Debug;
+use std::hash::Hash;
 use std::mem;
 use std::collections::BTreeMap;
 use std::ops::Deref;
@@ -31,7 +35,7 @@ use std::rc::Rc;
 use syntax_pos::{Span, DUMMY_SP};
 use syntax::symbol::Symbol;
 
-trait Key {
+pub trait Key: Clone + Hash + Eq + Debug {
     fn map_crate(&self) -> CrateNum;
     fn default_span(&self, tcx: TyCtxt) -> Span;
 }
@@ -101,6 +105,24 @@ impl<'tcx> Key for (DefId, &'tcx Substs<'tcx>) {
     }
 }
 
+impl Key for (MirSuite, DefId) {
+    fn map_crate(&self) -> CrateNum {
+        self.1.map_crate()
+    }
+    fn default_span(&self, tcx: TyCtxt) -> Span {
+        self.1.default_span(tcx)
+    }
+}
+
+impl Key for (MirSuite, MirPassIndex, DefId) {
+    fn map_crate(&self) -> CrateNum {
+        self.2.map_crate()
+    }
+    fn default_span(&self, tcx: TyCtxt) -> Span {
+        self.2.default_span(tcx)
+    }
+}
+
 trait Value<'tcx>: Sized {
     fn from_cycle_error<'a>(tcx: TyCtxt<'a, 'tcx, 'tcx>) -> Self;
 }
@@ -270,8 +292,13 @@ impl<'tcx> QueryDescription for queries::reachable_set<'tcx> {
 
 impl<'tcx> QueryDescription for queries::const_eval<'tcx> {
     fn describe(tcx: TyCtxt, (def_id, _): (DefId, &'tcx Substs<'tcx>)) -> String {
-        format!("const-evaluating `{}`",
-                tcx.item_path_str(def_id))
+        format!("const-evaluating `{}`", tcx.item_path_str(def_id))
+    }
+}
+
+impl<'tcx> QueryDescription for queries::mir_keys<'tcx> {
+    fn describe(_: TyCtxt, _: CrateNum) -> String {
+        format!("getting a list of all mir_keys")
     }
 }
 
@@ -306,7 +333,7 @@ impl<'tcx> QueryDescription for queries::const_is_rvalue_promotable_to_static<'t
     }
 }
 
-impl<'tcx> QueryDescription for queries::is_item_mir_available<'tcx> {
+impl<'tcx> QueryDescription for queries::is_mir_available<'tcx> {
     fn describe(tcx: TyCtxt, def_id: DefId) -> String {
         format!("checking if item is mir available: `{}`",
             tcx.item_path_str(def_id))
@@ -316,11 +343,10 @@ impl<'tcx> QueryDescription for queries::is_item_mir_available<'tcx> {
 macro_rules! define_maps {
     (<$tcx:tt>
      $($(#[$attr:meta])*
-       [$($pub:tt)*] $name:ident: $node:ident($K:ty) -> $V:ty,)*) => {
-        pub struct Maps<$tcx> {
-            providers: IndexVec<CrateNum, Providers<$tcx>>,
-            query_stack: RefCell<Vec<(Span, Query<$tcx>)>>,
-            $($(#[$attr])* $($pub)* $name: RefCell<DepTrackingMap<queries::$name<$tcx>>>),*
+       [$($modifiers:tt)*] $name:ident: $node:ident($K:ty) -> $V:ty,)*) => {
+        define_map_struct! {
+            tcx: $tcx,
+            input: ($(([$($modifiers)*] [$($attr)*] [$name]))*)
         }
 
         impl<$tcx> Maps<$tcx> {
@@ -400,7 +426,7 @@ macro_rules! define_maps {
                     provider(tcx.global_tcx(), key)
                 })?;
 
-                Ok(f(&tcx.maps.$name.borrow_mut().entry(key).or_insert(result)))
+                Ok(f(tcx.maps.$name.borrow_mut().entry(key).or_insert(result)))
             }
 
             pub fn try_get(tcx: TyCtxt<'a, $tcx, 'lcx>, span: Span, key: $K)
@@ -461,25 +487,153 @@ macro_rules! define_maps {
             })*
         }
 
-        pub struct Providers<$tcx> {
-            $(pub $name: for<'a> fn(TyCtxt<'a, $tcx, $tcx>, $K) -> $V),*
+        define_provider_struct! {
+            tcx: $tcx,
+            input: ($(([$($modifiers)*] [$name] [$K] [$V]))*),
+            output: ()
         }
 
         impl<$tcx> Copy for Providers<$tcx> {}
         impl<$tcx> Clone for Providers<$tcx> {
             fn clone(&self) -> Self { *self }
         }
+    }
+}
+
+macro_rules! define_map_struct {
+    // Initial state
+    (tcx: $tcx:tt,
+     input: $input:tt) => {
+        define_map_struct! {
+            tcx: $tcx,
+            input: $input,
+            output: ()
+        }
+    };
+
+    // Final output
+    (tcx: $tcx:tt,
+     input: (),
+     output: ($($output:tt)*)) => {
+        pub struct Maps<$tcx> {
+            providers: IndexVec<CrateNum, Providers<$tcx>>,
+            query_stack: RefCell<Vec<(Span, Query<$tcx>)>>,
+            $($output)*
+        }
+    };
+
+    // Field recognized and ready to shift into the output
+    (tcx: $tcx:tt,
+     ready: ([$($pub:tt)*] [$($attr:tt)*] [$name:ident]),
+     input: $input:tt,
+     output: ($($output:tt)*)) => {
+        define_map_struct! {
+            tcx: $tcx,
+            input: $input,
+            output: ($($output)*
+                     $(#[$attr])* $($pub)* $name: RefCell<DepTrackingMap<queries::$name<$tcx>>>,)
+        }
+    };
+
+    // Detect things with the `pub` modifier
+    (tcx: $tcx:tt,
+     input: (([pub $($other_modifiers:tt)*] $attrs:tt $name:tt) $($input:tt)*),
+     output: $output:tt) => {
+        define_map_struct! {
+            tcx: $tcx,
+            ready: ([pub] $attrs $name),
+            input: ($($input)*),
+            output: $output
+        }
+    };
+
+    // No modifiers left? This is a private item.
+    (tcx: $tcx:tt,
+     input: (([] $attrs:tt $name:tt) $($input:tt)*),
+     output: $output:tt) => {
+        define_map_struct! {
+            tcx: $tcx,
+            ready: ([pub] $attrs $name),
+            input: ($($input)*),
+            output: $output
+        }
+    };
+
+    // Skip other modifiers
+    (tcx: $tcx:tt,
+     input: (([$other_modifier:tt $($modifiers:tt)*] $($fields:tt)*) $($input:tt)*),
+     output: $output:tt) => {
+        define_map_struct! {
+            tcx: $tcx,
+            input: (([$($modifiers)*] $($fields)*) $($input)*),
+            output: $output
+        }
+    };
+}
+
+macro_rules! define_provider_struct {
+    // Initial state:
+    (tcx: $tcx:tt, input: $input:tt) => {
+        define_provider_struct! {
+            tcx: $tcx,
+            input: $input,
+            output: ()
+        }
+    };
+
+    // Final state:
+    (tcx: $tcx:tt,
+     input: (),
+     output: ($(([$name:ident] [$K:ty] [$R:ty]))*)) => {
+        pub struct Providers<$tcx> {
+            $(pub $name: for<'a> fn(TyCtxt<'a, $tcx, $tcx>, $K) -> $R,)*
+        }
 
         impl<$tcx> Default for Providers<$tcx> {
             fn default() -> Self {
-                $(fn $name<'a, $tcx>(_: TyCtxt<'a, $tcx, $tcx>, key: $K) -> $V {
+                $(fn $name<'a, $tcx>(_: TyCtxt<'a, $tcx, $tcx>, key: $K) -> $R {
                     bug!("tcx.maps.{}({:?}) unsupported by its crate",
                          stringify!($name), key);
                 })*
                 Providers { $($name),* }
             }
         }
-    }
+    };
+
+    // Something ready to shift:
+    (tcx: $tcx:tt,
+     ready: ($name:tt $K:tt $V:tt),
+     input: $input:tt,
+     output: ($($output:tt)*)) => {
+        define_provider_struct! {
+            tcx: $tcx,
+            input: $input,
+            output: ($($output)* ($name $K $V))
+        }
+    };
+
+    // Regular queries produce a `V` only.
+    (tcx: $tcx:tt,
+     input: (([] $name:tt $K:tt $V:tt) $($input:tt)*),
+     output: $output:tt) => {
+        define_provider_struct! {
+            tcx: $tcx,
+            ready: ($name $K $V),
+            input: ($($input)*),
+            output: $output
+        }
+    };
+
+    // Skip modifiers.
+    (tcx: $tcx:tt,
+     input: (([$other_modifier:tt $($modifiers:tt)*] $($fields:tt)*) $($input:tt)*),
+     output: $output:tt) => {
+        define_provider_struct! {
+            tcx: $tcx,
+            input: (([$($modifiers)*] $($fields)*) $($input)*),
+            output: $output
+        }
+    };
 }
 
 // Each of these maps also corresponds to a method on a
@@ -537,20 +691,28 @@ define_maps! { <'tcx>
     /// Methods in these implementations don't need to be exported.
     [] inherent_impls: InherentImpls(DefId) -> Rc<Vec<DefId>>,
 
-    /// Maps from the def-id of a function/method or const/static
-    /// to its MIR. Mutation is done at an item granularity to
-    /// allow MIR optimization passes to function and still
-    /// access cross-crate MIR (e.g. inlining or const eval).
-    ///
-    /// Note that cross-crate MIR appears to be always borrowed
-    /// (in the `RefCell` sense) to prevent accidental mutation.
-    [pub] mir: Mir(DefId) -> &'tcx RefCell<mir::Mir<'tcx>>,
+    /// Set of all the def-ids in this crate that have MIR associated with
+    /// them. This includes all the body owners, but also things like struct
+    /// constructors.
+    [] mir_keys: mir_keys(CrateNum) -> Rc<DefIdSet>,
 
     /// Maps DefId's that have an associated Mir to the result
     /// of the MIR qualify_consts pass. The actual meaning of
     /// the value isn't known except to the pass itself.
     [] mir_const_qualif: Mir(DefId) -> u8,
 
+    /// Fetch the MIR for a given def-id up till the point where it is
+    /// ready for const evaluation.
+    ///
+    /// See the README for the `mir` module for details.
+    [] mir_const: Mir(DefId) -> &'tcx Steal<mir::Mir<'tcx>>,
+
+    [] mir_validated: Mir(DefId) -> &'tcx Steal<mir::Mir<'tcx>>,
+
+    /// MIR after our optimization passes have run. This is MIR that is ready
+    /// for trans. This is also the only query that can fetch non-local MIR, at present.
+    [] optimized_mir: Mir(DefId) -> &'tcx mir::Mir<'tcx>,
+
     /// Records the type of each closure. The def ID is the ID of the
     /// expression defining the closure.
     [] closure_kind: ItemSignature(DefId) -> ty::ClosureKind,
@@ -598,7 +760,7 @@ define_maps! { <'tcx>
     /// fn item.
     [] region_maps: RegionMaps(DefId) -> Rc<RegionMaps<'tcx>>,
 
-    [] mir_shims: mir_shim_dep_node(ty::InstanceDef<'tcx>) -> &'tcx RefCell<mir::Mir<'tcx>>,
+    [] mir_shims: mir_shim_dep_node(ty::InstanceDef<'tcx>) -> &'tcx mir::Mir<'tcx>,
 
     [] def_symbol_name: SymbolName(DefId) -> ty::SymbolName,
     [] symbol_name: symbol_name_dep_node(ty::Instance<'tcx>) -> ty::SymbolName,
@@ -608,7 +770,7 @@ define_maps! { <'tcx>
 
     [] item_body_nested_bodies: metadata_dep_node(DefId) -> Rc<BTreeMap<hir::BodyId, hir::Body>>,
     [] const_is_rvalue_promotable_to_static: metadata_dep_node(DefId) -> bool,
-    [] is_item_mir_available: metadata_dep_node(DefId) -> bool,
+    [] is_mir_available: metadata_dep_node(DefId) -> bool,
 }
 
 fn coherent_trait_dep_node((_, def_id): (CrateNum, DefId)) -> DepNode<DefId> {
@@ -644,3 +806,7 @@ fn typeck_item_bodies_dep_node(_: CrateNum) -> DepNode<DefId> {
 fn const_eval_dep_node((def_id, _): (DefId, &Substs)) -> DepNode<DefId> {
     DepNode::ConstEval(def_id)
 }
+
+fn mir_keys(_: CrateNum) -> DepNode<DefId> {
+    DepNode::MirKeys
+}

src/librustc/ty/mod.rs

@@ -35,7 +35,7 @@ use util::common::ErrorReported;
 use util::nodemap::{NodeSet, DefIdMap, FxHashMap, FxHashSet};
 
 use serialize::{self, Encodable, Encoder};
-use std::cell::{Cell, RefCell, Ref};
+use std::cell::{Cell, RefCell};
 use std::collections::BTreeMap;
 use std::cmp;
 use std::fmt;
@@ -96,6 +96,7 @@ pub mod _match;
 pub mod maps;
 pub mod outlives;
 pub mod relate;
+pub mod steal;
 pub mod subst;
 pub mod trait_def;
 pub mod walk;
@@ -2049,6 +2050,16 @@ impl<'a, 'gcx, 'tcx> TyCtxt<'a, 'gcx, 'tcx> {
         self.typeck_tables_of(self.hir.body_owner_def_id(body))
     }
 
+    /// Returns an iterator of the def-ids for all body-owners in this
+    /// crate. If you would prefer to iterate over the bodies
+    /// themselves, you can do `self.hir.krate().body_ids.iter()`.
+    pub fn body_owners(self) -> impl Iterator<Item = DefId> + 'a {
+        self.hir.krate()
+                .body_ids
+                .iter()
+                .map(move |&body_id| self.hir.body_owner_def_id(body_id))
+    }
+
     pub fn expr_span(self, id: NodeId) -> Span {
         match self.hir.find(id) {
             Some(hir_map::NodeExpr(e)) => {
@@ -2313,33 +2324,32 @@ impl<'a, 'gcx, 'tcx> TyCtxt<'a, 'gcx, 'tcx> {
         }
     }
 
-    /// Given the did of an item, returns its MIR, borrowed immutably.
-    pub fn item_mir(self, did: DefId) -> Ref<'gcx, Mir<'gcx>> {
-        self.mir(did).borrow()
-    }
-
     /// Return the possibly-auto-generated MIR of a (DefId, Subst) pair.
     pub fn instance_mir(self, instance: ty::InstanceDef<'gcx>)
-                        -> Ref<'gcx, Mir<'gcx>>
+                        -> &'gcx Mir<'gcx>
     {
         match instance {
-            ty::InstanceDef::Item(did) if true => self.item_mir(did),
-            _ => self.mir_shims(instance).borrow(),
+            ty::InstanceDef::Item(did) => {
+                self.optimized_mir(did)
+            }
+            ty::InstanceDef::Intrinsic(..) |
+            ty::InstanceDef::FnPtrShim(..) |
+            ty::InstanceDef::Virtual(..) |
+            ty::InstanceDef::ClosureOnceShim { .. } |
+            ty::InstanceDef::DropGlue(..) => {
+                self.mir_shims(instance)
+            }
         }
     }
 
     /// Given the DefId of an item, returns its MIR, borrowed immutably.
     /// Returns None if there is no MIR for the DefId
-    pub fn maybe_item_mir(self, did: DefId) -> Option<Ref<'gcx, Mir<'gcx>>> {
-        if did.is_local() && !self.maps.mir.borrow().contains_key(&did) {
-            return None;
+    pub fn maybe_optimized_mir(self, did: DefId) -> Option<&'gcx Mir<'gcx>> {
+        if self.is_mir_available(did) {
+            Some(self.optimized_mir(did))
+        } else {
+            None
         }
-
-        if !did.is_local() && !self.is_item_mir_available(did) {
-            return None;
-        }
-
-        Some(self.item_mir(did))
     }
 
     /// Get the attributes of a definition.
@@ -2541,17 +2551,6 @@ impl<'a, 'gcx, 'tcx> TyCtxt<'a, 'gcx, 'tcx> {
         dep_graph::visit_all_item_likes_in_krate(self.global_tcx(), dep_node_fn, visitor);
     }
 
-    /// Invokes `callback` for each body in the krate. This will
-    /// create a read edge from `DepNode::Krate` to the current task;
-    /// it is meant to be run in the context of some global task like
-    /// `BorrowckCrate`. The callback would then create a task like
-    /// `BorrowckBody(DefId)` to process each individual item.
-    pub fn visit_all_bodies_in_krate<C>(self, callback: C)
-        where C: Fn(/* body_owner */ DefId, /* body id */ hir::BodyId),
-    {
-        dep_graph::visit_all_bodies_in_krate(self.global_tcx(), callback)
-    }
-
     /// Looks up the span of `impl_did` if the impl is local; otherwise returns `Err`
     /// with the name of the crate containing the impl.
     pub fn span_of_impl(self, impl_did: DefId) -> Result<Span, Symbol> {

src/librustc/ty/steal.rs

@@ -0,0 +1,57 @@
+// Copyright 2012-2015 The Rust Project Developers. See the COPYRIGHT
+// file at the top-level directory of this distribution and at
+// http://rust-lang.org/COPYRIGHT.
+//
+// Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
+// http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
+// <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your
+// option. This file may not be copied, modified, or distributed
+// except according to those terms.
+
+use std::cell::{Ref, RefCell};
+use std::mem;
+
+/// The `Steal` struct is intended to used as the value for a query.
+/// Specifically, we sometimes have queries (*cough* MIR *cough*)
+/// where we create a large, complex value that we want to iteratively
+/// update (e.g., optimize). We could clone the value for each
+/// optimization, but that'd be expensive. And yet we don't just want
+/// to mutate it in place, because that would spoil the idea that
+/// queries are these pure functions that produce an immutable value
+/// (since if you did the query twice, you could observe the
+/// mutations). So instead we have the query produce a `&'tcx
+/// Steal<Mir<'tcx>>` (to be very specific). Now we can read from this
+/// as much as we want (using `borrow()`), but you can also
+/// `steal()`. Once you steal, any further attempt to read will panic.
+/// Therefore we know that -- assuming no ICE -- nobody is observing
+/// the fact that the MIR was updated.
+///
+/// Obviously, whenever you have a query that yields a `Steal` value,
+/// you must treat it with caution, and make sure that you know that
+/// -- once the value is stolen -- it will never be read from again.
+///
+/// FIXME(#41710) -- what is the best way to model linear queries?
+pub struct Steal<T> {
+    value: RefCell<Option<T>>
+}
+
+impl<T> Steal<T> {
+    pub fn new(value: T) -> Self {
+        Steal {
+            value: RefCell::new(Some(value))
+        }
+    }
+
+    pub fn borrow(&self) -> Ref<T> {
+        Ref::map(self.value.borrow(), |opt| match *opt {
+            None => bug!("attempted to read from stolen value"),
+            Some(ref v) => v
+        })
+    }
+
+    pub fn steal(&self) -> T {
+        let value_ref = &mut *self.value.borrow_mut();
+        let value = mem::replace(value_ref, None);
+        value.expect("attempt to read from stolen value")
+    }
+}

src/librustc_borrowck/borrowck/mir/elaborate_drops.rs

@@ -16,7 +16,7 @@ use super::{drop_flag_effects_for_location, on_lookup_result_bits};
 use super::MoveDataParamEnv;
 use rustc::ty::{self, TyCtxt};
 use rustc::mir::*;
-use rustc::mir::transform::{Pass, MirPass, MirSource};
+use rustc::mir::transform::{MirPass, MirSource};
 use rustc::middle::const_val::ConstVal;
 use rustc::util::nodemap::FxHashMap;
 use rustc_data_structures::indexed_set::IdxSetBuf;
@@ -32,9 +32,11 @@ use std::u32;
 
 pub struct ElaborateDrops;
 
-impl<'tcx> MirPass<'tcx> for ElaborateDrops {
-    fn run_pass<'a>(&mut self, tcx: TyCtxt<'a, 'tcx, 'tcx>,
-                    src: MirSource, mir: &mut Mir<'tcx>)
+impl MirPass for ElaborateDrops {
+    fn run_pass<'a, 'tcx>(&self,
+                          tcx: TyCtxt<'a, 'tcx, 'tcx>,
+                          src: MirSource,
+                          mir: &mut Mir<'tcx>)
     {
         debug!("elaborate_drops({:?} @ {:?})", src, mir.span);
         match src {
@@ -74,8 +76,6 @@ impl<'tcx> MirPass<'tcx> for ElaborateDrops {
     }
 }
 
-impl Pass for ElaborateDrops {}
-
 /// Return the set of basic blocks whose unwind edges are known
 /// to not be reachable, because they are `drop` terminators
 /// that can't drop anything.

src/librustc_borrowck/borrowck/mir/mod.rs

@@ -61,7 +61,10 @@ pub fn borrowck_mir(bcx: &mut BorrowckCtxt,
     let def_id = tcx.hir.local_def_id(id);
     debug!("borrowck_mir({}) UNIMPLEMENTED", tcx.item_path_str(def_id));
 
-    let mir = &tcx.item_mir(def_id);
+    // It is safe for us to borrow `mir_validated()`: `optimized_mir`
+    // steals it, but it forces the `borrowck` query.
+    let mir = &tcx.mir_validated(def_id).borrow();
+
     let param_env = ty::ParameterEnvironment::for_item(tcx, id);
     let move_data = MoveData::gather_moves(mir, tcx, &param_env);
     let mdpe = MoveDataParamEnv { move_data: move_data, param_env: param_env };

src/librustc_borrowck/borrowck/mod.rs

@@ -63,9 +63,9 @@ pub struct LoanDataFlowOperator;
 pub type LoanDataFlow<'a, 'tcx> = DataFlowContext<'a, 'tcx, LoanDataFlowOperator>;
 
 pub fn check_crate<'a, 'tcx>(tcx: TyCtxt<'a, 'tcx, 'tcx>) {
-    tcx.visit_all_bodies_in_krate(|body_owner_def_id, _body_id| {
+    for body_owner_def_id in tcx.body_owners() {
         tcx.borrowck(body_owner_def_id);
-    });
+    }
 }
 
 pub fn provide(providers: &mut Providers) {
@@ -86,6 +86,19 @@ fn borrowck<'a, 'tcx>(tcx: TyCtxt<'a, 'tcx, 'tcx>, owner_def_id: DefId) {
     debug!("borrowck(body_owner_def_id={:?})", owner_def_id);
 
     let owner_id = tcx.hir.as_local_node_id(owner_def_id).unwrap();
+
+    match tcx.hir.get(owner_id) {
+        hir_map::NodeStructCtor(_) |
+        hir_map::NodeVariant(_) => {
+            // We get invoked with anything that has MIR, but some of
+            // those things (notably the synthesized constructors from
+            // tuple structs/variants) do not have an associated body
+            // and do not need borrowchecking.
+            return;
+        }
+        _ => { }
+    }
+
     let body_id = tcx.hir.body_owned_by(owner_id);
     let attributes = tcx.get_attrs(owner_def_id);
     let tables = tcx.typeck_tables_of(owner_def_id);
@@ -96,6 +109,16 @@ fn borrowck<'a, 'tcx>(tcx: TyCtxt<'a, 'tcx, 'tcx>, owner_def_id: DefId) {
 
     if bccx.tcx.has_attr(owner_def_id, "rustc_mir_borrowck") {
         mir::borrowck_mir(bccx, owner_id, &attributes);
+    } else {
+        // Eventually, borrowck will always read the MIR, but at the
+        // moment we do not. So, for now, we always force MIR to be
+        // constructed for a given fn, since this may result in errors
+        // being reported and we want that to happen.
+        //
+        // Note that `mir_validated` is a "stealable" result; the
+        // thief, `optimized_mir()`, forces borrowck, so we know that
+        // is not yet stolen.
+        tcx.mir_validated(owner_def_id).borrow();
     }
 
     let cfg = cfg::CFG::new(bccx.tcx, &body);

src/librustc_driver/driver.rs

@@ -20,6 +20,7 @@ use rustc::session::search_paths::PathKind;
 use rustc::lint;
 use rustc::middle::{self, dependency_format, stability, reachable};
 use rustc::middle::privacy::AccessLevels;
+use rustc::mir::transform::{MIR_CONST, MIR_VALIDATED, MIR_OPTIMIZED, Passes};
 use rustc::ty::{self, TyCtxt, Resolutions, GlobalArenas};
 use rustc::util::common::time;
 use rustc::util::nodemap::NodeSet;
@@ -35,8 +36,7 @@ use rustc_typeck as typeck;
 use rustc_privacy;
 use rustc_plugin::registry::Registry;
 use rustc_plugin as plugin;
-use rustc_passes::{ast_validation, no_asm, loops, consts,
-                   static_recursion, hir_stats, mir_stats};
+use rustc_passes::{ast_validation, no_asm, loops, consts, static_recursion, hir_stats};
 use rustc_const_eval::{self, check_match};
 use super::Compilation;
 
@@ -903,9 +903,44 @@ pub fn phase_3_run_analysis_passes<'tcx, F, R>(sess: &'tcx Session,
     // FIXME(eddyb) get rid of this once we replace const_eval with miri.
     rustc_const_eval::provide(&mut extern_providers);
 
+    // Setup the MIR passes that we want to run.
+    let mut passes = Passes::new();
+    passes.push_hook(mir::transform::dump_mir::DumpMir);
+
+    // What we need to do constant evaluation.
+    passes.push_pass(MIR_CONST, mir::transform::simplify::SimplifyCfg::new("initial"));
+    passes.push_pass(MIR_CONST, mir::transform::type_check::TypeckMir);
+
+    // What we need to run borrowck etc.
+    passes.push_pass(MIR_VALIDATED, mir::transform::qualify_consts::QualifyAndPromoteConstants);
+    passes.push_pass(MIR_VALIDATED,
+                     mir::transform::simplify_branches::SimplifyBranches::new("initial"));
+    passes.push_pass(MIR_VALIDATED, mir::transform::simplify::SimplifyCfg::new("qualify-consts"));
+
+    // Optimizations begin.
+    passes.push_pass(MIR_OPTIMIZED, mir::transform::no_landing_pads::NoLandingPads);
+    passes.push_pass(MIR_OPTIMIZED, mir::transform::simplify::SimplifyCfg::new("no-landing-pads"));
+
+    // From here on out, regions are gone.
+    passes.push_pass(MIR_OPTIMIZED, mir::transform::erase_regions::EraseRegions);
+    passes.push_pass(MIR_OPTIMIZED, mir::transform::add_call_guards::AddCallGuards);
+    passes.push_pass(MIR_OPTIMIZED, borrowck::ElaborateDrops);
+    passes.push_pass(MIR_OPTIMIZED, mir::transform::no_landing_pads::NoLandingPads);
+    passes.push_pass(MIR_OPTIMIZED, mir::transform::simplify::SimplifyCfg::new("elaborate-drops"));
+
+    // No lifetime analysis based on borrowing can be done from here on out.
+    passes.push_pass(MIR_OPTIMIZED, mir::transform::inline::Inline);
+    passes.push_pass(MIR_OPTIMIZED, mir::transform::instcombine::InstCombine);
+    passes.push_pass(MIR_OPTIMIZED, mir::transform::deaggregator::Deaggregator);
+    passes.push_pass(MIR_OPTIMIZED, mir::transform::copy_prop::CopyPropagation);
+    passes.push_pass(MIR_OPTIMIZED, mir::transform::simplify::SimplifyLocals);
+    passes.push_pass(MIR_OPTIMIZED, mir::transform::add_call_guards::AddCallGuards);
+    passes.push_pass(MIR_OPTIMIZED, mir::transform::dump_mir::Marker("PreTrans"));
+
     TyCtxt::create_and_enter(sess,
                              local_providers,
                              extern_providers,
+                             Rc::new(passes),
                              arenas,
                              arena,
                              resolutions,
@@ -962,30 +997,6 @@ pub fn phase_3_run_analysis_passes<'tcx, F, R>(sess: &'tcx Session,
              "liveness checking",
              || middle::liveness::check_crate(tcx));
 
-        time(time_passes,
-             "MIR dump",
-             || mir::mir_map::build_mir_for_crate(tcx));
-
-        if sess.opts.debugging_opts.mir_stats {
-            mir_stats::print_mir_stats(tcx, "PRE CLEANUP MIR STATS");
-        }
-
-        time(time_passes, "MIR cleanup and validation", || {
-            let mut passes = sess.mir_passes.borrow_mut();
-            // Push all the built-in validation passes.
-            // NB: if you’re adding an *optimisation* it ought to go to another set of passes
-            // in stage 4 below.
-            passes.push_hook(box mir::transform::dump_mir::DumpMir);
-            passes.push_pass(box mir::transform::simplify::SimplifyCfg::new("initial"));
-            passes.push_pass(box mir::transform::type_check::TypeckMir);
-            passes.push_pass(box mir::transform::qualify_consts::QualifyAndPromoteConstants);
-            passes.push_pass(
-                box mir::transform::simplify_branches::SimplifyBranches::new("initial"));
-            passes.push_pass(box mir::transform::simplify::SimplifyCfg::new("qualify-consts"));
-            // And run everything.
-            passes.run_passes(tcx);
-        });
-
         time(time_passes,
              "borrow checking",
              || borrowck::check_crate(tcx));
@@ -1034,43 +1045,6 @@ pub fn phase_4_translate_to_llvm<'a, 'tcx>(tcx: TyCtxt<'a, 'tcx, 'tcx>,
          "resolving dependency formats",
          || dependency_format::calculate(&tcx.sess));
 
-    if tcx.sess.opts.debugging_opts.mir_stats {
-        mir_stats::print_mir_stats(tcx, "PRE OPTIMISATION MIR STATS");
-    }
-
-    // Run the passes that transform the MIR into a more suitable form for translation to LLVM
-    // code.
-    time(time_passes, "MIR optimisations", || {
-        let mut passes = ::rustc::mir::transform::Passes::new();
-        passes.push_hook(box mir::transform::dump_mir::DumpMir);
-        passes.push_pass(box mir::transform::no_landing_pads::NoLandingPads);
-        passes.push_pass(box mir::transform::simplify::SimplifyCfg::new("no-landing-pads"));
-
-        // From here on out, regions are gone.
-        passes.push_pass(box mir::transform::erase_regions::EraseRegions);
-
-        passes.push_pass(box mir::transform::add_call_guards::AddCallGuards);
-        passes.push_pass(box borrowck::ElaborateDrops);
-        passes.push_pass(box mir::transform::no_landing_pads::NoLandingPads);
-        passes.push_pass(box mir::transform::simplify::SimplifyCfg::new("elaborate-drops"));
-
-        // No lifetime analysis based on borrowing can be done from here on out.
-        passes.push_pass(box mir::transform::inline::Inline);
-        passes.push_pass(box mir::transform::instcombine::InstCombine::new());
-        passes.push_pass(box mir::transform::deaggregator::Deaggregator);
-        passes.push_pass(box mir::transform::copy_prop::CopyPropagation);
-
-        passes.push_pass(box mir::transform::simplify::SimplifyLocals);
-        passes.push_pass(box mir::transform::add_call_guards::AddCallGuards);
-        passes.push_pass(box mir::transform::dump_mir::Marker("PreTrans"));
-
-        passes.run_passes(tcx);
-    });
-
-    if tcx.sess.opts.debugging_opts.mir_stats {
-        mir_stats::print_mir_stats(tcx, "POST OPTIMISATION MIR STATS");
-    }
-
     let translation =
         time(time_passes,
              "translation",

src/librustc_driver/pretty.rs

@@ -41,7 +41,6 @@ use graphviz as dot;
 use std::cell::Cell;
 use std::fs::File;
 use std::io::{self, Write};
-use std::iter;
 use std::option;
 use std::path::Path;
 use std::str::FromStr;
@@ -999,22 +998,14 @@ fn print_with_analysis<'tcx, 'a: 'tcx>(sess: &'a Session,
                 if let Some(nodeid) = nodeid {
                     let def_id = tcx.hir.local_def_id(nodeid);
                     match ppm {
-                        PpmMir => write_mir_pretty(tcx, iter::once(def_id), &mut out),
-                        PpmMirCFG => write_mir_graphviz(tcx, iter::once(def_id), &mut out),
+                        PpmMir => write_mir_pretty(tcx, Some(def_id), &mut out),
+                        PpmMirCFG => write_mir_graphviz(tcx, Some(def_id), &mut out),
                         _ => unreachable!(),
                     }?;
                 } else {
                     match ppm {
-                        PpmMir => {
-                            write_mir_pretty(tcx,
-                                             tcx.maps.mir.borrow().keys().into_iter(),
-                                             &mut out)
-                        }
-                        PpmMirCFG => {
-                            write_mir_graphviz(tcx,
-                                               tcx.maps.mir.borrow().keys().into_iter(),
-                                               &mut out)
-                        }
+                        PpmMir => write_mir_pretty(tcx, None, &mut out),
+                        PpmMirCFG => write_mir_graphviz(tcx, None, &mut out),
                         _ => unreachable!(),
                     }?;
                 }

src/librustc_driver/test.rs

@@ -27,6 +27,7 @@ use rustc::infer::{self, InferOk, InferResult};
 use rustc::infer::type_variable::TypeVariableOrigin;
 use rustc_metadata::cstore::CStore;
 use rustc::hir::map as hir_map;
+use rustc::mir::transform::Passes;
 use rustc::session::{self, config};
 use std::rc::Rc;
 use syntax::ast;
@@ -141,6 +142,7 @@ fn test_env<F>(source_string: &str,
     TyCtxt::create_and_enter(&sess,
                              ty::maps::Providers::default(),
                              ty::maps::Providers::default(),
+                             Rc::new(Passes::new()),
                              &arenas,
                              &arena,
                              resolutions,

src/librustc_metadata/cstore_impl.rs

@@ -30,7 +30,6 @@ use rustc::util::nodemap::{NodeSet, DefIdMap};
 use rustc_back::PanicStrategy;
 
 use std::any::Any;
-use std::mem;
 use std::rc::Rc;
 
 use syntax::ast;
@@ -95,16 +94,13 @@ provide! { <'tcx> tcx, def_id, cdata
             bug!("coerce_unsized_info: `{:?}` is missing its info", def_id);
         })
     }
-    mir => {
-        let mir = cdata.maybe_get_item_mir(tcx, def_id.index).unwrap_or_else(|| {
-            bug!("get_item_mir: missing MIR for `{:?}`", def_id)
+    optimized_mir => {
+        let mir = cdata.maybe_get_optimized_mir(tcx, def_id.index).unwrap_or_else(|| {
+            bug!("get_optimized_mir: missing MIR for `{:?}`", def_id)
         });
 
         let mir = tcx.alloc_mir(mir);
 
-        // Perma-borrow MIR from extern crates to prevent mutation.
-        mem::forget(mir.borrow());
-
         mir
     }
     mir_const_qualif => { cdata.mir_const_qualif(def_id.index) }
@@ -126,7 +122,7 @@ provide! { <'tcx> tcx, def_id, cdata
         cdata.entry(def_id.index).ast.expect("const item missing `ast`")
             .decode(cdata).rvalue_promotable_to_static
     }
-    is_item_mir_available => {
+    is_mir_available => {
         !cdata.is_proc_macro(def_id.index) &&
         cdata.maybe_entry(def_id.index).and_then(|item| item.decode(cdata).mir).is_some()
     }

src/librustc_metadata/decoder.rs

@@ -779,10 +779,10 @@ impl<'a, 'tcx> CrateMetadata {
         tcx.alloc_tables(ast.tables.decode((self, tcx)))
     }
 
-    pub fn maybe_get_item_mir(&self,
-                              tcx: TyCtxt<'a, 'tcx, 'tcx>,
-                              id: DefIndex)
-                              -> Option<Mir<'tcx>> {
+    pub fn maybe_get_optimized_mir(&self,
+                                   tcx: TyCtxt<'a, 'tcx, 'tcx>,
+                                   id: DefIndex)
+                                   -> Option<Mir<'tcx>> {
         match self.is_proc_macro(id) {
             true => None,
             false => self.entry(id).mir.map(|mir| mir.decode((self, tcx))),

src/librustc_metadata/encoder.rs

@@ -295,7 +295,7 @@ impl<'a, 'b: 'a, 'tcx: 'b> EntryBuilder<'a, 'b, 'tcx> {
             predicates: Some(self.encode_predicates(def_id)),
 
             ast: None,
-            mir: self.encode_mir(def_id),
+            mir: self.encode_optimized_mir(def_id),
         }
     }
 
@@ -433,7 +433,7 @@ impl<'a, 'b: 'a, 'tcx: 'b> EntryBuilder<'a, 'b, 'tcx> {
             predicates: Some(self.encode_predicates(def_id)),
 
             ast: None,
-            mir: self.encode_mir(def_id),
+            mir: self.encode_optimized_mir(def_id),
         }
     }
 
@@ -528,7 +528,7 @@ impl<'a, 'b: 'a, 'tcx: 'b> EntryBuilder<'a, 'b, 'tcx> {
             } else {
                 None
             },
-            mir: self.encode_mir(def_id),
+            mir: self.encode_optimized_mir(def_id),
         }
     }
 
@@ -598,7 +598,7 @@ impl<'a, 'b: 'a, 'tcx: 'b> EntryBuilder<'a, 'b, 'tcx> {
             predicates: Some(self.encode_predicates(def_id)),
 
             ast: ast.map(|body| self.encode_body(body)),
-            mir: if mir { self.encode_mir(def_id) } else { None },
+            mir: if mir { self.encode_optimized_mir(def_id) } else { None },
         }
     }
 
@@ -619,9 +619,14 @@ impl<'a, 'b: 'a, 'tcx: 'b> EntryBuilder<'a, 'b, 'tcx> {
         self.lazy_seq(names.iter().map(|name| name.node))
     }
 
-    fn encode_mir(&mut self, def_id: DefId) -> Option<Lazy<mir::Mir<'tcx>>> {
+    fn encode_optimized_mir(&mut self, def_id: DefId) -> Option<Lazy<mir::Mir<'tcx>>> {
         debug!("EntryBuilder::encode_mir({:?})", def_id);
-        self.tcx.maps.mir.borrow().get(&def_id).map(|mir| self.lazy(&*mir.borrow()))
+        if self.tcx.mir_keys(LOCAL_CRATE).contains(&def_id) {
+            let mir = self.tcx.optimized_mir(def_id);
+            Some(self.lazy(&mir))
+        } else {
+            None
+        }
     }
 
     // Encodes the inherent implementations of a structure, enumeration, or trait.
@@ -856,15 +861,15 @@ impl<'a, 'b: 'a, 'tcx: 'b> EntryBuilder<'a, 'b, 'tcx> {
             },
             mir: match item.node {
                 hir::ItemStatic(..) if self.tcx.sess.opts.debugging_opts.always_encode_mir => {
-                    self.encode_mir(def_id)
+                    self.encode_optimized_mir(def_id)
                 }
-                hir::ItemConst(..) => self.encode_mir(def_id),
+                hir::ItemConst(..) => self.encode_optimized_mir(def_id),
                 hir::ItemFn(_, _, constness, _, ref generics, _) => {
                     let tps_len = generics.ty_params.len();
                     let needs_inline = tps_len > 0 || attr::requests_inline(&item.attrs);
                     let always_encode_mir = self.tcx.sess.opts.debugging_opts.always_encode_mir;
                     if needs_inline || constness == hir::Constness::Const || always_encode_mir {
-                        self.encode_mir(def_id)
+                        self.encode_optimized_mir(def_id)
                     } else {
                         None
                     }
@@ -1161,7 +1166,7 @@ impl<'a, 'b: 'a, 'tcx: 'b> EntryBuilder<'a, 'b, 'tcx> {
             predicates: None,
 
             ast: None,
-            mir: self.encode_mir(def_id),
+            mir: self.encode_optimized_mir(def_id),
         }
     }
 
@@ -1187,7 +1192,7 @@ impl<'a, 'b: 'a, 'tcx: 'b> EntryBuilder<'a, 'b, 'tcx> {
             predicates: Some(self.encode_predicates(def_id)),
 
             ast: Some(self.encode_body(body)),
-            mir: self.encode_mir(def_id),
+            mir: self.encode_optimized_mir(def_id),
         }
     }
 

src/librustc_mir/build/into.rs

@@ -18,7 +18,7 @@ use build::{BlockAnd, Builder};
 use hair::*;
 use rustc::mir::*;
 
-pub trait EvalInto<'tcx> {
+pub(in build) trait EvalInto<'tcx> {
     fn eval_into<'a, 'gcx>(self,
                            builder: &mut Builder<'a, 'gcx, 'tcx>,
                            destination: &Lvalue<'tcx>,

src/librustc_mir/build/mod.rs

@@ -8,24 +8,225 @@
 // option. This file may not be copied, modified, or distributed
 // except according to those terms.
 
+
+use build;
 use hair::cx::Cx;
 use hair::Pattern;
-
-use rustc::middle::region::{CodeExtent, CodeExtentData};
-use rustc::ty::{self, Ty};
-use rustc::mir::*;
-use rustc::util::nodemap::NodeMap;
 use rustc::hir;
+use rustc::hir::def_id::DefId;
+use rustc::middle::region::{CodeExtent, CodeExtentData};
+use rustc::mir::*;
+use rustc::mir::transform::MirSource;
+use rustc::mir::visit::MutVisitor;
+use rustc::traits::Reveal;
+use rustc::ty::{self, Ty, TyCtxt};
+use rustc::ty::subst::Substs;
+use rustc::util::nodemap::NodeMap;
+use rustc_data_structures::indexed_vec::{IndexVec, Idx};
+use shim;
+use std::mem;
+use std::u32;
 use syntax::abi::Abi;
 use syntax::ast;
 use syntax::symbol::keywords;
 use syntax_pos::Span;
+use util as mir_util;
 
-use rustc_data_structures::indexed_vec::{IndexVec, Idx};
+/// Construct the MIR for a given def-id.
+pub fn mir_build<'a, 'tcx>(tcx: TyCtxt<'a, 'tcx, 'tcx>, def_id: DefId) -> Mir<'tcx> {
+    let id = tcx.hir.as_local_node_id(def_id).unwrap();
+    let unsupported = || {
+        span_bug!(tcx.hir.span(id), "can't build MIR for {:?}", def_id);
+    };
 
-use std::u32;
+    // Figure out what primary body this item has.
+    let body_id = match tcx.hir.get(id) {
+        hir::map::NodeItem(item) => {
+            match item.node {
+                hir::ItemConst(_, body) |
+                hir::ItemStatic(_, _, body) |
+                hir::ItemFn(.., body) => body,
+                _ => unsupported()
+            }
+        }
+        hir::map::NodeTraitItem(item) => {
+            match item.node {
+                hir::TraitItemKind::Const(_, Some(body)) |
+                hir::TraitItemKind::Method(_,
+                    hir::TraitMethod::Provided(body)) => body,
+                _ => unsupported()
+            }
+        }
+        hir::map::NodeImplItem(item) => {
+            match item.node {
+                hir::ImplItemKind::Const(_, body) |
+                hir::ImplItemKind::Method(_, body) => body,
+                _ => unsupported()
+            }
+        }
+        hir::map::NodeExpr(expr) => {
+            // FIXME(eddyb) Closures should have separate
+            // function definition IDs and expression IDs.
+            // Type-checking should not let closures get
+            // this far in a constant position.
+            // Assume that everything other than closures
+            // is a constant "initializer" expression.
+            match expr.node {
+                hir::ExprClosure(_, _, body, _) => body,
+                _ => hir::BodyId { node_id: expr.id }
+            }
+        }
+        hir::map::NodeVariant(variant) =>
+            return create_constructor_shim(tcx, id, &variant.node.data),
+        hir::map::NodeStructCtor(ctor) =>
+            return create_constructor_shim(tcx, id, ctor),
+        _ => unsupported()
+    };
 
-pub struct Builder<'a, 'gcx: 'a+'tcx, 'tcx: 'a> {
+    let src = MirSource::from_node(tcx, id);
+    tcx.infer_ctxt(body_id, Reveal::UserFacing).enter(|infcx| {
+        let cx = Cx::new(&infcx, src);
+        let mut mir = if cx.tables().tainted_by_errors {
+            build::construct_error(cx, body_id)
+        } else if let MirSource::Fn(id) = src {
+            // fetch the fully liberated fn signature (that is, all bound
+            // types/lifetimes replaced)
+            let fn_sig = cx.tables().liberated_fn_sigs[&id].clone();
+
+            let ty = tcx.type_of(tcx.hir.local_def_id(id));
+            let mut abi = fn_sig.abi;
+            let implicit_argument = if let ty::TyClosure(..) = ty.sty {
+                // HACK(eddyb) Avoid having RustCall on closures,
+                // as it adds unnecessary (and wrong) auto-tupling.
+                abi = Abi::Rust;
+                Some((closure_self_ty(tcx, id, body_id), None))
+            } else {
+                None
+            };
+
+            let body = tcx.hir.body(body_id);
+            let explicit_arguments =
+                body.arguments
+                    .iter()
+                    .enumerate()
+                    .map(|(index, arg)| {
+                        (fn_sig.inputs()[index], Some(&*arg.pat))
+                    });
+
+            let arguments = implicit_argument.into_iter().chain(explicit_arguments);
+            build::construct_fn(cx, id, arguments, abi, fn_sig.output(), body)
+        } else {
+            build::construct_const(cx, body_id)
+        };
+
+        // Convert the Mir to global types.
+        let mut globalizer = GlobalizeMir {
+            tcx: tcx,
+            span: mir.span
+        };
+        globalizer.visit_mir(&mut mir);
+        let mir = unsafe {
+            mem::transmute::<Mir, Mir<'tcx>>(mir)
+        };
+
+        mir_util::dump_mir(tcx, None, "mir_map", &0, src, &mir);
+
+        mir
+    })
+}
+
+/// A pass to lift all the types and substitutions in a Mir
+/// to the global tcx. Sadly, we don't have a "folder" that
+/// can change 'tcx so we have to transmute afterwards.
+struct GlobalizeMir<'a, 'gcx: 'a> {
+    tcx: TyCtxt<'a, 'gcx, 'gcx>,
+    span: Span
+}
+
+impl<'a, 'gcx: 'tcx, 'tcx> MutVisitor<'tcx> for GlobalizeMir<'a, 'gcx> {
+    fn visit_ty(&mut self, ty: &mut Ty<'tcx>) {
+        if let Some(lifted) = self.tcx.lift(ty) {
+            *ty = lifted;
+        } else {
+            span_bug!(self.span,
+                      "found type `{:?}` with inference types/regions in MIR",
+                      ty);
+        }
+    }
+
+    fn visit_substs(&mut self, substs: &mut &'tcx Substs<'tcx>) {
+        if let Some(lifted) = self.tcx.lift(substs) {
+            *substs = lifted;
+        } else {
+            span_bug!(self.span,
+                      "found substs `{:?}` with inference types/regions in MIR",
+                      substs);
+        }
+    }
+}
+
+fn create_constructor_shim<'a, 'tcx>(tcx: TyCtxt<'a, 'tcx, 'tcx>,
+                                     ctor_id: ast::NodeId,
+                                     v: &'tcx hir::VariantData)
+                                     -> Mir<'tcx>
+{
+    let span = tcx.hir.span(ctor_id);
+    if let hir::VariantData::Tuple(ref fields, ctor_id) = *v {
+        let pe = ty::ParameterEnvironment::for_item(tcx, ctor_id);
+        tcx.infer_ctxt(pe, Reveal::UserFacing).enter(|infcx| {
+            let (mut mir, src) =
+                shim::build_adt_ctor(&infcx, ctor_id, fields, span);
+
+            // Convert the Mir to global types.
+            let tcx = infcx.tcx.global_tcx();
+            let mut globalizer = GlobalizeMir {
+                tcx: tcx,
+                span: mir.span
+            };
+            globalizer.visit_mir(&mut mir);
+            let mir = unsafe {
+                mem::transmute::<Mir, Mir<'tcx>>(mir)
+            };
+
+            mir_util::dump_mir(tcx, None, "mir_map", &0, src, &mir);
+
+            mir
+        })
+    } else {
+        span_bug!(span, "attempting to create MIR for non-tuple variant {:?}", v);
+    }
+}
+
+///////////////////////////////////////////////////////////////////////////
+// BuildMir -- walks a crate, looking for fn items and methods to build MIR from
+
+fn closure_self_ty<'a, 'tcx>(tcx: TyCtxt<'a, 'tcx, 'tcx>,
+                             closure_expr_id: ast::NodeId,
+                             body_id: hir::BodyId)
+                             -> Ty<'tcx> {
+    let closure_ty = tcx.body_tables(body_id).node_id_to_type(closure_expr_id);
+
+    let region = ty::ReFree(ty::FreeRegion {
+        scope: Some(tcx.item_extent(body_id.node_id)),
+        bound_region: ty::BoundRegion::BrEnv,
+    });
+    let region = tcx.mk_region(region);
+
+    match tcx.closure_kind(tcx.hir.local_def_id(closure_expr_id)) {
+        ty::ClosureKind::Fn =>
+            tcx.mk_ref(region,
+                       ty::TypeAndMut { ty: closure_ty,
+                                        mutbl: hir::MutImmutable }),
+        ty::ClosureKind::FnMut =>
+            tcx.mk_ref(region,
+                       ty::TypeAndMut { ty: closure_ty,
+                                        mutbl: hir::MutMutable }),
+        ty::ClosureKind::FnOnce =>
+            closure_ty
+    }
+}
+
+struct Builder<'a, 'gcx: 'a+'tcx, 'tcx: 'a> {
     hir: Cx<'a, 'gcx, 'tcx>,
     cfg: CFG<'tcx>,
 
@@ -82,7 +283,7 @@ impl Idx for ScopeId {
 /// convenient.
 
 #[must_use] // if you don't use one of these results, you're leaving a dangling edge
-pub struct BlockAnd<T>(BasicBlock, T);
+struct BlockAnd<T>(BasicBlock, T);
 
 trait BlockAndExtension {
     fn and<T>(self, v: T) -> BlockAnd<T>;
@@ -121,13 +322,13 @@ macro_rules! unpack {
 ///////////////////////////////////////////////////////////////////////////
 /// the main entry point for building MIR for a function
 
-pub fn construct_fn<'a, 'gcx, 'tcx, A>(hir: Cx<'a, 'gcx, 'tcx>,
-                                       fn_id: ast::NodeId,
-                                       arguments: A,
-                                       abi: Abi,
-                                       return_ty: Ty<'gcx>,
-                                       body: &'gcx hir::Body)
-                                       -> Mir<'tcx>
+fn construct_fn<'a, 'gcx, 'tcx, A>(hir: Cx<'a, 'gcx, 'tcx>,
+                                   fn_id: ast::NodeId,
+                                   arguments: A,
+                                   abi: Abi,
+                                   return_ty: Ty<'gcx>,
+                                   body: &'gcx hir::Body)
+                                   -> Mir<'tcx>
     where A: Iterator<Item=(Ty<'gcx>, Option<&'gcx hir::Pat>)>
 {
     let arguments: Vec<_> = arguments.collect();

src/librustc_mir/callgraph.rs

@@ -1,252 +0,0 @@
-// Copyright 2016 The Rust Project Developers. See the COPYRIGHT
-// file at the top-level directory of this distribution and at
-// http://rust-lang.org/COPYRIGHT.
-//
-// Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
-// http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
-// <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your
-// option. This file may not be copied, modified, or distributed
-// except according to those terms.
-
-//! MIR-based callgraph.
-//!
-//! This only considers direct calls
-
-use rustc::hir::def_id::DefId;
-use rustc_data_structures::graph;
-
-use rustc::mir::*;
-use rustc::mir::visit::*;
-
-use rustc::ty;
-
-use rustc::util::nodemap::DefIdMap;
-
-pub struct CallGraph {
-    node_map: DefIdMap<graph::NodeIndex>,
-    graph: graph::Graph<DefId, ()>
-}
-
-impl CallGraph {
-    // FIXME: allow for construction of a callgraph that inspects
-    // cross-crate MIRs if available.
-    pub fn build<'a, 'tcx>(tcx: ty::TyCtxt<'a, 'tcx, 'tcx>) -> CallGraph {
-        let def_ids = tcx.maps.mir.borrow().keys();
-
-        let mut callgraph = CallGraph {
-            node_map: DefIdMap(),
-            graph: graph::Graph::new()
-        };
-
-        for def_id in def_ids {
-            if !def_id.is_local() { continue; }
-
-            let idx = callgraph.add_node(def_id);
-
-            let mut call_visitor = CallVisitor {
-                caller: idx,
-                graph: &mut callgraph
-            };
-
-            let mir = tcx.item_mir(def_id);
-            call_visitor.visit_mir(&mir);
-        }
-
-        callgraph
-    }
-
-    // Iterate over the strongly-connected components of the graph
-    pub fn scc_iter(&self) -> SCCIterator {
-        SCCIterator::new(&self.graph)
-    }
-
-    // Get the def_id for the given graph node
-    pub fn def_id(&self, node: graph::NodeIndex) -> DefId {
-        *self.graph.node_data(node)
-    }
-
-    fn add_node(&mut self, id: DefId) -> graph::NodeIndex {
-        let graph = &mut self.graph;
-        *self.node_map.entry(id).or_insert_with(|| {
-            graph.add_node(id)
-        })
-    }
-}
-
-struct CallVisitor<'a> {
-    caller: graph::NodeIndex,
-    graph: &'a mut CallGraph
-}
-
-impl<'a, 'tcx> Visitor<'tcx> for CallVisitor<'a> {
-    fn visit_terminator_kind(&mut self, _block: BasicBlock,
-                             kind: &TerminatorKind<'tcx>, _loc: Location) {
-        if let TerminatorKind::Call {
-            func: Operand::Constant(ref f)
-            , .. } = *kind {
-            if let ty::TyFnDef(def_id, _, _) = f.ty.sty {
-                let callee = self.graph.add_node(def_id);
-                self.graph.graph.add_edge(self.caller, callee, ());
-            }
-        }
-    }
-}
-
-struct StackElement<'g> {
-    node: graph::NodeIndex,
-    lowlink: usize,
-    children: graph::AdjacentTargets<'g, DefId, ()>
-}
-
-/**
- * Iterator over strongly-connected-components using Tarjan's algorithm[1]
- *
- * [1]: https://en.wikipedia.org/wiki/Tarjan%27s_strongly_connected_components_algorithm
- */
-pub struct SCCIterator<'g> {
-    graph: &'g graph::Graph<DefId, ()>,
-    index: usize,
-    node_indices: Vec<Option<usize>>,
-    scc_stack: Vec<graph::NodeIndex>,
-    current_scc: Vec<graph::NodeIndex>,
-    visit_stack: Vec<StackElement<'g>>,
-}
-
-impl<'g> SCCIterator<'g> {
-    pub fn new(graph: &'g graph::Graph<DefId, ()>) -> SCCIterator<'g> {
-        if graph.len_nodes() == 0 {
-            return SCCIterator {
-                graph: graph,
-                index: 0,
-                node_indices: Vec::new(),
-                scc_stack: Vec::new(),
-                current_scc: Vec::new(),
-                visit_stack: Vec::new()
-            };
-        }
-
-        let first = graph::NodeIndex(0);
-
-        SCCIterator::with_entry(graph, first)
-    }
-
-    pub fn with_entry(graph: &'g graph::Graph<DefId, ()>,
-                      entry: graph::NodeIndex) -> SCCIterator<'g> {
-        let mut iter = SCCIterator {
-            graph: graph,
-            index: 0,
-            node_indices: Vec::with_capacity(graph.len_nodes()),
-            scc_stack: Vec::new(),
-            current_scc: Vec::new(),
-            visit_stack: Vec::new()
-        };
-
-        iter.visit_one(entry);
-
-        iter
-    }
-
-    fn get_next(&mut self) {
-        self.current_scc.clear();
-
-        while !self.visit_stack.is_empty() {
-            self.visit_children();
-
-            let node = self.visit_stack.pop().unwrap();
-
-            if let Some(last) = self.visit_stack.last_mut() {
-                if last.lowlink > node.lowlink {
-                    last.lowlink = node.lowlink;
-                }
-            }
-
-            debug!("TarjanSCC: Popped node {:?} : lowlink = {:?}; index = {:?}",
-                   node.node, node.lowlink, self.node_index(node.node).unwrap());
-
-            if node.lowlink != self.node_index(node.node).unwrap() {
-                continue;
-            }
-
-            loop {
-                let n = self.scc_stack.pop().unwrap();
-                self.current_scc.push(n);
-                self.set_node_index(n, !0);
-                if n == node.node { return; }
-            }
-        }
-    }
-
-    fn visit_one(&mut self, node: graph::NodeIndex) {
-        self.index += 1;
-        let idx =  self.index;
-        self.set_node_index(node, idx);
-        self.scc_stack.push(node);
-        self.visit_stack.push(StackElement {
-            node: node,
-            lowlink: self.index,
-            children: self.graph.successor_nodes(node)
-        });
-        debug!("TarjanSCC: Node {:?} : index = {:?}", node, idx);
-    }
-
-    fn visit_children(&mut self) {
-        while let Some(child) = self.visit_stack.last_mut().unwrap().children.next() {
-            if let Some(child_num) = self.node_index(child) {
-                let cur = self.visit_stack.last_mut().unwrap();
-                if cur.lowlink > child_num {
-                    cur.lowlink = child_num;
-                }
-            } else {
-                self.visit_one(child);
-            }
-        }
-    }
-
-    fn node_index(&self, node: graph::NodeIndex) -> Option<usize> {
-        self.node_indices.get(node.node_id()).and_then(|&idx| idx)
-    }
-
-    fn set_node_index(&mut self, node: graph::NodeIndex, idx: usize) {
-        let i = node.node_id();
-        if i >= self.node_indices.len() {
-            self.node_indices.resize(i + 1, None);
-        }
-        self.node_indices[i] = Some(idx);
-    }
-}
-
-impl<'g> Iterator for SCCIterator<'g> {
-    type Item = Vec<graph::NodeIndex>;
-
-    fn next(&mut self) -> Option<Vec<graph::NodeIndex>> {
-        self.get_next();
-
-        if self.current_scc.is_empty() {
-            // Try a new root for the next SCC, if the node_indices
-            // map is doesn't contain all nodes, use the smallest one
-            // with no entry, otherwise find the first empty node.
-            //
-            // FIXME: This should probably use a set of precomputed
-            // roots instead
-            if self.node_indices.len() < self.graph.len_nodes() {
-                let idx = graph::NodeIndex(self.node_indices.len());
-                self.visit_one(idx);
-            } else {
-                for idx in 0..self.node_indices.len() {
-                    if self.node_indices[idx].is_none() {
-                        let idx = graph::NodeIndex(idx);
-                        self.visit_one(idx);
-                        break;
-                    }
-                }
-            }
-            self.get_next();
-        }
-
-        if self.current_scc.is_empty() {
-            None
-        } else {
-            Some(self.current_scc.clone())
-        }
-    }
-}

src/librustc_mir/hair/cx/mod.rs

@@ -26,7 +26,7 @@ use rustc::middle::region::RegionMaps;
 use rustc::infer::InferCtxt;
 use rustc::ty::subst::Subst;
 use rustc::ty::{self, Ty, TyCtxt};
-use syntax::symbol::{Symbol, InternedString};
+use syntax::symbol::Symbol;
 use rustc::hir;
 use rustc_const_math::{ConstInt, ConstUsize};
 use std::rc::Rc;
@@ -103,10 +103,6 @@ impl<'a, 'gcx, 'tcx> Cx<'a, 'gcx, 'tcx> {
         self.tcx.mk_nil()
     }
 
-    pub fn str_literal(&mut self, value: InternedString) -> Literal<'tcx> {
-        Literal::Value { value: ConstVal::Str(value) }
-    }
-
     pub fn true_literal(&mut self) -> Literal<'tcx> {
         Literal::Value { value: ConstVal::Bool(true) }
     }

src/librustc_mir/lib.rs

@@ -46,18 +46,15 @@ extern crate rustc_const_eval;
 
 pub mod diagnostics;
 
-pub mod build;
-pub mod callgraph;
+mod build;
 mod hair;
 mod shim;
-pub mod mir_map;
 pub mod transform;
 pub mod util;
 
 use rustc::ty::maps::Providers;
 
 pub fn provide(providers: &mut Providers) {
-    mir_map::provide(providers);
     shim::provide(providers);
-    transform::qualify_consts::provide(providers);
+    transform::provide(providers);
 }

src/librustc_mir/mir_map.rs

@@ -1,273 +0,0 @@
-// Copyright 2015 The Rust Project Developers. See the COPYRIGHT
-// file at the top-level directory of this distribution and at
-// http://rust-lang.org/COPYRIGHT.
-//
-// Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
-// http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
-// <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your
-// option. This file may not be copied, modified, or distributed
-// except according to those terms.
-
-//! An experimental pass that scources for `#[rustc_mir]` attributes,
-//! builds the resulting MIR, and dumps it out into a file for inspection.
-//!
-//! The attribute formats that are currently accepted are:
-//!
-//! - `#[rustc_mir(graphviz="file.gv")]`
-//! - `#[rustc_mir(pretty="file.mir")]`
-
-use build;
-use rustc::hir::def_id::DefId;
-use rustc::dep_graph::DepNode;
-use rustc::mir::Mir;
-use rustc::mir::transform::MirSource;
-use rustc::mir::visit::MutVisitor;
-use shim;
-use hair::cx::Cx;
-use util as mir_util;
-
-use rustc::traits::Reveal;
-use rustc::ty::{self, Ty, TyCtxt};
-use rustc::ty::maps::Providers;
-use rustc::ty::subst::Substs;
-use rustc::hir;
-use rustc::hir::intravisit::{self, Visitor, NestedVisitorMap};
-use syntax::abi::Abi;
-use syntax::ast;
-use syntax_pos::Span;
-
-use std::cell::RefCell;
-use std::mem;
-
-pub fn build_mir_for_crate<'a, 'tcx>(tcx: TyCtxt<'a, 'tcx, 'tcx>) {
-    tcx.dep_graph.with_task(DepNode::MirKrate, tcx, (), build_mir_for_crate_task);
-
-    fn build_mir_for_crate_task<'a, 'tcx>(tcx: TyCtxt<'a, 'tcx, 'tcx>, (): ()) {
-        tcx.visit_all_bodies_in_krate(|body_owner_def_id, _body_id| {
-            tcx.item_mir(body_owner_def_id);
-        });
-
-        // Tuple struct/variant constructors don't have a BodyId, so we need
-        // to build them separately.
-        struct GatherCtors<'a, 'tcx: 'a> {
-            tcx: TyCtxt<'a, 'tcx, 'tcx>
-        }
-        impl<'a, 'tcx> Visitor<'tcx> for GatherCtors<'a, 'tcx> {
-            fn visit_variant_data(&mut self,
-                                  v: &'tcx hir::VariantData,
-                                  _: ast::Name,
-                                  _: &'tcx hir::Generics,
-                                  _: ast::NodeId,
-                                  _: Span) {
-                if let hir::VariantData::Tuple(_, node_id) = *v {
-                    self.tcx.item_mir(self.tcx.hir.local_def_id(node_id));
-                }
-                intravisit::walk_struct_def(self, v)
-            }
-            fn nested_visit_map<'b>(&'b mut self) -> NestedVisitorMap<'b, 'tcx> {
-                NestedVisitorMap::None
-            }
-        }
-        tcx.hir.krate().visit_all_item_likes(&mut GatherCtors {
-            tcx: tcx
-        }.as_deep_visitor());
-    }
-}
-
-pub fn provide(providers: &mut Providers) {
-    providers.mir = build_mir;
-}
-
-fn build_mir<'a, 'tcx>(tcx: TyCtxt<'a, 'tcx, 'tcx>, def_id: DefId)
-                       -> &'tcx RefCell<Mir<'tcx>> {
-    let id = tcx.hir.as_local_node_id(def_id).unwrap();
-    let unsupported = || {
-        span_bug!(tcx.hir.span(id), "can't build MIR for {:?}", def_id);
-    };
-
-    // Figure out what primary body this item has.
-    let body_id = match tcx.hir.get(id) {
-        hir::map::NodeItem(item) => {
-            match item.node {
-                hir::ItemConst(_, body) |
-                hir::ItemStatic(_, _, body) |
-                hir::ItemFn(.., body) => body,
-                _ => unsupported()
-            }
-        }
-        hir::map::NodeTraitItem(item) => {
-            match item.node {
-                hir::TraitItemKind::Const(_, Some(body)) |
-                hir::TraitItemKind::Method(_,
-                    hir::TraitMethod::Provided(body)) => body,
-                _ => unsupported()
-            }
-        }
-        hir::map::NodeImplItem(item) => {
-            match item.node {
-                hir::ImplItemKind::Const(_, body) |
-                hir::ImplItemKind::Method(_, body) => body,
-                _ => unsupported()
-            }
-        }
-        hir::map::NodeExpr(expr) => {
-            // FIXME(eddyb) Closures should have separate
-            // function definition IDs and expression IDs.
-            // Type-checking should not let closures get
-            // this far in a constant position.
-            // Assume that everything other than closures
-            // is a constant "initializer" expression.
-            match expr.node {
-                hir::ExprClosure(_, _, body, _) => body,
-                _ => hir::BodyId { node_id: expr.id }
-            }
-        }
-        hir::map::NodeVariant(variant) =>
-            return create_constructor_shim(tcx, id, &variant.node.data),
-        hir::map::NodeStructCtor(ctor) =>
-            return create_constructor_shim(tcx, id, ctor),
-        _ => unsupported()
-    };
-
-    let src = MirSource::from_node(tcx, id);
-    tcx.infer_ctxt(body_id, Reveal::UserFacing).enter(|infcx| {
-        let cx = Cx::new(&infcx, src);
-        let mut mir = if cx.tables().tainted_by_errors {
-            build::construct_error(cx, body_id)
-        } else if let MirSource::Fn(id) = src {
-            // fetch the fully liberated fn signature (that is, all bound
-            // types/lifetimes replaced)
-            let fn_sig = cx.tables().liberated_fn_sigs[&id].clone();
-
-            let ty = tcx.type_of(tcx.hir.local_def_id(id));
-            let mut abi = fn_sig.abi;
-            let implicit_argument = if let ty::TyClosure(..) = ty.sty {
-                // HACK(eddyb) Avoid having RustCall on closures,
-                // as it adds unnecessary (and wrong) auto-tupling.
-                abi = Abi::Rust;
-                Some((closure_self_ty(tcx, id, body_id), None))
-            } else {
-                None
-            };
-
-            let body = tcx.hir.body(body_id);
-            let explicit_arguments =
-                body.arguments
-                    .iter()
-                    .enumerate()
-                    .map(|(index, arg)| {
-                        (fn_sig.inputs()[index], Some(&*arg.pat))
-                    });
-
-            let arguments = implicit_argument.into_iter().chain(explicit_arguments);
-            build::construct_fn(cx, id, arguments, abi, fn_sig.output(), body)
-        } else {
-            build::construct_const(cx, body_id)
-        };
-
-        // Convert the Mir to global types.
-        let mut globalizer = GlobalizeMir {
-            tcx: tcx,
-            span: mir.span
-        };
-        globalizer.visit_mir(&mut mir);
-        let mir = unsafe {
-            mem::transmute::<Mir, Mir<'tcx>>(mir)
-        };
-
-        mir_util::dump_mir(tcx, "mir_map", &0, src, &mir);
-
-        tcx.alloc_mir(mir)
-    })
-}
-
-/// A pass to lift all the types and substitutions in a Mir
-/// to the global tcx. Sadly, we don't have a "folder" that
-/// can change 'tcx so we have to transmute afterwards.
-struct GlobalizeMir<'a, 'gcx: 'a> {
-    tcx: TyCtxt<'a, 'gcx, 'gcx>,
-    span: Span
-}
-
-impl<'a, 'gcx: 'tcx, 'tcx> MutVisitor<'tcx> for GlobalizeMir<'a, 'gcx> {
-    fn visit_ty(&mut self, ty: &mut Ty<'tcx>) {
-        if let Some(lifted) = self.tcx.lift(ty) {
-            *ty = lifted;
-        } else {
-            span_bug!(self.span,
-                      "found type `{:?}` with inference types/regions in MIR",
-                      ty);
-        }
-    }
-
-    fn visit_substs(&mut self, substs: &mut &'tcx Substs<'tcx>) {
-        if let Some(lifted) = self.tcx.lift(substs) {
-            *substs = lifted;
-        } else {
-            span_bug!(self.span,
-                      "found substs `{:?}` with inference types/regions in MIR",
-                      substs);
-        }
-    }
-}
-
-fn create_constructor_shim<'a, 'tcx>(tcx: TyCtxt<'a, 'tcx, 'tcx>,
-                                     ctor_id: ast::NodeId,
-                                     v: &'tcx hir::VariantData)
-                                     -> &'tcx RefCell<Mir<'tcx>>
-{
-    let span = tcx.hir.span(ctor_id);
-    if let hir::VariantData::Tuple(ref fields, ctor_id) = *v {
-        let pe = ty::ParameterEnvironment::for_item(tcx, ctor_id);
-        tcx.infer_ctxt(pe, Reveal::UserFacing).enter(|infcx| {
-            let (mut mir, src) =
-                shim::build_adt_ctor(&infcx, ctor_id, fields, span);
-
-            // Convert the Mir to global types.
-            let tcx = infcx.tcx.global_tcx();
-            let mut globalizer = GlobalizeMir {
-                tcx: tcx,
-                span: mir.span
-            };
-            globalizer.visit_mir(&mut mir);
-            let mir = unsafe {
-                mem::transmute::<Mir, Mir<'tcx>>(mir)
-            };
-
-            mir_util::dump_mir(tcx, "mir_map", &0, src, &mir);
-
-            tcx.alloc_mir(mir)
-        })
-    } else {
-        span_bug!(span, "attempting to create MIR for non-tuple variant {:?}", v);
-    }
-}
-
-///////////////////////////////////////////////////////////////////////////
-// BuildMir -- walks a crate, looking for fn items and methods to build MIR from
-
-fn closure_self_ty<'a, 'tcx>(tcx: TyCtxt<'a, 'tcx, 'tcx>,
-                             closure_expr_id: ast::NodeId,
-                             body_id: hir::BodyId)
-                             -> Ty<'tcx> {
-    let closure_ty = tcx.body_tables(body_id).node_id_to_type(closure_expr_id);
-
-    let region = ty::ReFree(ty::FreeRegion {
-        scope: Some(tcx.item_extent(body_id.node_id)),
-        bound_region: ty::BoundRegion::BrEnv,
-    });
-    let region = tcx.mk_region(region);
-
-    match tcx.closure_kind(tcx.hir.local_def_id(closure_expr_id)) {
-        ty::ClosureKind::Fn =>
-            tcx.mk_ref(region,
-                       ty::TypeAndMut { ty: closure_ty,
-                                        mutbl: hir::MutImmutable }),
-        ty::ClosureKind::FnMut =>
-            tcx.mk_ref(region,
-                       ty::TypeAndMut { ty: closure_ty,
-                                        mutbl: hir::MutMutable }),
-        ty::ClosureKind::FnOnce =>
-            closure_ty
-    }
-}

src/librustc_mir/shim.rs

@@ -24,10 +24,8 @@ use syntax::abi::Abi;
 use syntax::ast;
 use syntax_pos::Span;
 
-use std::cell::RefCell;
 use std::fmt;
 use std::iter;
-use std::mem;
 
 use transform::{add_call_guards, no_landing_pads, simplify};
 use util::elaborate_drops::{self, DropElaborator, DropStyle, DropFlagMode};
@@ -39,7 +37,7 @@ pub fn provide(providers: &mut Providers) {
 
 fn make_shim<'a, 'tcx>(tcx: ty::TyCtxt<'a, 'tcx, 'tcx>,
                        instance: ty::InstanceDef<'tcx>)
-                       -> &'tcx RefCell<Mir<'tcx>>
+                       -> &'tcx Mir<'tcx>
 {
     debug!("make_shim({:?})", instance);
     let did = instance.def_id();
@@ -116,10 +114,7 @@ fn make_shim<'a, 'tcx>(tcx: ty::TyCtxt<'a, 'tcx, 'tcx>,
         add_call_guards::add_call_guards(&mut result);
     debug!("make_shim({:?}) = {:?}", instance, result);
 
-    let result = tcx.alloc_mir(result);
-    // Perma-borrow MIR from shims to prevent mutation.
-    mem::forget(result.borrow());
-    result
+    tcx.alloc_mir(result)
 }
 
 #[derive(Copy, Clone, Debug, PartialEq)]

src/librustc_mir/transform/add_call_guards.rs

@@ -10,7 +10,7 @@
 
 use rustc::ty::TyCtxt;
 use rustc::mir::*;
-use rustc::mir::transform::{MirPass, MirSource, Pass};
+use rustc::mir::transform::{MirPass, MirSource};
 use rustc_data_structures::indexed_vec::{Idx, IndexVec};
 
 pub struct AddCallGuards;
@@ -35,8 +35,11 @@ pub struct AddCallGuards;
  *
  */
 
-impl<'tcx> MirPass<'tcx> for AddCallGuards {
-    fn run_pass<'a>(&mut self, _tcx: TyCtxt<'a, 'tcx, 'tcx>, _src: MirSource, mir: &mut Mir<'tcx>) {
+impl MirPass for AddCallGuards {
+    fn run_pass<'a, 'tcx>(&self,
+                          _tcx: TyCtxt<'a, 'tcx, 'tcx>,
+                          _src: MirSource,
+                          mir: &mut Mir<'tcx>) {
         add_call_guards(mir);
     }
 }
@@ -82,5 +85,3 @@ pub fn add_call_guards(mir: &mut Mir) {
 
     mir.basic_blocks_mut().extend(new_blocks);
 }
-
-impl Pass for AddCallGuards {}

src/librustc_mir/transform/copy_prop.rs

@@ -30,7 +30,7 @@
 //! future.
 
 use rustc::mir::{Constant, Local, LocalKind, Location, Lvalue, Mir, Operand, Rvalue, StatementKind};
-use rustc::mir::transform::{MirPass, MirSource, Pass};
+use rustc::mir::transform::{MirPass, MirSource};
 use rustc::mir::visit::MutVisitor;
 use rustc::ty::TyCtxt;
 use util::def_use::DefUseAnalysis;
@@ -38,13 +38,11 @@ use transform::qualify_consts;
 
 pub struct CopyPropagation;
 
-impl Pass for CopyPropagation {}
-
-impl<'tcx> MirPass<'tcx> for CopyPropagation {
-    fn run_pass<'a>(&mut self,
-                    tcx: TyCtxt<'a, 'tcx, 'tcx>,
-                    source: MirSource,
-                    mir: &mut Mir<'tcx>) {
+impl MirPass for CopyPropagation {
+    fn run_pass<'a, 'tcx>(&self,
+                          tcx: TyCtxt<'a, 'tcx, 'tcx>,
+                          source: MirSource,
+                          mir: &mut Mir<'tcx>) {
         match source {
             MirSource::Const(_) => {
                 // Don't run on constants, because constant qualification might reject the

src/librustc_mir/transform/deaggregator.rs

@@ -10,16 +10,16 @@
 
 use rustc::ty::TyCtxt;
 use rustc::mir::*;
-use rustc::mir::transform::{MirPass, MirSource, Pass};
+use rustc::mir::transform::{MirPass, MirSource};
 use rustc_data_structures::indexed_vec::Idx;
 
 pub struct Deaggregator;
 
-impl Pass for Deaggregator {}
-
-impl<'tcx> MirPass<'tcx> for Deaggregator {
-    fn run_pass<'a>(&mut self, tcx: TyCtxt<'a, 'tcx, 'tcx>,
-                    source: MirSource, mir: &mut Mir<'tcx>) {
+impl MirPass for Deaggregator {
+    fn run_pass<'a, 'tcx>(&self,
+                          tcx: TyCtxt<'a, 'tcx, 'tcx>,
+                          source: MirSource,
+                          mir: &mut Mir<'tcx>) {
         let node_id = source.item_id();
         let node_path = tcx.item_path_str(tcx.hir.local_def_id(node_id));
         debug!("running on: {:?}", node_path);

src/librustc_mir/transform/dump_mir.rs

@@ -10,70 +10,66 @@
 
 //! This pass just dumps MIR at a specified point.
 
+use std::borrow::Cow;
 use std::fmt;
 use std::fs::File;
 use std::io;
 
+use rustc::mir::Mir;
+use rustc::mir::transform::{MirPass, MirPassIndex, MirSource, MirSuite, PassHook};
 use rustc::session::config::{OutputFilenames, OutputType};
 use rustc::ty::TyCtxt;
-use rustc::mir::*;
-use rustc::mir::transform::{Pass, MirPass, MirPassHook, MirSource};
 use util as mir_util;
 
-pub struct Marker<'a>(pub &'a str);
+pub struct Marker(pub &'static str);
 
-impl<'b, 'tcx> MirPass<'tcx> for Marker<'b> {
-    fn run_pass<'a>(&mut self, _tcx: TyCtxt<'a, 'tcx, 'tcx>,
-                    _src: MirSource, _mir: &mut Mir<'tcx>)
-    {}
+impl MirPass for Marker {
+    fn name<'a>(&'a self) -> Cow<'a, str> {
+        Cow::Borrowed(self.0)
+    }
+
+    fn run_pass<'a, 'tcx>(&self,
+                          _tcx: TyCtxt<'a, 'tcx, 'tcx>,
+                          _source: MirSource,
+                          _mir: &mut Mir<'tcx>)
+    {
+    }
 }
 
-impl<'b> Pass for Marker<'b> {
-    fn name(&self) -> ::std::borrow::Cow<'static, str> { String::from(self.0).into() }
-}
-
-pub struct Disambiguator<'a> {
-    pass: &'a Pass,
+pub struct Disambiguator {
     is_after: bool
 }
 
-impl<'a> fmt::Display for Disambiguator<'a> {
+impl fmt::Display for Disambiguator {
     fn fmt(&self, formatter: &mut fmt::Formatter) -> fmt::Result {
         let title = if self.is_after { "after" } else { "before" };
-        if let Some(fmt) = self.pass.disambiguator() {
-            write!(formatter, "{}-{}", fmt, title)
-        } else {
-            write!(formatter, "{}", title)
-        }
+        write!(formatter, "{}", title)
     }
 }
 
 pub struct DumpMir;
 
-impl<'tcx> MirPassHook<'tcx> for DumpMir {
-    fn on_mir_pass<'a>(
-        &mut self,
-        tcx: TyCtxt<'a, 'tcx, 'tcx>,
-        src: MirSource,
-        mir: &Mir<'tcx>,
-        pass: &Pass,
-        is_after: bool)
+impl PassHook for DumpMir {
+    fn on_mir_pass<'a, 'tcx: 'a>(&self,
+                                 tcx: TyCtxt<'a, 'tcx, 'tcx>,
+                                 suite: MirSuite,
+                                 pass_num: MirPassIndex,
+                                 pass_name: &str,
+                                 source: MirSource,
+                                 mir: &Mir<'tcx>,
+                                 is_after: bool)
     {
-        mir_util::dump_mir(
-            tcx,
-            &*pass.name(),
-            &Disambiguator {
-                pass: pass,
-                is_after: is_after
-            },
-            src,
-            mir
-        );
+        if mir_util::dump_enabled(tcx, pass_name, source) {
+            mir_util::dump_mir(tcx,
+                               Some((suite, pass_num)),
+                               pass_name,
+                               &Disambiguator { is_after },
+                               source,
+                               mir);
+        }
     }
 }
 
-impl<'b> Pass for DumpMir {}
-
 pub fn emit_mir<'a, 'tcx>(
     tcx: TyCtxt<'a, 'tcx, 'tcx>,
     outputs: &OutputFilenames)
@@ -81,6 +77,6 @@ pub fn emit_mir<'a, 'tcx>(
 {
     let path = outputs.path(OutputType::Mir);
     let mut f = File::create(&path)?;
-    mir_util::write_mir_pretty(tcx, tcx.maps.mir.borrow().keys().into_iter(), &mut f)?;
+    mir_util::write_mir_pretty(tcx, None, &mut f)?;
     Ok(())
 }

src/librustc_mir/transform/erase_regions.rs

@@ -16,7 +16,7 @@ use rustc::ty::subst::Substs;
 use rustc::ty::{Ty, TyCtxt, ClosureSubsts};
 use rustc::mir::*;
 use rustc::mir::visit::MutVisitor;
-use rustc::mir::transform::{MirPass, MirSource, Pass};
+use rustc::mir::transform::{MirPass, MirSource};
 
 struct EraseRegionsVisitor<'a, 'tcx: 'a> {
     tcx: TyCtxt<'a, 'tcx, 'tcx>,
@@ -69,11 +69,11 @@ impl<'a, 'tcx> MutVisitor<'tcx> for EraseRegionsVisitor<'a, 'tcx> {
 
 pub struct EraseRegions;
 
-impl Pass for EraseRegions {}
-
-impl<'tcx> MirPass<'tcx> for EraseRegions {
-    fn run_pass<'a>(&mut self, tcx: TyCtxt<'a, 'tcx, 'tcx>,
-                    _: MirSource, mir: &mut Mir<'tcx>) {
+impl MirPass for EraseRegions {
+    fn run_pass<'a, 'tcx>(&self,
+                          tcx: TyCtxt<'a, 'tcx, 'tcx>,
+                          _: MirSource,
+                          mir: &mut Mir<'tcx>) {
         EraseRegionsVisitor::new(tcx).visit_mir(mir);
     }
 }

src/librustc_mir/transform/inline.rs

@@ -14,24 +14,20 @@ use rustc::hir::def_id::DefId;
 
 use rustc_data_structures::bitvec::BitVector;
 use rustc_data_structures::indexed_vec::{Idx, IndexVec};
-use rustc_data_structures::graph;
 
-use rustc::dep_graph::DepNode;
 use rustc::mir::*;
-use rustc::mir::transform::{MirMapPass, MirPassHook, MirSource, Pass};
+use rustc::mir::transform::{MirPass, MirSource};
 use rustc::mir::visit::*;
 use rustc::traits;
 use rustc::ty::{self, Ty, TyCtxt};
 use rustc::ty::subst::{Subst,Substs};
-use rustc::util::nodemap::{DefIdSet};
 
+use std::collections::VecDeque;
 use super::simplify::{remove_dead_blocks, CfgSimplifier};
 
 use syntax::{attr};
 use syntax::abi::Abi;
 
-use callgraph;
-
 const DEFAULT_THRESHOLD: usize = 50;
 const HINT_THRESHOLD: usize = 100;
 
@@ -42,135 +38,65 @@ const UNKNOWN_SIZE_COST: usize = 10;
 
 pub struct Inline;
 
-impl<'tcx> MirMapPass<'tcx> for Inline {
-    fn run_pass<'a>(
-        &mut self,
-        tcx: TyCtxt<'a, 'tcx, 'tcx>,
-        hooks: &mut [Box<for<'s> MirPassHook<'s>>]) {
-
-        if tcx.sess.opts.debugging_opts.mir_opt_level < 2 { return; }
-
-        let _ignore = tcx.dep_graph.in_ignore();
-
-        let callgraph = callgraph::CallGraph::build(tcx);
-
-        let mut inliner = Inliner {
-            tcx: tcx,
-        };
-
-        let def_ids = tcx.maps.mir.borrow().keys();
-        for &def_id in &def_ids {
-            if !def_id.is_local() { continue; }
-
-            let _task = tcx.dep_graph.in_task(DepNode::Mir(def_id));
-            let mut mir = if let Some(mir) = tcx.maps.mir.borrow().get(&def_id) {
-                mir.borrow_mut()
-            } else {
-                continue;
-            };
-
-            tcx.dep_graph.write(DepNode::Mir(def_id));
-
-            let id = tcx.hir.as_local_node_id(def_id).unwrap();
-            let src = MirSource::from_node(tcx, id);
-
-            for hook in &mut *hooks {
-                hook.on_mir_pass(tcx, src, &mut mir, self, false);
-            }
-        }
-
-        for scc in callgraph.scc_iter() {
-            inliner.inline_scc(&callgraph, &scc);
-        }
-
-        for def_id in def_ids {
-            if !def_id.is_local() { continue; }
-
-            let _task = tcx.dep_graph.in_task(DepNode::Mir(def_id));
-            let mut mir = tcx.maps.mir.borrow()[&def_id].borrow_mut();
-            tcx.dep_graph.write(DepNode::Mir(def_id));
-
-            let id = tcx.hir.as_local_node_id(def_id).unwrap();
-            let src = MirSource::from_node(tcx, id);
-
-            for hook in &mut *hooks {
-                hook.on_mir_pass(tcx, src, &mut mir, self, true);
-            }
-        }
-    }
-}
-
-impl<'tcx> Pass for Inline { }
-
-struct Inliner<'a, 'tcx: 'a> {
-    tcx: TyCtxt<'a, 'tcx, 'tcx>,
-}
-
 #[derive(Copy, Clone)]
 struct CallSite<'tcx> {
-    caller: DefId,
     callee: DefId,
     substs: &'tcx Substs<'tcx>,
     bb: BasicBlock,
     location: SourceInfo,
 }
 
-impl<'a, 'tcx> Inliner<'a, 'tcx> {
-    fn inline_scc(&mut self, callgraph: &callgraph::CallGraph, scc: &[graph::NodeIndex]) -> bool {
-        let mut callsites = Vec::new();
-        let mut in_scc = DefIdSet();
-
-        let mut inlined_into = DefIdSet();
-
-        for &node in scc {
-            let def_id = callgraph.def_id(node);
-
-            // Don't inspect functions from other crates
-            let id = if let Some(id) = self.tcx.hir.as_local_node_id(def_id) {
-                id
-            } else {
-                continue;
-            };
-            let src = MirSource::from_node(self.tcx, id);
-            if let MirSource::Fn(_) = src {
-                if let Some(mir) = self.tcx.maybe_item_mir(def_id) {
-                    for (bb, bb_data) in mir.basic_blocks().iter_enumerated() {
-                        // Don't inline calls that are in cleanup blocks.
-                        if bb_data.is_cleanup { continue; }
-
-                        // Only consider direct calls to functions
-                        let terminator = bb_data.terminator();
-                        if let TerminatorKind::Call {
-                            func: Operand::Constant(ref f), .. } = terminator.kind {
-                            if let ty::TyFnDef(callee_def_id, substs, _) = f.ty.sty {
-                                callsites.push(CallSite {
-                                    caller: def_id,
-                                    callee: callee_def_id,
-                                    substs: substs,
-                                    bb: bb,
-                                    location: terminator.source_info
-                                });
-                            }
-                        }
-                    }
-
-                    in_scc.insert(def_id);
-                }
-            }
+impl MirPass for Inline {
+    fn run_pass<'a, 'tcx>(&self,
+                          tcx: TyCtxt<'a, 'tcx, 'tcx>,
+                          source: MirSource,
+                          mir: &mut Mir<'tcx>) {
+        if tcx.sess.opts.debugging_opts.mir_opt_level >= 2 {
+            Inliner { tcx, source }.run_pass(mir);
         }
+    }
+}
 
-        // Move callsites that are in the the SCC to the end so
-        // they're inlined after calls to outside the SCC
-        let mut first_call_in_scc = callsites.len();
+struct Inliner<'a, 'tcx: 'a> {
+    tcx: TyCtxt<'a, 'tcx, 'tcx>,
+    source: MirSource,
+}
 
-        let mut i = 0;
-        while i < first_call_in_scc {
-            let f = callsites[i].caller;
-            if in_scc.contains(&f) {
-                first_call_in_scc -= 1;
-                callsites.swap(i, first_call_in_scc);
-            } else {
-                i += 1;
+impl<'a, 'tcx> Inliner<'a, 'tcx> {
+    fn run_pass(&self, caller_mir: &mut Mir<'tcx>) {
+        // Keep a queue of callsites to try inlining on. We take
+        // advantage of the fact that queries detect cycles here to
+        // allow us to try and fetch the fully optimized MIR of a
+        // call; if it succeeds, we can inline it and we know that
+        // they do not call us.  Otherwise, we just don't try to
+        // inline.
+        //
+        // We use a queue so that we inline "broadly" before we inline
+        // in depth. It is unclear if this is the best heuristic,
+        // really, but that's true of all the heuristics in this
+        // file. =)
+
+        let mut callsites = VecDeque::new();
+
+        // Only do inlining into fn bodies.
+        if let MirSource::Fn(_) = self.source {
+            for (bb, bb_data) in caller_mir.basic_blocks().iter_enumerated() {
+                // Don't inline calls that are in cleanup blocks.
+                if bb_data.is_cleanup { continue; }
+
+                // Only consider direct calls to functions
+                let terminator = bb_data.terminator();
+                if let TerminatorKind::Call {
+                    func: Operand::Constant(ref f), .. } = terminator.kind {
+                    if let ty::TyFnDef(callee_def_id, substs, _) = f.ty.sty {
+                        callsites.push_back(CallSite {
+                            callee: callee_def_id,
+                            substs: substs,
+                            bb: bb,
+                            location: terminator.source_info
+                        });
+                    }
+                }
             }
         }
 
@@ -179,41 +105,27 @@ impl<'a, 'tcx> Inliner<'a, 'tcx> {
 
         loop {
             local_change = false;
-            let mut csi = 0;
-            while csi < callsites.len() {
-                let callsite = callsites[csi];
-                csi += 1;
-
-                let _task = self.tcx.dep_graph.in_task(DepNode::Mir(callsite.caller));
-                self.tcx.dep_graph.write(DepNode::Mir(callsite.caller));
-
-                let callee_mir = {
-                    if let Some(callee_mir) = self.tcx.maybe_item_mir(callsite.callee) {
-                        if !self.should_inline(callsite, &callee_mir) {
-                            continue;
-                        }
+            while let Some(callsite) = callsites.pop_front() {
+                if !self.tcx.is_mir_available(callsite.callee) {
+                    continue;
+                }
 
+                let callee_mir = match ty::queries::optimized_mir::try_get(self.tcx,
+                                                                           callsite.location.span,
+                                                                           callsite.callee) {
+                    Ok(ref callee_mir) if self.should_inline(callsite, callee_mir) => {
                         callee_mir.subst(self.tcx, callsite.substs)
-                    } else {
-                        continue;
                     }
 
-                };
-
-                let mut caller_mir = {
-                    let map = self.tcx.maps.mir.borrow();
-                    let mir = map.get(&callsite.caller).unwrap();
-                    mir.borrow_mut()
+                    _ => continue,
                 };
 
                 let start = caller_mir.basic_blocks().len();
 
-                if !self.inline_call(callsite, &mut caller_mir, callee_mir) {
+                if !self.inline_call(callsite, caller_mir, callee_mir) {
                     continue;
                 }
 
-                inlined_into.insert(callsite.caller);
-
                 // Add callsites from inlined function
                 for (bb, bb_data) in caller_mir.basic_blocks().iter_enumerated().skip(start) {
                     // Only consider direct calls to functions
@@ -223,8 +135,7 @@ impl<'a, 'tcx> Inliner<'a, 'tcx> {
                         if let ty::TyFnDef(callee_def_id, substs, _) = f.ty.sty {
                             // Don't inline the same function multiple times.
                             if callsite.callee != callee_def_id {
-                                callsites.push(CallSite {
-                                    caller: callsite.caller,
+                                callsites.push_back(CallSite {
                                     callee: callee_def_id,
                                     substs: substs,
                                     bb: bb,
@@ -235,13 +146,6 @@ impl<'a, 'tcx> Inliner<'a, 'tcx> {
                     }
                 }
 
-                csi -= 1;
-                if scc.len() == 1 {
-                    callsites.swap_remove(csi);
-                } else {
-                    callsites.remove(csi);
-                }
-
                 local_change = true;
                 changed = true;
             }
@@ -251,27 +155,19 @@ impl<'a, 'tcx> Inliner<'a, 'tcx> {
             }
         }
 
-        // Simplify functions we inlined into.
-        for def_id in inlined_into {
-            let _task = self.tcx.dep_graph.in_task(DepNode::Mir(def_id));
-            self.tcx.dep_graph.write(DepNode::Mir(def_id));
-
-            let mut caller_mir = {
-                let map = self.tcx.maps.mir.borrow();
-                let mir = map.get(&def_id).unwrap();
-                mir.borrow_mut()
-            };
-
-            debug!("Running simplify cfg on {:?}", def_id);
-            CfgSimplifier::new(&mut caller_mir).simplify();
-            remove_dead_blocks(&mut caller_mir);
+        // Simplify if we inlined anything.
+        if changed {
+            debug!("Running simplify cfg on {:?}", self.source);
+            CfgSimplifier::new(caller_mir).simplify();
+            remove_dead_blocks(caller_mir);
         }
-        changed
     }
 
-    fn should_inline(&self, callsite: CallSite<'tcx>,
-                     callee_mir: &'a Mir<'tcx>) -> bool {
-
+    fn should_inline(&self,
+                     callsite: CallSite<'tcx>,
+                     callee_mir: &Mir<'tcx>)
+                     -> bool
+    {
         let tcx = self.tcx;
 
         // Don't inline closures that have captures
@@ -323,8 +219,7 @@ impl<'a, 'tcx> Inliner<'a, 'tcx> {
 
         // FIXME: Give a bonus to functions with only a single caller
 
-        let id = tcx.hir.as_local_node_id(callsite.caller).expect("Caller not local");
-        let param_env = ty::ParameterEnvironment::for_item(tcx, id);
+        let param_env = ty::ParameterEnvironment::for_item(tcx, self.source.item_id());
 
         let mut first_block = true;
         let mut cost = 0;
@@ -423,22 +318,15 @@ impl<'a, 'tcx> Inliner<'a, 'tcx> {
         }
     }
 
-
-    fn inline_call(&self, callsite: CallSite<'tcx>,
-                             caller_mir: &mut Mir<'tcx>, mut callee_mir: Mir<'tcx>) -> bool {
-
-        // Don't inline a function into itself
-        if callsite.caller == callsite.callee { return false; }
-
-        let _task = self.tcx.dep_graph.in_task(DepNode::Mir(callsite.caller));
-
-
+    fn inline_call(&self,
+                   callsite: CallSite<'tcx>,
+                   caller_mir: &mut Mir<'tcx>,
+                   mut callee_mir: Mir<'tcx>) -> bool {
         let terminator = caller_mir[callsite.bb].terminator.take().unwrap();
         match terminator.kind {
             // FIXME: Handle inlining of diverging calls
             TerminatorKind::Call { args, destination: Some(destination), cleanup, .. } => {
-
-                debug!("Inlined {:?} into {:?}", callsite.callee, callsite.caller);
+                debug!("Inlined {:?} into {:?}", callsite.callee, self.source);
 
                 let is_box_free = Some(callsite.callee) == self.tcx.lang_items.box_free_fn();
 

src/librustc_mir/transform/instcombine.rs

@@ -11,32 +11,20 @@
 //! Performs various peephole optimizations.
 
 use rustc::mir::{Location, Lvalue, Mir, Operand, ProjectionElem, Rvalue, Local};
-use rustc::mir::transform::{MirPass, MirSource, Pass};
+use rustc::mir::transform::{MirPass, MirSource};
 use rustc::mir::visit::{MutVisitor, Visitor};
 use rustc::ty::TyCtxt;
 use rustc::util::nodemap::FxHashSet;
 use rustc_data_structures::indexed_vec::Idx;
 use std::mem;
 
-pub struct InstCombine {
-    optimizations: OptimizationList,
-}
+pub struct InstCombine;
 
-impl InstCombine {
-    pub fn new() -> InstCombine {
-        InstCombine {
-            optimizations: OptimizationList::default(),
-        }
-    }
-}
-
-impl Pass for InstCombine {}
-
-impl<'tcx> MirPass<'tcx> for InstCombine {
-    fn run_pass<'a>(&mut self,
-                    tcx: TyCtxt<'a, 'tcx, 'tcx>,
-                    _: MirSource,
-                    mir: &mut Mir<'tcx>) {
+impl MirPass for InstCombine {
+    fn run_pass<'a, 'tcx>(&self,
+                          tcx: TyCtxt<'a, 'tcx, 'tcx>,
+                          _: MirSource,
+                          mir: &mut Mir<'tcx>) {
         // We only run when optimizing MIR (at any level).
         if tcx.sess.opts.debugging_opts.mir_opt_level == 0 {
             return
@@ -45,18 +33,22 @@ impl<'tcx> MirPass<'tcx> for InstCombine {
         // First, find optimization opportunities. This is done in a pre-pass to keep the MIR
         // read-only so that we can do global analyses on the MIR in the process (e.g.
         // `Lvalue::ty()`).
-        {
+        let optimizations = {
             let mut optimization_finder = OptimizationFinder::new(mir, tcx);
             optimization_finder.visit_mir(mir);
-            self.optimizations = optimization_finder.optimizations
-        }
+            optimization_finder.optimizations
+        };
 
         // Then carry out those optimizations.
-        MutVisitor::visit_mir(&mut *self, mir);
+        MutVisitor::visit_mir(&mut InstCombineVisitor { optimizations }, mir);
     }
 }
 
-impl<'tcx> MutVisitor<'tcx> for InstCombine {
+pub struct InstCombineVisitor {
+    optimizations: OptimizationList,
+}
+
+impl<'tcx> MutVisitor<'tcx> for InstCombineVisitor {
     fn visit_rvalue(&mut self, rvalue: &mut Rvalue<'tcx>, location: Location) {
         if self.optimizations.and_stars.remove(&location) {
             debug!("Replacing `&*`: {:?}", rvalue);

src/librustc_mir/transform/mod.rs

@@ -8,6 +8,22 @@
 // option. This file may not be copied, modified, or distributed
 // except according to those terms.
 
+use build;
+use rustc::hir::def_id::{CrateNum, DefId, LOCAL_CRATE};
+use rustc::mir::Mir;
+use rustc::mir::transform::{MirPassIndex, MirSuite, MirSource,
+                            MIR_CONST, MIR_VALIDATED, MIR_OPTIMIZED};
+use rustc::ty::{self, TyCtxt};
+use rustc::ty::maps::Providers;
+use rustc::ty::steal::Steal;
+use rustc::hir;
+use rustc::hir::intravisit::{self, Visitor, NestedVisitorMap};
+use rustc::util::nodemap::DefIdSet;
+use std::rc::Rc;
+use syntax::ast;
+use syntax_pos::{DUMMY_SP, Span};
+use transform;
+
 pub mod simplify_branches;
 pub mod simplify;
 pub mod erase_regions;
@@ -21,3 +37,114 @@ pub mod deaggregator;
 pub mod instcombine;
 pub mod copy_prop;
 pub mod inline;
+
+pub(crate) fn provide(providers: &mut Providers) {
+    self::qualify_consts::provide(providers);
+    *providers = Providers {
+        mir_keys,
+        mir_const,
+        mir_validated,
+        optimized_mir,
+        is_mir_available,
+        ..*providers
+    };
+}
+
+fn is_mir_available<'a, 'tcx>(tcx: TyCtxt<'a, 'tcx, 'tcx>, def_id: DefId) -> bool {
+    tcx.mir_keys(def_id.krate).contains(&def_id)
+}
+
+/// Finds the full set of def-ids within the current crate that have
+/// MIR associated with them.
+fn mir_keys<'a, 'tcx>(tcx: TyCtxt<'a, 'tcx, 'tcx>, krate: CrateNum)
+                      -> Rc<DefIdSet> {
+    assert_eq!(krate, LOCAL_CRATE);
+
+    let mut set = DefIdSet();
+
+    // All body-owners have MIR associated with them.
+    set.extend(tcx.body_owners());
+
+    // Additionally, tuple struct/variant constructors have MIR, but
+    // they don't have a BodyId, so we need to build them separately.
+    struct GatherCtors<'a, 'tcx: 'a> {
+        tcx: TyCtxt<'a, 'tcx, 'tcx>,
+        set: &'a mut DefIdSet,
+    }
+    impl<'a, 'tcx> Visitor<'tcx> for GatherCtors<'a, 'tcx> {
+        fn visit_variant_data(&mut self,
+                              v: &'tcx hir::VariantData,
+                              _: ast::Name,
+                              _: &'tcx hir::Generics,
+                              _: ast::NodeId,
+                              _: Span) {
+            if let hir::VariantData::Tuple(_, node_id) = *v {
+                self.set.insert(self.tcx.hir.local_def_id(node_id));
+            }
+            intravisit::walk_struct_def(self, v)
+        }
+        fn nested_visit_map<'b>(&'b mut self) -> NestedVisitorMap<'b, 'tcx> {
+            NestedVisitorMap::None
+        }
+    }
+    tcx.hir.krate().visit_all_item_likes(&mut GatherCtors {
+        tcx: tcx,
+        set: &mut set,
+    }.as_deep_visitor());
+
+    Rc::new(set)
+}
+
+fn mir_const<'a, 'tcx>(tcx: TyCtxt<'a, 'tcx, 'tcx>, def_id: DefId) -> &'tcx Steal<Mir<'tcx>> {
+    let mut mir = build::mir_build(tcx, def_id);
+    let source = MirSource::from_local_def_id(tcx, def_id);
+    transform::run_suite(tcx, source, MIR_CONST, &mut mir);
+    tcx.alloc_steal_mir(mir)
+}
+
+fn mir_validated<'a, 'tcx>(tcx: TyCtxt<'a, 'tcx, 'tcx>, def_id: DefId) -> &'tcx Steal<Mir<'tcx>> {
+    let source = MirSource::from_local_def_id(tcx, def_id);
+    if let MirSource::Const(_) = source {
+        // Ensure that we compute the `mir_const_qualif` for constants at
+        // this point, before we steal the mir-const result. We don't
+        // directly need the result or `mir_const_qualif`, so we can just force it.
+        ty::queries::mir_const_qualif::force(tcx, DUMMY_SP, def_id);
+    }
+
+    let mut mir = tcx.mir_const(def_id).steal();
+    transform::run_suite(tcx, source, MIR_VALIDATED, &mut mir);
+    tcx.alloc_steal_mir(mir)
+}
+
+fn optimized_mir<'a, 'tcx>(tcx: TyCtxt<'a, 'tcx, 'tcx>, def_id: DefId) -> &'tcx Mir<'tcx> {
+    // Borrowck uses `mir_validated`, so we have to force it to
+    // execute before we can steal.
+    ty::queries::borrowck::force(tcx, DUMMY_SP, def_id);
+
+    let mut mir = tcx.mir_validated(def_id).steal();
+    let source = MirSource::from_local_def_id(tcx, def_id);
+    transform::run_suite(tcx, source, MIR_OPTIMIZED, &mut mir);
+    tcx.alloc_mir(mir)
+}
+
+fn run_suite<'a, 'tcx>(tcx: TyCtxt<'a, 'tcx, 'tcx>,
+                       source: MirSource,
+                       suite: MirSuite,
+                       mir: &mut Mir<'tcx>)
+{
+    let passes = tcx.mir_passes.passes(suite);
+
+    for (pass, index) in passes.iter().zip(0..) {
+        let pass_num = MirPassIndex(index);
+
+        for hook in tcx.mir_passes.hooks() {
+            hook.on_mir_pass(tcx, suite, pass_num, &pass.name(), source, &mir, false);
+        }
+
+        pass.run_pass(tcx, source, mir);
+
+        for hook in tcx.mir_passes.hooks() {
+            hook.on_mir_pass(tcx, suite, pass_num, &pass.name(), source, &mir, true);
+        }
+    }
+}

src/librustc_mir/transform/no_landing_pads.rs

@@ -14,10 +14,25 @@
 use rustc::ty::TyCtxt;
 use rustc::mir::*;
 use rustc::mir::visit::MutVisitor;
-use rustc::mir::transform::{Pass, MirPass, MirSource};
+use rustc::mir::transform::{MirPass, MirSource};
 
 pub struct NoLandingPads;
 
+impl MirPass for NoLandingPads {
+    fn run_pass<'a, 'tcx>(&self,
+                          tcx: TyCtxt<'a, 'tcx, 'tcx>,
+                          _: MirSource,
+                          mir: &mut Mir<'tcx>) {
+        no_landing_pads(tcx, mir)
+    }
+}
+
+pub fn no_landing_pads<'a, 'tcx>(tcx: TyCtxt<'a, 'tcx, 'tcx>, mir: &mut Mir<'tcx>) {
+    if tcx.sess.no_landing_pads() {
+        NoLandingPads.visit_mir(mir);
+    }
+}
+
 impl<'tcx> MutVisitor<'tcx> for NoLandingPads {
     fn visit_terminator(&mut self,
                         bb: BasicBlock,
@@ -41,18 +56,3 @@ impl<'tcx> MutVisitor<'tcx> for NoLandingPads {
         self.super_terminator(bb, terminator, location);
     }
 }
-
-pub fn no_landing_pads<'a, 'tcx>(tcx: TyCtxt<'a, 'tcx, 'tcx>, mir: &mut Mir<'tcx>) {
-    if tcx.sess.no_landing_pads() {
-        NoLandingPads.visit_mir(mir);
-    }
-}
-
-impl<'tcx> MirPass<'tcx> for NoLandingPads {
-    fn run_pass<'a>(&mut self, tcx: TyCtxt<'a, 'tcx, 'tcx>,
-                    _: MirSource, mir: &mut Mir<'tcx>) {
-        no_landing_pads(tcx, mir)
-    }
-}
-
-impl Pass for NoLandingPads {}

src/librustc_mir/transform/qualify_consts.rs

@@ -16,7 +16,6 @@
 
 use rustc_data_structures::bitvec::BitVector;
 use rustc_data_structures::indexed_vec::{IndexVec, Idx};
-use rustc::dep_graph::DepNode;
 use rustc::hir;
 use rustc::hir::map as hir_map;
 use rustc::hir::def_id::DefId;
@@ -27,7 +26,7 @@ use rustc::ty::cast::CastTy;
 use rustc::ty::maps::Providers;
 use rustc::mir::*;
 use rustc::mir::traversal::ReversePostorder;
-use rustc::mir::transform::{Pass, MirMapPass, MirPassHook, MirSource};
+use rustc::mir::transform::{MirPass, MirSource};
 use rustc::mir::visit::{LvalueContext, Visitor};
 use rustc::middle::lang_items;
 use syntax::abi::Abi;
@@ -919,13 +918,21 @@ impl<'a, 'tcx> Visitor<'tcx> for Qualifier<'a, 'tcx, 'tcx> {
 }
 
 pub fn provide(providers: &mut Providers) {
-    providers.mir_const_qualif = qualify_const_item;
+    *providers = Providers {
+        mir_const_qualif,
+        ..*providers
+    };
 }
 
-fn qualify_const_item<'a, 'tcx>(tcx: TyCtxt<'a, 'tcx, 'tcx>,
-                                def_id: DefId)
-                                -> u8 {
-    let mir = &tcx.item_mir(def_id);
+fn mir_const_qualif<'a, 'tcx>(tcx: TyCtxt<'a, 'tcx, 'tcx>,
+                              def_id: DefId)
+                              -> u8 {
+    // NB: This `borrow()` is guaranteed to be valid (i.e., the value
+    // cannot yet be stolen), because `mir_validated()`, which steals
+    // from `mir_const(), forces this query to execute before
+    // performing the steal.
+    let mir = &tcx.mir_const(def_id).borrow();
+
     if mir.return_ty.references_error() {
         return Qualif::NOT_CONST.bits();
     }
@@ -939,45 +946,11 @@ fn qualify_const_item<'a, 'tcx>(tcx: TyCtxt<'a, 'tcx, 'tcx>,
 
 pub struct QualifyAndPromoteConstants;
 
-impl Pass for QualifyAndPromoteConstants {}
-
-impl<'tcx> MirMapPass<'tcx> for QualifyAndPromoteConstants {
-    fn run_pass<'a>(&mut self,
-                    tcx: TyCtxt<'a, 'tcx, 'tcx>,
-                    hooks: &mut [Box<for<'s> MirPassHook<'s>>])
-    {
-        let def_ids = tcx.maps.mir.borrow().keys();
-        for def_id in def_ids {
-            if !def_id.is_local() {
-                continue;
-            }
-
-            let _task = tcx.dep_graph.in_task(DepNode::Mir(def_id));
-            let id = tcx.hir.as_local_node_id(def_id).unwrap();
-            let src = MirSource::from_node(tcx, id);
-
-            if let MirSource::Const(_) = src {
-                tcx.mir_const_qualif(def_id);
-                continue;
-            }
-
-            let mir = &mut tcx.maps.mir.borrow()[&def_id].borrow_mut();
-            tcx.dep_graph.write(DepNode::Mir(def_id));
-
-            for hook in &mut *hooks {
-                hook.on_mir_pass(tcx, src, mir, self, false);
-            }
-            self.run_pass(tcx, src, mir);
-            for hook in &mut *hooks {
-                hook.on_mir_pass(tcx, src, mir, self, true);
-            }
-        }
-    }
-}
-
-impl<'tcx> QualifyAndPromoteConstants {
-    fn run_pass<'a>(&mut self, tcx: TyCtxt<'a, 'tcx, 'tcx>,
-                    src: MirSource, mir: &mut Mir<'tcx>) {
+impl MirPass for QualifyAndPromoteConstants {
+    fn run_pass<'a, 'tcx>(&self,
+                          tcx: TyCtxt<'a, 'tcx, 'tcx>,
+                          src: MirSource,
+                          mir: &mut Mir<'tcx>) {
         let id = src.item_id();
         let def_id = tcx.hir.local_def_id(id);
         let mode = match src {

src/librustc_mir/transform/simplify.rs

@@ -41,15 +41,15 @@ use rustc_data_structures::bitvec::BitVector;
 use rustc_data_structures::indexed_vec::{Idx, IndexVec};
 use rustc::ty::TyCtxt;
 use rustc::mir::*;
-use rustc::mir::transform::{MirPass, MirSource, Pass};
+use rustc::mir::transform::{MirPass, MirSource};
 use rustc::mir::visit::{MutVisitor, Visitor, LvalueContext};
-use std::fmt;
+use std::borrow::Cow;
 
-pub struct SimplifyCfg<'a> { label: &'a str }
+pub struct SimplifyCfg { label: String }
 
-impl<'a> SimplifyCfg<'a> {
-    pub fn new(label: &'a str) -> Self {
-        SimplifyCfg { label: label }
+impl SimplifyCfg {
+    pub fn new(label: &str) -> Self {
+        SimplifyCfg { label: format!("SimplifyCfg-{}", label) }
     }
 }
 
@@ -61,22 +61,20 @@ pub fn simplify_cfg(mir: &mut Mir) {
     mir.basic_blocks_mut().raw.shrink_to_fit();
 }
 
-impl<'l, 'tcx> MirPass<'tcx> for SimplifyCfg<'l> {
-    fn run_pass<'a>(&mut self, _tcx: TyCtxt<'a, 'tcx, 'tcx>, _src: MirSource, mir: &mut Mir<'tcx>) {
+impl MirPass for SimplifyCfg {
+    fn name<'a>(&'a self) -> Cow<'a, str> {
+        Cow::Borrowed(&self.label)
+    }
+
+    fn run_pass<'a, 'tcx>(&self,
+                          _tcx: TyCtxt<'a, 'tcx, 'tcx>,
+                          _src: MirSource,
+                          mir: &mut Mir<'tcx>) {
         debug!("SimplifyCfg({:?}) - simplifying {:?}", self.label, mir);
         simplify_cfg(mir);
     }
 }
 
-impl<'l> Pass for SimplifyCfg<'l> {
-    fn disambiguator<'a>(&'a self) -> Option<Box<fmt::Display+'a>> {
-        Some(Box::new(self.label))
-    }
-
-    // avoid calling `type_name` - it contains `<'static>`
-    fn name(&self) -> ::std::borrow::Cow<'static, str> { "SimplifyCfg".into() }
-}
-
 pub struct CfgSimplifier<'a, 'tcx: 'a> {
     basic_blocks: &'a mut IndexVec<BasicBlock, BasicBlockData<'tcx>>,
     pred_count: IndexVec<BasicBlock, u32>
@@ -315,12 +313,11 @@ pub fn remove_dead_blocks(mir: &mut Mir) {
 
 pub struct SimplifyLocals;
 
-impl Pass for SimplifyLocals {
-    fn name(&self) -> ::std::borrow::Cow<'static, str> { "SimplifyLocals".into() }
-}
-
-impl<'tcx> MirPass<'tcx> for SimplifyLocals {
-    fn run_pass<'a>(&mut self, _: TyCtxt<'a, 'tcx, 'tcx>, _: MirSource, mir: &mut Mir<'tcx>) {
+impl MirPass for SimplifyLocals {
+    fn run_pass<'a, 'tcx>(&self,
+                          _: TyCtxt<'a, 'tcx, 'tcx>,
+                          _: MirSource,
+                          mir: &mut Mir<'tcx>) {
         let mut marker = DeclMarker { locals: BitVector::new(mir.local_decls.len()) };
         marker.visit_mir(mir);
         // Return pointer and arguments are always live

src/librustc_mir/transform/simplify_branches.rs

@@ -12,21 +12,28 @@
 
 use rustc::ty::TyCtxt;
 use rustc::middle::const_val::ConstVal;
-use rustc::mir::transform::{MirPass, MirSource, Pass};
+use rustc::mir::transform::{MirPass, MirSource};
 use rustc::mir::*;
 
-use std::fmt;
+use std::borrow::Cow;
 
-pub struct SimplifyBranches<'a> { label: &'a str }
+pub struct SimplifyBranches { label: String }
 
-impl<'a> SimplifyBranches<'a> {
-    pub fn new(label: &'a str) -> Self {
-        SimplifyBranches { label: label }
+impl SimplifyBranches {
+    pub fn new(label: &str) -> Self {
+        SimplifyBranches { label: format!("SimplifyBranches-{}", label) }
     }
 }
 
-impl<'l, 'tcx> MirPass<'tcx> for SimplifyBranches<'l> {
-    fn run_pass<'a>(&mut self, _tcx: TyCtxt<'a, 'tcx, 'tcx>, _src: MirSource, mir: &mut Mir<'tcx>) {
+impl MirPass for SimplifyBranches {
+    fn name<'a>(&'a self) -> Cow<'a, str> {
+        Cow::Borrowed(&self.label)
+    }
+
+    fn run_pass<'a, 'tcx>(&self,
+                          _tcx: TyCtxt<'a, 'tcx, 'tcx>,
+                          _src: MirSource,
+                          mir: &mut Mir<'tcx>) {
         for block in mir.basic_blocks_mut() {
             let terminator = block.terminator_mut();
             terminator.kind = match terminator.kind {
@@ -60,11 +67,3 @@ impl<'l, 'tcx> MirPass<'tcx> for SimplifyBranches<'l> {
     }
 }
 
-impl<'l> Pass for SimplifyBranches<'l> {
-    fn disambiguator<'a>(&'a self) -> Option<Box<fmt::Display+'a>> {
-        Some(Box::new(self.label))
-    }
-
-    // avoid calling `type_name` - it contains `<'static>`
-    fn name(&self) -> ::std::borrow::Cow<'static, str> { "SimplifyBranches".into() }
-}

src/librustc_mir/transform/type_check.rs

@@ -18,7 +18,7 @@ use rustc::ty::{self, Ty, TyCtxt, TypeVariants};
 use rustc::middle::const_val::ConstVal;
 use rustc::mir::*;
 use rustc::mir::tcx::LvalueTy;
-use rustc::mir::transform::{MirPass, MirSource, Pass};
+use rustc::mir::transform::{MirPass, MirSource};
 use rustc::mir::visit::Visitor;
 use std::fmt;
 use syntax::ast;
@@ -737,9 +737,11 @@ impl TypeckMir {
     }
 }
 
-impl<'tcx> MirPass<'tcx> for TypeckMir {
-    fn run_pass<'a>(&mut self, tcx: TyCtxt<'a, 'tcx, 'tcx>,
-                    src: MirSource, mir: &mut Mir<'tcx>) {
+impl MirPass for TypeckMir {
+    fn run_pass<'a, 'tcx>(&self,
+                          tcx: TyCtxt<'a, 'tcx, 'tcx>,
+                          src: MirSource,
+                          mir: &mut Mir<'tcx>) {
         let item_id = src.item_id();
         let def_id = tcx.hir.local_def_id(item_id);
         debug!("run_pass: {}", tcx.item_path_str(def_id));
@@ -765,6 +767,3 @@ impl<'tcx> MirPass<'tcx> for TypeckMir {
         });
     }
 }
-
-impl Pass for TypeckMir {
-}

src/librustc_mir/util/graphviz.rs

@@ -18,16 +18,18 @@ use syntax::ast::NodeId;
 
 use rustc_data_structures::indexed_vec::Idx;
 
+use super::pretty::dump_mir_def_ids;
+
 /// Write a graphviz DOT graph of a list of MIRs.
-pub fn write_mir_graphviz<'a, 'b, 'tcx, W, I>(tcx: TyCtxt<'b, 'tcx, 'tcx>,
-                                              iter: I,
-                                              w: &mut W)
-                                              -> io::Result<()>
-    where W: Write, I: Iterator<Item=DefId>
+pub fn write_mir_graphviz<'a, 'tcx, W>(tcx: TyCtxt<'a, 'tcx, 'tcx>,
+                                       single: Option<DefId>,
+                                       w: &mut W)
+                                       -> io::Result<()>
+    where W: Write
 {
-    for def_id in iter {
+    for def_id in dump_mir_def_ids(tcx, single) {
         let nodeid = tcx.hir.as_local_node_id(def_id).unwrap();
-        let mir = &tcx.item_mir(def_id);
+        let mir = &tcx.optimized_mir(def_id);
 
         writeln!(w, "digraph Mir_{} {{", nodeid)?;
 

src/librustc_mir/util/mod.rs

@@ -15,6 +15,6 @@ pub mod patch;
 mod graphviz;
 mod pretty;
 
-pub use self::pretty::{dump_mir, write_mir_pretty};
+pub use self::pretty::{dump_enabled, dump_mir, write_mir_pretty};
 pub use self::graphviz::{write_mir_graphviz};
 pub use self::graphviz::write_node_label as write_graphviz_node_label;

src/librustc_mir/util/pretty.rs

@@ -9,9 +9,9 @@
 // except according to those terms.
 
 use rustc::hir;
-use rustc::hir::def_id::DefId;
+use rustc::hir::def_id::{DefId, LOCAL_CRATE};
 use rustc::mir::*;
-use rustc::mir::transform::MirSource;
+use rustc::mir::transform::{MirSuite, MirPassIndex, MirSource};
 use rustc::ty::TyCtxt;
 use rustc_data_structures::fx::FxHashMap;
 use rustc_data_structures::indexed_vec::{Idx};
@@ -28,7 +28,7 @@ const ALIGN: usize = 40;
 /// representation of the mir into:
 ///
 /// ```text
-/// rustc.node<node_id>.<pass_name>.<disambiguator>
+/// rustc.node<node_id>.<pass_num>.<pass_name>.<disambiguator>
 /// ```
 ///
 /// Output from this function is controlled by passing `-Z dump-mir=<filter>`,
@@ -39,64 +39,95 @@ const ALIGN: usize = 40;
 ///   that can appear in the pass-name or the `item_path_str` for the given
 ///   node-id. If any one of the substrings match, the data is dumped out.
 pub fn dump_mir<'a, 'tcx>(tcx: TyCtxt<'a, 'tcx, 'tcx>,
+                          pass_num: Option<(MirSuite, MirPassIndex)>,
                           pass_name: &str,
                           disambiguator: &Display,
-                          src: MirSource,
+                          source: MirSource,
                           mir: &Mir<'tcx>) {
-    let filters = match tcx.sess.opts.debugging_opts.dump_mir {
-        None => return,
-        Some(ref filters) => filters,
-    };
-    let node_id = src.item_id();
-    let node_path = tcx.item_path_str(tcx.hir.local_def_id(node_id));
-    let is_matched =
-        filters.split("&")
-               .any(|filter| {
-                   filter == "all" ||
-                       pass_name.contains(filter) ||
-                       node_path.contains(filter)
-               });
-    if !is_matched {
+    if !dump_enabled(tcx, pass_name, source) {
         return;
     }
 
-    let promotion_id = match src {
+    let node_path = tcx.item_path_str(tcx.hir.local_def_id(source.item_id()));
+    dump_matched_mir_node(tcx, pass_num, pass_name, &node_path,
+                          disambiguator, source, mir);
+    for (index, promoted_mir) in mir.promoted.iter_enumerated() {
+        let promoted_source = MirSource::Promoted(source.item_id(), index);
+        dump_matched_mir_node(tcx, pass_num, pass_name, &node_path, disambiguator,
+                              promoted_source, promoted_mir);
+    }
+}
+
+pub fn dump_enabled<'a, 'tcx>(tcx: TyCtxt<'a, 'tcx, 'tcx>,
+                              pass_name: &str,
+                              source: MirSource)
+                              -> bool {
+    let filters = match tcx.sess.opts.debugging_opts.dump_mir {
+        None => return false,
+        Some(ref filters) => filters,
+    };
+    let node_id = source.item_id();
+    let node_path = tcx.item_path_str(tcx.hir.local_def_id(node_id));
+    filters.split("&")
+           .any(|filter| {
+               filter == "all" ||
+                   pass_name.contains(filter) ||
+                   node_path.contains(filter)
+           })
+}
+
+fn dump_matched_mir_node<'a, 'tcx>(tcx: TyCtxt<'a, 'tcx, 'tcx>,
+                                   pass_num: Option<(MirSuite, MirPassIndex)>,
+                                   pass_name: &str,
+                                   node_path: &str,
+                                   disambiguator: &Display,
+                                   source: MirSource,
+                                   mir: &Mir<'tcx>) {
+    let promotion_id = match source {
         MirSource::Promoted(_, id) => format!("-{:?}", id),
         _ => String::new()
     };
 
+    let pass_num = if tcx.sess.opts.debugging_opts.dump_mir_exclude_pass_number {
+        format!("")
+    } else {
+        match pass_num {
+            None => format!(".-------"),
+            Some((suite, pass_num)) => format!(".{:03}-{:03}", suite.0, pass_num.0),
+        }
+    };
+
     let mut file_path = PathBuf::new();
     if let Some(ref file_dir) = tcx.sess.opts.debugging_opts.dump_mir_dir {
         let p = Path::new(file_dir);
         file_path.push(p);
     };
-    let file_name = format!("rustc.node{}{}.{}.{}.mir",
-                            node_id, promotion_id, pass_name, disambiguator);
+    let file_name = format!("rustc.node{}{}{}.{}.{}.mir",
+                            source.item_id(), promotion_id, pass_num, pass_name, disambiguator);
     file_path.push(&file_name);
     let _ = fs::File::create(&file_path).and_then(|mut file| {
         writeln!(file, "// MIR for `{}`", node_path)?;
-        writeln!(file, "// node_id = {}", node_id)?;
+        writeln!(file, "// source = {:?}", source)?;
         writeln!(file, "// pass_name = {}", pass_name)?;
         writeln!(file, "// disambiguator = {}", disambiguator)?;
         writeln!(file, "")?;
-        write_mir_fn(tcx, src, mir, &mut file)?;
+        write_mir_fn(tcx, source, mir, &mut file)?;
         Ok(())
     });
 }
 
 /// Write out a human-readable textual representation for the given MIR.
-pub fn write_mir_pretty<'a, 'b, 'tcx, I>(tcx: TyCtxt<'b, 'tcx, 'tcx>,
-                                         iter: I,
-                                         w: &mut Write)
-                                         -> io::Result<()>
-    where I: Iterator<Item=DefId>, 'tcx: 'a
+pub fn write_mir_pretty<'a, 'tcx>(tcx: TyCtxt<'a, 'tcx, 'tcx>,
+                                  single: Option<DefId>,
+                                  w: &mut Write)
+                                  -> io::Result<()>
 {
     writeln!(w, "// WARNING: This output format is intended for human consumers only")?;
     writeln!(w, "// and is subject to change without notice. Knock yourself out.")?;
 
     let mut first = true;
-    for def_id in iter.filter(DefId::is_local) {
-        let mir = &tcx.item_mir(def_id);
+    for def_id in dump_mir_def_ids(tcx, single) {
+        let mir = &tcx.optimized_mir(def_id);
 
         if first {
             first = false;
@@ -312,3 +343,11 @@ fn write_temp_decls(mir: &Mir, w: &mut Write) -> io::Result<()> {
 
     Ok(())
 }
+
+pub fn dump_mir_def_ids(tcx: TyCtxt, single: Option<DefId>) -> Vec<DefId> {
+    if let Some(i) = single {
+        vec![i]
+    } else {
+        tcx.mir_keys(LOCAL_CRATE).iter().cloned().collect()
+    }
+}

src/librustc_passes/mir_stats.rs

@@ -13,6 +13,7 @@
 // completely accurate (some things might be counted twice, others missed).
 
 use rustc_const_math::{ConstUsize};
+use rustc::hir::def_id::LOCAL_CRATE;
 use rustc::middle::const_val::{ConstVal};
 use rustc::mir::{AggregateKind, AssertMessage, BasicBlock, BasicBlockData};
 use rustc::mir::{Constant, Literal, Location, LocalDecl};
@@ -44,10 +45,9 @@ pub fn print_mir_stats<'tcx, 'a>(tcx: TyCtxt<'a, 'tcx, 'tcx>, title: &str) {
     // For debugging instrumentation like this, we don't need to worry
     // about maintaining the dep graph.
     let _ignore = tcx.dep_graph.in_ignore();
-    let mir_map = tcx.maps.mir.borrow();
-    for def_id in mir_map.keys() {
-        let mir = mir_map.get(&def_id).unwrap();
-        collector.visit_mir(&mir.borrow());
+    for &def_id in tcx.mir_keys(LOCAL_CRATE).iter() {
+        let mir = tcx.optimized_mir(def_id);
+        collector.visit_mir(&mir);
     }
     collector.print(title);
 }

src/librustc_trans/collector.rs

@@ -659,7 +659,7 @@ fn should_trans_locally<'a, 'tcx>(tcx: TyCtxt<'a, 'tcx, 'tcx>, instance: &Instan
                 // in this crate
                 false
             } else {
-                if !tcx.is_item_mir_available(def_id) {
+                if !tcx.is_mir_available(def_id) {
                     bug!("Cannot create local trans-item for {:?}", def_id)
                 }
                 true

src/librustc_typeck/check/mod.rs

@@ -640,9 +640,9 @@ pub fn check_item_bodies<'a, 'tcx>(tcx: TyCtxt<'a, 'tcx, 'tcx>) -> CompileResult
 fn typeck_item_bodies<'a, 'tcx>(tcx: TyCtxt<'a, 'tcx, 'tcx>, crate_num: CrateNum) -> CompileResult {
     debug_assert!(crate_num == LOCAL_CRATE);
     tcx.sess.track_errors(|| {
-        tcx.visit_all_bodies_in_krate(|body_owner_def_id, _body_id| {
+        for body_owner_def_id in tcx.body_owners() {
             tcx.typeck_tables_of(body_owner_def_id);
-        });
+        }
     })
 }
 

src/test/mir-opt/basic_assignment.rs

@@ -36,7 +36,7 @@ fn main() {
 }
 
 // END RUST SOURCE
-// START rustc.node4.SimplifyCfg.initial-after.mir
+// START rustc.node4.SimplifyCfg-initial.after.mir
 //     bb0: {
 //         StorageLive(_1);
 //         _1 = const false;
@@ -82,4 +82,4 @@ fn main() {
 //         StorageDead(_1);
 //         return;
 //     }
-// END rustc.node4.SimplifyCfg.initial-after.mir
+// END rustc.node4.SimplifyCfg-initial.after.mir

src/test/mir-opt/issue-38669.rs

@@ -21,7 +21,7 @@ fn main() {
 }
 
 // END RUST SOURCE
-// START rustc.node4.SimplifyCfg.initial-after.mir
+// START rustc.node4.SimplifyCfg-initial.after.mir
 //     bb0: {
 //         StorageLive(_1);
 //         _1 = const false;
@@ -48,4 +48,4 @@ fn main() {
 //         _2 = ();
 //         goto -> bb1;
 //     }
-// END rustc.node4.SimplifyCfg.initial-after.mir
+// END rustc.node4.SimplifyCfg-initial.after.mir

src/test/mir-opt/simplify_if.rs

@@ -15,13 +15,13 @@ fn main() {
 }
 
 // END RUST SOURCE
-// START rustc.node4.SimplifyBranches.initial-before.mir
+// START rustc.node4.SimplifyBranches-initial.before.mir
 // bb0: {
 //     switchInt(const false) -> [0u8: bb2, otherwise: bb1];
 // }
-// END rustc.node4.SimplifyBranches.initial-before.mir
-// START rustc.node4.SimplifyBranches.initial-after.mir
+// END rustc.node4.SimplifyBranches-initial.before.mir
+// START rustc.node4.SimplifyBranches-initial.after.mir
 // bb0: {
 //     goto -> bb2;
 // }
-// END rustc.node4.SimplifyBranches.initial-after.mir
+// END rustc.node4.SimplifyBranches-initial.after.mir

src/tools/compiletest/src/runtest.rs

@@ -1402,18 +1402,16 @@ actual:\n\
                 }
             }
             MirOpt => {
-                args.extend(["-Z",
-                             "dump-mir=all",
-                             "-Z",
-                             "mir-opt-level=3",
-                             "-Z"]
+                args.extend(["-Zdump-mir=all",
+                             "-Zmir-opt-level=3",
+                             "-Zdump-mir-exclude-pass-number"]
                             .iter()
                             .map(|s| s.to_string()));
 
 
                 let mir_dump_dir = self.get_mir_dump_dir();
                 create_dir_all(mir_dump_dir.as_path()).unwrap();
-                let mut dir_opt = "dump-mir-dir=".to_string();
+                let mut dir_opt = "-Zdump-mir-dir=".to_string();
                 dir_opt.push_str(mir_dump_dir.to_str().unwrap());
                 debug!("dir_opt: {:?}", dir_opt);