bjoernager/rust
bjoernager/rust
1
Fork 0
Code Activity

Auto merge of #47046 - Manishearth:intra-doc-links, r=eddyb,GuillaumeGomez,QuietMisdreavus,Manishearth

Browse source 
Implement RFC 1946 - intra-rustdoc links

https://github.com/rust-lang/rfcs/pull/1946 https://github.com/rust-lang/rust/issues/43466

Note for reviewers: The plain line counts are a little inflated because of how the markdown link parsing was done. [Read the file diff with "whitespace only" changes removed](https://github.com/rust-lang/rust/pull/47046/files?w=1) to get a better view of what actually changed there.

This pulls the name/path resolution mechanisms out of the compiler and runs it on the markdown in a crate's docs, so that links can be made to `SomeStruct` directly rather than finding the folder path to `struct.SomeStruct.html`. Check the `src/test/rustdoc/intra-paths.rs` test in this PR for a demo. The change was... a little invasive, but unlocks a really powerful mechanism for writing documentation that doesn't care about where an item was written to on the hard disk.

Items included:

 - [x] Make work with the hoedown renderer
 - [x] Handle relative paths
 - [x] Parse out the "path ambiguities" qualifiers (`[crate foo]`, `[struct Foo]`, `[foo()]`, `[static FOO]`, `[foo!]`, etc)
 - [x] Resolve foreign macros
 - [x] Resolve local macros
 - [x] Handle the use of inner/outer attributes giving different resolution scopes (handling for non-modules pushed to different PR)

Items not included:

 - [ ] Make sure cross-crate inlining works (blocked on refactor described in https://github.com/rust-lang/rust/pull/47046#issuecomment-354824520)
 - [ ] Implied Shortcut Reference Links (where just doing `[::std::iter::Iterator][]` without a reference anchor will resolve using the reference name rather than the link target) (requires modifying the markdown parser - blocked on Hoedown/Pulldown switch and https://github.com/google/pulldown-cmark/issues/121)
 - [ ] Handle enum variants and UFCS methods (Enum variants link to the enum page, associated methods don't link at all)
 - [ ] Emit more warnings/errors when things fail to resolve (linking to a value-namespaced item without a qualifier will emit an error, otherwise the link is just treated as a url, not a rust path)
 - [ ] Give better spans for resolution errors (currently the span for the first doc comment is used)
 - [ ] Check for inner doc comments on things that aren't modules

I'm making the PR, but it should be noted that most of the work was done by Misdreavus 😄

(Editor's note: This has become a lie, check that commit log, Manish did a ton of work after this PR was opened `>_>`)
This commit is contained in:
bors 2018-01-23 07:38:53 +00:00
commit 48a7ea9c40
17 changed files with 1003 additions and 272 deletions
Download patch file Download diff file Expand all files Collapse all files

16
src/librustc/hir/lowering.rs
View file

@ -151,6 +151,11 @@ pub trait Resolver {
/// We must keep the set of definitions up to date as we add nodes that weren't in the AST.
/// This should only return `None` during testing.
fn definitions(&mut self) -> &mut Definitions;
/// Given suffix ["b","c","d"], creates a HIR path for `[::crate_root]::b::c::d` and resolves
/// it based on `is_value`.
fn resolve_str_path(&mut self, span: Span, crate_root: Option<&str>,
components: &[&str], is_value: bool) -> hir::Path;
}
#[derive(Clone, Copy, Debug)]
@ -3625,16 +3630,7 @@ impl<'a> LoweringContext<'a> {
/// `fld.cx.use_std`, and `::core::b::c::d` otherwise.
/// The path is also resolved according to `is_value`.
fn std_path(&mut self, span: Span, components: &[&str], is_value: bool) -> hir::Path {
let mut path = hir::Path {
span,
def: Def::Err,
segments: iter::once(keywords::CrateRoot.name()).chain({
self.crate_root.into_iter().chain(components.iter().cloned()).map(Symbol::intern)
}).map(hir::PathSegment::from_name).collect(),
};
self.resolver.resolve_hir_path(&mut path, is_value);
path
self.resolver.resolve_str_path(span, self.crate_root, components, is_value)
}
fn signal_block_expr(&mut self,

83
src/librustc_driver/driver.rs
View file

@ -28,7 +28,7 @@ use rustc::util::common::{ErrorReported, time};
use rustc_allocator as allocator;
use rustc_borrowck as borrowck;
use rustc_incremental;
use rustc_resolve::{MakeGlobMap, Resolver};
use rustc_resolve::{MakeGlobMap, Resolver, ResolverArenas};
use rustc_metadata::creader::CrateLoader;
use rustc_metadata::cstore::{self, CStore};
use rustc_trans_utils::trans_crate::TransCrate;
@ -139,6 +139,7 @@ pub fn compile_input(trans: Box<TransCrate>,
let crate_name =
::rustc_trans_utils::link::find_crate_name(Some(sess), &krate.attrs, input);
let ExpansionResult { expanded_crate, defs, analysis, resolutions, mut hir_forest } = {
phase_2_configure_and_expand(
sess,
@ -562,6 +563,12 @@ pub struct ExpansionResult {
pub hir_forest: hir_map::Forest,
}
pub struct InnerExpansionResult<'a> {
pub expanded_crate: ast::Crate,
pub resolver: Resolver<'a>,
pub hir_forest: hir_map::Forest,
}
/// Run the "early phases" of the compiler: initial `cfg` processing,
/// loading compiler plugins (including those from `addl_plugins`),
/// syntax expansion, secondary `cfg` expansion, synthesis of a test
@ -578,6 +585,55 @@ pub fn phase_2_configure_and_expand<F>(sess: &Session,
make_glob_map: MakeGlobMap,
after_expand: F)
-> Result<ExpansionResult, CompileIncomplete>
where F: FnOnce(&ast::Crate) -> CompileResult {
// Currently, we ignore the name resolution data structures for the purposes of dependency
// tracking. Instead we will run name resolution and include its output in the hash of each
// item, much like we do for macro expansion. In other words, the hash reflects not just
// its contents but the results of name resolution on those contents. Hopefully we'll push
// this back at some point.
let mut crate_loader = CrateLoader::new(sess, &cstore, &crate_name);
let resolver_arenas = Resolver::arenas();
let result = phase_2_configure_and_expand_inner(sess, cstore, krate, registry, crate_name,
addl_plugins, make_glob_map, &resolver_arenas,
&mut crate_loader, after_expand);
match result {
Ok(InnerExpansionResult {expanded_crate, resolver, hir_forest}) => {
Ok(ExpansionResult {
expanded_crate,
defs: resolver.definitions,
hir_forest,
resolutions: Resolutions {
freevars: resolver.freevars,
export_map: resolver.export_map,
trait_map: resolver.trait_map,
maybe_unused_trait_imports: resolver.maybe_unused_trait_imports,
maybe_unused_extern_crates: resolver.maybe_unused_extern_crates,
},
analysis: ty::CrateAnalysis {
access_levels: Rc::new(AccessLevels::default()),
name: crate_name.to_string(),
glob_map: if resolver.make_glob_map { Some(resolver.glob_map) } else { None },
},
})
}
Err(x) => Err(x)
}
}
/// Same as phase_2_configure_and_expand, but doesn't let you keep the resolver
/// around
pub fn phase_2_configure_and_expand_inner<'a, F>(sess: &'a Session,
cstore: &'a CStore,
krate: ast::Crate,
registry: Option<Registry>,
crate_name: &str,
addl_plugins: Option<Vec<String>>,
make_glob_map: MakeGlobMap,
resolver_arenas: &'a ResolverArenas<'a>,
crate_loader: &'a mut CrateLoader,
after_expand: F)
-> Result<InnerExpansionResult<'a>, CompileIncomplete>
where F: FnOnce(&ast::Crate) -> CompileResult,
{
let time_passes = sess.time_passes();
@ -666,19 +722,12 @@ pub fn phase_2_configure_and_expand<F>(sess: &Session,
return Err(CompileIncomplete::Stopped);
}
// Currently, we ignore the name resolution data structures for the purposes of dependency
// tracking. Instead we will run name resolution and include its output in the hash of each
// item, much like we do for macro expansion. In other words, the hash reflects not just
// its contents but the results of name resolution on those contents. Hopefully we'll push
// this back at some point.
let mut crate_loader = CrateLoader::new(sess, &cstore, crate_name);
let resolver_arenas = Resolver::arenas();
let mut resolver = Resolver::new(sess,
cstore,
&krate,
crate_name,
make_glob_map,
&mut crate_loader,
crate_loader,
&resolver_arenas);
resolver.whitelisted_legacy_custom_derives = whitelisted_legacy_custom_derives;
syntax_ext::register_builtins(&mut resolver, syntax_exts, sess.features.borrow().quote);
@ -855,21 +904,9 @@ pub fn phase_2_configure_and_expand<F>(sess: &Session,
syntax::ext::hygiene::clear_markings();
}
Ok(ExpansionResult {
Ok(InnerExpansionResult {
expanded_crate: krate,
defs: resolver.definitions,
analysis: ty::CrateAnalysis {
access_levels: Rc::new(AccessLevels::default()),
name: crate_name.to_string(),
glob_map: if resolver.make_glob_map { Some(resolver.glob_map) } else { None },
},
resolutions: Resolutions {
freevars: resolver.freevars,
export_map: resolver.export_map,
trait_map: resolver.trait_map,
maybe_unused_trait_imports: resolver.maybe_unused_trait_imports,
maybe_unused_extern_crates: resolver.maybe_unused_extern_crates,
},
resolver,
hir_forest,
})
}

89
src/librustc_resolve/lib.rs
View file

@ -67,6 +67,7 @@ use std::cell::{Cell, RefCell};
use std::cmp;
use std::collections::BTreeSet;
use std::fmt;
use std::iter;
use std::mem::replace;
use std::rc::Rc;
@ -1320,6 +1321,7 @@ pub struct Resolver<'a> {
crate_loader: &'a mut CrateLoader,
macro_names: FxHashSet<Ident>,
global_macros: FxHashMap<Name, &'a NameBinding<'a>>,
pub all_macros: FxHashMap<Name, Def>,
lexical_macro_resolutions: Vec<(Ident, &'a Cell<LegacyScope<'a>>)>,
macro_map: FxHashMap<DefId, Rc<SyntaxExtension>>,
macro_defs: FxHashMap<Mark, DefId>,
@ -1407,6 +1409,71 @@ impl<'a, 'b: 'a> ty::DefIdTree for &'a Resolver<'b> {
impl<'a> hir::lowering::Resolver for Resolver<'a> {
fn resolve_hir_path(&mut self, path: &mut hir::Path, is_value: bool) {
self.resolve_hir_path_cb(path, is_value,
|resolver, span, error| resolve_error(resolver, span, error))
}
fn resolve_str_path(&mut self, span: Span, crate_root: Option<&str>,
components: &[&str], is_value: bool) -> hir::Path {
let mut path = hir::Path {
span,
def: Def::Err,
segments: iter::once(keywords::CrateRoot.name()).chain({
crate_root.into_iter().chain(components.iter().cloned()).map(Symbol::intern)
}).map(hir::PathSegment::from_name).collect(),
};
self.resolve_hir_path(&mut path, is_value);
path
}
fn get_resolution(&mut self, id: NodeId) -> Option<PathResolution> {
self.def_map.get(&id).cloned()
}
fn definitions(&mut self) -> &mut Definitions {
&mut self.definitions
}
}
impl<'a> Resolver<'a> {
/// Rustdoc uses this to resolve things in a recoverable way. ResolutionError<'a>
/// isn't something that can be returned because it can't be made to live that long,
/// and also it's a private type. Fortunately rustdoc doesn't need to know the error,
/// just that an error occured.
pub fn resolve_str_path_error(&mut self, span: Span, path_str: &str, is_value: bool)
-> Result<hir::Path, ()> {
use std::iter;
let mut errored = false;
let mut path = if path_str.starts_with("::") {
hir::Path {
span,
def: Def::Err,
segments: iter::once(keywords::CrateRoot.name()).chain({
path_str.split("::").skip(1).map(Symbol::intern)
}).map(hir::PathSegment::from_name).collect(),
}
} else {
hir::Path {
span,
def: Def::Err,
segments: path_str.split("::").map(Symbol::intern)
.map(hir::PathSegment::from_name).collect(),
}
};
self.resolve_hir_path_cb(&mut path, is_value, |_, _, _| errored = true);
if errored || path.def == Def::Err {
Err(())
} else {
Ok(path)
}
}
/// resolve_hir_path, but takes a callback in case there was an error
fn resolve_hir_path_cb<F>(&mut self, path: &mut hir::Path, is_value: bool, error_callback: F)
where F: for<'c, 'b> FnOnce(&'c mut Resolver, Span, ResolutionError<'b>)
{
let namespace = if is_value { ValueNS } else { TypeNS };
let hir::Path { ref segments, span, ref mut def } = *path;
let path: Vec<SpannedIdent> = segments.iter()
@ -1418,24 +1485,16 @@ impl<'a> hir::lowering::Resolver for Resolver<'a> {
*def = path_res.base_def(),
PathResult::NonModule(..) => match self.resolve_path(&path, None, true, span) {
PathResult::Failed(span, msg, _) => {
resolve_error(self, span, ResolutionError::FailedToResolve(&msg));
error_callback(self, span, ResolutionError::FailedToResolve(&msg));
}
_ => {}
},
PathResult::Indeterminate => unreachable!(),
PathResult::Failed(span, msg, _) => {
resolve_error(self, span, ResolutionError::FailedToResolve(&msg));
error_callback(self, span, ResolutionError::FailedToResolve(&msg));
}
}
}
fn get_resolution(&mut self, id: NodeId) -> Option<PathResolution> {
self.def_map.get(&id).cloned()
}
fn definitions(&mut self) -> &mut Definitions {
&mut self.definitions
}
}
impl<'a> Resolver<'a> {
@ -1538,6 +1597,7 @@ impl<'a> Resolver<'a> {
crate_loader,
macro_names: FxHashSet(),
global_macros: FxHashMap(),
all_macros: FxHashMap(),
lexical_macro_resolutions: Vec::new(),
macro_map: FxHashMap(),
macro_exports: Vec::new(),
@ -1833,8 +1893,8 @@ impl<'a> Resolver<'a> {
// generate a fake "implementation scope" containing all the
// implementations thus found, for compatibility with old resolve pass.
fn with_scope<F>(&mut self, id: NodeId, f: F)
where F: FnOnce(&mut Resolver)
pub fn with_scope<F, T>(&mut self, id: NodeId, f: F) -> T
where F: FnOnce(&mut Resolver) -> T
{
let id = self.definitions.local_def_id(id);
let module = self.module_map.get(&id).cloned(); // clones a reference
@ -1845,13 +1905,14 @@ impl<'a> Resolver<'a> {
self.ribs[TypeNS].push(Rib::new(ModuleRibKind(module)));
self.finalize_current_module_macro_resolutions();
f(self);
let ret = f(self);
self.current_module = orig_module;
self.ribs[ValueNS].pop();
self.ribs[TypeNS].pop();
ret
} else {
f(self);
f(self)
}
}

5
src/librustc_resolve/macros.rs
View file

@ -409,7 +409,7 @@ impl<'a> Resolver<'a> {
def
}
fn resolve_macro_to_def_inner(&mut self, scope: Mark, path: &ast::Path,
pub fn resolve_macro_to_def_inner(&mut self, scope: Mark, path: &ast::Path,
kind: MacroKind, force: bool)
-> Result<Def, Determinacy> {
let ast::Path { ref segments, span } = *path;
@ -755,8 +755,9 @@ impl<'a> Resolver<'a> {
*legacy_scope = LegacyScope::Binding(self.arenas.alloc_legacy_binding(LegacyBinding {
parent: Cell::new(*legacy_scope), ident: ident, def_id: def_id, span: item.span,
}));
let def = Def::Macro(def_id, MacroKind::Bang);
self.all_macros.insert(ident.name, def);
if attr::contains_name(&item.attrs, "macro_export") {
let def = Def::Macro(def_id, MacroKind::Bang);
self.macro_exports.push(Export {
ident: ident.modern(),
def: def,

6
src/librustdoc/clean/inline.rs
View file

@ -135,7 +135,11 @@ pub fn record_extern_fqn(cx: &DocContext, did: DefId, kind: clean::TypeKind) {
None
}
});
let fqn = once(crate_name).chain(relative).collect();
let fqn = if let clean::TypeKind::Macro = kind {
vec![crate_name, relative.last().unwrap()]
} else {
once(crate_name).chain(relative).collect()
};
cx.renderinfo.borrow_mut().external_paths.insert(did, (fqn, kind));
}

287
src/librustdoc/clean/mod.rs
View file

@ -20,9 +20,10 @@ pub use self::FunctionRetTy::*;
pub use self::Visibility::*;
use syntax::abi::Abi;
use syntax::ast;
use syntax::ast::{self, AttrStyle};
use syntax::attr;
use syntax::codemap::Spanned;
use syntax::feature_gate::UnstableFeatures;
use syntax::ptr::P;
use syntax::symbol::keywords;
use syntax_pos::{self, DUMMY_SP, Pos, FileName};
@ -53,6 +54,7 @@ use core::DocContext;
use doctree;
use visit_ast;
use html::item_type::ItemType;
use html::markdown::markdown_links;
pub mod inline;
pub mod cfg;
@ -124,7 +126,7 @@ pub struct Crate {
pub masked_crates: FxHashSet<CrateNum>,
}
impl<'a, 'tcx> Clean<Crate> for visit_ast::RustdocVisitor<'a, 'tcx> {
impl<'a, 'tcx, 'rcx> Clean<Crate> for visit_ast::RustdocVisitor<'a, 'tcx, 'rcx> {
fn clean(&self, cx: &DocContext) -> Crate {
use ::visit_lib::LibEmbargoVisitor;
@ -305,6 +307,11 @@ impl Item {
pub fn collapsed_doc_value(&self) -> Option<String> {
self.attrs.collapsed_doc_value()
}
pub fn links(&self) -> Vec<(String, String)> {
self.attrs.links()
}
pub fn is_crate(&self) -> bool {
match self.inner {
StrippedItem(box ModuleItem(Module { is_crate: true, ..})) |
@ -465,6 +472,23 @@ impl Clean<Item> for doctree::Module {
"".to_string()
};
// maintain a stack of mod ids, for doc comment path resolution
// but we also need to resolve the module's own docs based on whether its docs were written
// inside or outside the module, so check for that
let attrs = if self.attrs.iter()
.filter(|a| a.check_name("doc"))
.next()
.map_or(true, |a| a.style == AttrStyle::Inner) {
// inner doc comment, use the module's own scope for resolution
cx.mod_ids.borrow_mut().push(self.id);
self.attrs.clean(cx)
} else {
// outer doc comment, use its parent's scope
let attrs = self.attrs.clean(cx);
cx.mod_ids.borrow_mut().push(self.id);
attrs
};
let mut items: Vec<Item> = vec![];
items.extend(self.extern_crates.iter().map(|x| x.clean(cx)));
items.extend(self.imports.iter().flat_map(|x| x.clean(cx)));
@ -481,6 +505,8 @@ impl Clean<Item> for doctree::Module {
items.extend(self.impls.iter().flat_map(|x| x.clean(cx)));
items.extend(self.macros.iter().map(|x| x.clean(cx)));
cx.mod_ids.borrow_mut().pop();
// determine if we should display the inner contents or
// the outer `mod` item for the source code.
let whence = {
@ -498,7 +524,7 @@ impl Clean<Item> for doctree::Module {
Item {
name: Some(name),
attrs: self.attrs.clean(cx),
attrs,
source: whence.clean(cx),
visibility: self.vis.clean(cx),
stability: self.stab.clean(cx),
@ -633,6 +659,7 @@ pub struct Attributes {
pub other_attrs: Vec<ast::Attribute>,
pub cfg: Option<Rc<Cfg>>,
pub span: Option<syntax_pos::Span>,
pub links: Vec<(String, DefId)>,
}
impl Attributes {
@ -762,11 +789,13 @@ impl Attributes {
Some(attr.clone())
})
}).collect();
Attributes {
doc_strings,
other_attrs,
cfg: if cfg == Cfg::True { None } else { Some(Rc::new(cfg)) },
span: sp,
links: vec![],
}
}
@ -785,6 +814,20 @@ impl Attributes {
None
}
}
/// Get links as a vector
///
/// Cache must be populated before call
pub fn links(&self) -> Vec<(String, String)> {
use html::format::href;
self.links.iter().filter_map(|&(ref s, did)| {
if let Some((href, ..)) = href(did) {
Some((s.clone(), href))
} else {
None
}
}).collect()
}
}
impl AttributesExt for Attributes {
@ -793,9 +836,243 @@ impl AttributesExt for Attributes {
}
}
/// Given a def, returns its name and disambiguator
/// for a value namespace
///
/// Returns None for things which cannot be ambiguous since
/// they exist in both namespaces (structs and modules)
fn value_ns_kind(def: Def, path_str: &str) -> Option<(&'static str, String)> {
match def {
// structs and mods exist in both namespaces. skip them
Def::StructCtor(..) | Def::Mod(..) => None,
Def::Variant(..) | Def::VariantCtor(..)
=> Some(("variant", format!("{}()", path_str))),
Def::Fn(..)
=> Some(("function", format!("{}()", path_str))),
Def::Method(..)
=> Some(("method", format!("{}()", path_str))),
Def::Const(..)
=> Some(("const", format!("const@{}", path_str))),
Def::Static(..)
=> Some(("static", format!("static@{}", path_str))),
_ => Some(("value", format!("value@{}", path_str))),
}
}
/// Given a def, returns its name, the article to be used, and a disambiguator
/// for the type namespace
fn type_ns_kind(def: Def, path_str: &str) -> (&'static str, &'static str, String) {
let (kind, article) = match def {
// we can still have non-tuple structs
Def::Struct(..) => ("struct", "a"),
Def::Enum(..) => ("enum", "an"),
Def::Trait(..) => ("trait", "a"),
Def::Union(..) => ("union", "a"),
_ => ("type", "a"),
};
(kind, article, format!("{}@{}", kind, path_str))
}
fn ambiguity_error(cx: &DocContext, attrs: &Attributes,
path_str: &str,
article1: &str, kind1: &str, disambig1: &str,
article2: &str, kind2: &str, disambig2: &str) {
let sp = attrs.doc_strings.first()
.map_or(DUMMY_SP, |a| a.span());
cx.sess()
.struct_span_err(sp,
&format!("`{}` is both {} {} and {} {}",
path_str, article1, kind1,
article2, kind2))
.help(&format!("try `{}` if you want to select the {}, \
or `{}` if you want to \
select the {}",
disambig1, kind1, disambig2,
kind2))
.emit();
}
/// Resolve a given string as a path, along with whether or not it is
/// in the value namespace
fn resolve(cx: &DocContext, path_str: &str, is_val: bool) -> Result<hir::Path, ()> {
// In case we're in a module, try to resolve the relative
// path
if let Some(id) = cx.mod_ids.borrow().last() {
cx.resolver.borrow_mut()
.with_scope(*id, |resolver| {
resolver.resolve_str_path_error(DUMMY_SP,
&path_str, is_val)
})
} else {
// FIXME(Manishearth) this branch doesn't seem to ever be hit, really
cx.resolver.borrow_mut()
.resolve_str_path_error(DUMMY_SP, &path_str, is_val)
}
}
/// Resolve a string as a macro
fn macro_resolve(cx: &DocContext, path_str: &str) -> Option<Def> {
use syntax::ext::base::MacroKind;
use syntax::ext::hygiene::Mark;
let segment = ast::PathSegment {
identifier: ast::Ident::from_str(path_str),
span: DUMMY_SP,
parameters: None,
};
let path = ast::Path {
span: DUMMY_SP,
segments: vec![segment],
};
let mut resolver = cx.resolver.borrow_mut();
let mark = Mark::root();
let res = resolver
.resolve_macro_to_def_inner(mark, &path, MacroKind::Bang, false);
if let Ok(def) = res {
Some(def)
} else if let Some(def) = resolver.all_macros.get(&path_str.into()) {
Some(*def)
} else {
None
}
}
enum PathKind {
/// can be either value or type, not a macro
Unknown,
/// macro
Macro,
/// values, functions, consts, statics, everything in the value namespace
Value,
/// types, traits, everything in the type namespace
Type
}
impl Clean<Attributes> for [ast::Attribute] {
fn clean(&self, cx: &DocContext) -> Attributes {
Attributes::from_ast(cx.sess().diagnostic(), self)
let mut attrs = Attributes::from_ast(cx.sess().diagnostic(), self);
if UnstableFeatures::from_environment().is_nightly_build() {
let dox = attrs.collapsed_doc_value().unwrap_or_else(String::new);
for link in markdown_links(&dox, cx.render_type) {
let def = {
let mut kind = PathKind::Unknown;
let path_str = if let Some(prefix) =
["struct@", "enum@", "type@",
"trait@", "union@"].iter()
.find(|p| link.starts_with(**p)) {
kind = PathKind::Type;
link.trim_left_matches(prefix)
} else if let Some(prefix) =
["const@", "static@",
"value@", "function@", "mod@", "fn@", "module@"]
.iter().find(|p| link.starts_with(**p)) {
kind = PathKind::Value;
link.trim_left_matches(prefix)
} else if link.ends_with("()") {
kind = PathKind::Value;
link.trim_right_matches("()")
} else if link.starts_with("macro@") {
kind = PathKind::Macro;
link.trim_left_matches("macro@")
} else if link.ends_with('!') {
kind = PathKind::Macro;
link.trim_right_matches('!')
} else {
&link[..]
}.trim();
// avoid resolving things (i.e. regular links) which aren't like paths
// FIXME(Manishearth) given that most links have slashes in them might be worth
// doing a check for slashes first
if path_str.contains(|ch: char| !(ch.is_alphanumeric() ||
ch == ':' || ch == '_')) {
continue;
}
match kind {
PathKind::Value => {
if let Ok(path) = resolve(cx, path_str, true) {
path.def
} else {
// this could just be a normal link or a broken link
// we could potentially check if something is
// "intra-doc-link-like" and warn in that case
continue;
}
}
PathKind::Type => {
if let Ok(path) = resolve(cx, path_str, false) {
path.def
} else {
// this could just be a normal link
continue;
}
}
PathKind::Unknown => {
// try everything!
if let Some(macro_def) = macro_resolve(cx, path_str) {
if let Ok(type_path) = resolve(cx, path_str, false) {
let (type_kind, article, type_disambig)
= type_ns_kind(type_path.def, path_str);
ambiguity_error(cx, &attrs, path_str,
article, type_kind, &type_disambig,
"a", "macro", &format!("macro@{}", path_str));
continue;
} else if let Ok(value_path) = resolve(cx, path_str, true) {
let (value_kind, value_disambig)
= value_ns_kind(value_path.def, path_str)
.expect("struct and mod cases should have been \
caught in previous branch");
ambiguity_error(cx, &attrs, path_str,
"a", value_kind, &value_disambig,
"a", "macro", &format!("macro@{}", path_str));
}
macro_def
} else if let Ok(type_path) = resolve(cx, path_str, false) {
// It is imperative we search for not-a-value first
// Otherwise we will find struct ctors for when we are looking
// for structs, and the link won't work.
// if there is something in both namespaces
if let Ok(value_path) = resolve(cx, path_str, true) {
let kind = value_ns_kind(value_path.def, path_str);
if let Some((value_kind, value_disambig)) = kind {
let (type_kind, article, type_disambig)
= type_ns_kind(type_path.def, path_str);
ambiguity_error(cx, &attrs, path_str,
article, type_kind, &type_disambig,
"a", value_kind, &value_disambig);
continue;
}
}
type_path.def
} else if let Ok(value_path) = resolve(cx, path_str, true) {
value_path.def
} else {
// this could just be a normal link
continue;
}
}
PathKind::Macro => {
if let Some(def) = macro_resolve(cx, path_str) {
def
} else {
continue
}
}
}
};
let id = register_def(cx, def);
attrs.links.push((link, id));
}
cx.sess().abort_if_errors();
}
attrs
}
}
@ -1853,6 +2130,7 @@ pub enum TypeKind {
Variant,
Typedef,
Foreign,
Macro,
}
pub trait GetDefId {
@ -3154,6 +3432,7 @@ fn register_def(cx: &DocContext, def: Def) -> DefId {
Def::TyForeign(i) => (i, TypeKind::Foreign),
Def::Static(i, _) => (i, TypeKind::Static),
Def::Variant(i) => (cx.tcx.parent_def_id(i).unwrap(), TypeKind::Enum),
Def::Macro(i, _) => (i, TypeKind::Macro),
Def::SelfTy(Some(def_id), _) => (def_id, TypeKind::Trait),
Def::SelfTy(_, Some(impl_def_id)) => {
return impl_def_id

64
src/librustdoc/core.rs
View file

@ -20,8 +20,10 @@ use rustc::lint;
use rustc::util::nodemap::FxHashMap;
use rustc_trans;
use rustc_resolve as resolve;
use rustc_metadata::creader::CrateLoader;
use rustc_metadata::cstore::CStore;
use syntax::ast::NodeId;
use syntax::codemap;
use syntax::feature_gate::UnstableFeatures;
use errors;
@ -35,6 +37,7 @@ use std::path::PathBuf;
use visit_ast::RustdocVisitor;
use clean;
use clean::Clean;
use html::markdown::RenderType;
use html::render::RenderInfo;
pub use rustc::session::config::Input;
@ -42,8 +45,11 @@ pub use rustc::session::search_paths::SearchPaths;
pub type ExternalPaths = FxHashMap<DefId, (Vec<String>, clean::TypeKind)>;
pub struct DocContext<'a, 'tcx: 'a> {
pub struct DocContext<'a, 'tcx: 'a, 'rcx: 'a> {
pub tcx: TyCtxt<'a, 'tcx, 'tcx>,
pub resolver: &'a RefCell<resolve::Resolver<'rcx>>,
/// The stack of module NodeIds up till this point
pub mod_ids: RefCell<Vec<NodeId>>,
pub populated_all_crate_impls: Cell<bool>,
// Note that external items for which `doc(hidden)` applies to are shown as
// non-reachable while local items aren't. This is because we're reusing
@ -54,6 +60,8 @@ pub struct DocContext<'a, 'tcx: 'a> {
pub renderinfo: RefCell<RenderInfo>,
/// Later on moved through `clean::Crate` into `html::render::CACHE_KEY`
pub external_traits: RefCell<FxHashMap<DefId, clean::Trait>>,
/// Which markdown renderer to use when extracting links.
pub render_type: RenderType,
// The current set of type and lifetime substitutions,
// for expanding type aliases at the HIR level:
@ -64,7 +72,7 @@ pub struct DocContext<'a, 'tcx: 'a> {
pub lt_substs: RefCell<FxHashMap<DefId, clean::Lifetime>>,
}
impl<'a, 'tcx> DocContext<'a, 'tcx> {
impl<'a, 'tcx, 'rcx> DocContext<'a, 'tcx, 'rcx> {
pub fn sess(&self) -> &session::Session {
&self.tcx.sess
}
@ -104,7 +112,8 @@ pub fn run_core(search_paths: SearchPaths,
triple: Option<String>,
maybe_sysroot: Option<PathBuf>,
allow_warnings: bool,
force_unstable_if_unmarked: bool) -> (clean::Crate, RenderInfo)
force_unstable_if_unmarked: bool,
render_type: RenderType) -> (clean::Crate, RenderInfo)
{
// Parse, resolve, and typecheck the given crate.
@ -156,16 +165,40 @@ pub fn run_core(search_paths: SearchPaths,
let name = ::rustc_trans_utils::link::find_crate_name(Some(&sess), &krate.attrs, &input);
let driver::ExpansionResult { defs, analysis, resolutions, mut hir_forest, .. } = {
let result = driver::phase_2_configure_and_expand(&sess,
&cstore,
krate,
None,
&name,
None,
resolve::MakeGlobMap::No,
|_| Ok(()));
abort_on_err(result, &sess)
let mut crate_loader = CrateLoader::new(&sess, &cstore, &name);
let resolver_arenas = resolve::Resolver::arenas();
let result = driver::phase_2_configure_and_expand_inner(&sess,
&cstore,
krate,
None,
&name,
None,
resolve::MakeGlobMap::No,
&resolver_arenas,
&mut crate_loader,
|_| Ok(()));
let driver::InnerExpansionResult {
mut hir_forest,
resolver,
..
} = abort_on_err(result, &sess);
// We need to hold on to the complete resolver, so we clone everything
// for the analysis passes to use. Suboptimal, but necessary in the
// current architecture.
let defs = resolver.definitions.clone();
let resolutions = ty::Resolutions {
freevars: resolver.freevars.clone(),
export_map: resolver.export_map.clone(),
trait_map: resolver.trait_map.clone(),
maybe_unused_trait_imports: resolver.maybe_unused_trait_imports.clone(),
maybe_unused_extern_crates: resolver.maybe_unused_extern_crates.clone(),
};
let analysis = ty::CrateAnalysis {
access_levels: Rc::new(AccessLevels::default()),
name: name.to_string(),
glob_map: if resolver.make_glob_map { Some(resolver.glob_map.clone()) } else { None },
};
let arenas = AllArenas::new();
@ -176,6 +209,8 @@ pub fn run_core(search_paths: SearchPaths,
&[],
&sess);
let resolver = RefCell::new(resolver);
abort_on_err(driver::phase_3_run_analysis_passes(&*trans,
control,
&sess,
@ -203,12 +238,15 @@ pub fn run_core(search_paths: SearchPaths,
let ctxt = DocContext {
tcx,
resolver: &resolver,
populated_all_crate_impls: Cell::new(false),
access_levels: RefCell::new(access_levels),
external_traits: Default::default(),
renderinfo: Default::default(),
render_type,
ty_substs: Default::default(),
lt_substs: Default::default(),
mod_ids: Default::default(),
};
debug!("crate: {:?}", tcx.hir.krate());

4
src/librustdoc/externalfiles.rs
View file

@ -37,7 +37,7 @@ impl ExternalHtml {
)
.and_then(|(ih, bc)|
load_external_files(md_before_content)
.map(|m_bc| (ih, format!("{}{}", bc, Markdown(&m_bc, render))))
.map(|m_bc| (ih, format!("{}{}", bc, Markdown(&m_bc, &[], render))))
)
.and_then(|(ih, bc)|
load_external_files(after_content)
@ -45,7 +45,7 @@ impl ExternalHtml {
)
.and_then(|(ih, bc, ac)|
load_external_files(md_after_content)
.map(|m_ac| (ih, bc, format!("{}{}", ac, Markdown(&m_ac, render))))
.map(|m_ac| (ih, bc, format!("{}{}", ac, Markdown(&m_ac, &[], render))))
)
.map(|(ih, bc, ac)|
ExternalHtml {

1
src/librustdoc/html/item_type.rs
View file

@ -102,6 +102,7 @@ impl From<clean::TypeKind> for ItemType {
clean::TypeKind::Variant => ItemType::Variant,
clean::TypeKind::Typedef => ItemType::Typedef,
clean::TypeKind::Foreign => ItemType::ForeignType,
clean::TypeKind::Macro => ItemType::Macro,
}
}
}

598
src/librustdoc/html/markdown.rs
View file

@ -56,15 +56,16 @@ pub enum RenderType {
/// A unit struct which has the `fmt::Display` trait implemented. When
/// formatted, this struct will emit the HTML corresponding to the rendered
/// version of the contained markdown string.
// The second parameter is whether we need a shorter version or not.
pub struct Markdown<'a>(pub &'a str, pub RenderType);
/// The second parameter is a list of link replacements
// The third parameter is whether we need a shorter version or not.
pub struct Markdown<'a>(pub &'a str, pub &'a [(String, String)], pub RenderType);
/// A unit struct like `Markdown`, that renders the markdown with a
/// table of contents.
pub struct MarkdownWithToc<'a>(pub &'a str, pub RenderType);
/// A unit struct like `Markdown`, that renders the markdown escaping HTML tags.
pub struct MarkdownHtml<'a>(pub &'a str, pub RenderType);
/// A unit struct like `Markdown`, that renders only the first paragraph.
pub struct MarkdownSummaryLine<'a>(pub &'a str);
pub struct MarkdownSummaryLine<'a>(pub &'a str, pub &'a [(String, String)]);
/// Controls whether a line will be hidden or shown in HTML output.
///
@ -247,6 +248,39 @@ impl<'a, I: Iterator<Item = Event<'a>>> Iterator for CodeBlocks<'a, I> {
}
}
/// Make headings links with anchor ids and build up TOC.
struct LinkReplacer<'a, 'b, I: Iterator<Item = Event<'a>>> {
inner: I,
links: &'b [(String, String)]
}
impl<'a, 'b, I: Iterator<Item = Event<'a>>> LinkReplacer<'a, 'b, I> {
fn new(iter: I, links: &'b [(String, String)]) -> Self {
LinkReplacer {
inner: iter,
links
}
}
}
impl<'a, 'b, I: Iterator<Item = Event<'a>>> Iterator for LinkReplacer<'a, 'b, I> {
type Item = Event<'a>;
fn next(&mut self) -> Option<Self::Item> {
let event = self.inner.next();
if let Some(Event::Start(Tag::Link(dest, text))) = event {
if let Some(&(_, ref replace)) = self.links.into_iter().find(|link| &*link.0 == &*dest)
{
Some(Event::Start(Tag::Link(replace.to_owned().into(), text)))
} else {
Some(Event::Start(Tag::Link(dest, text)))
}
} else {
event
}
}
}
/// Make headings links with anchor ids and build up TOC.
struct HeadingLinks<'a, 'b, I: Iterator<Item = Event<'a>>> {
inner: I,
@ -527,6 +561,8 @@ struct MyOpaque {
*const hoedown_buffer, *const hoedown_renderer_data,
libc::size_t),
toc_builder: Option<TocBuilder>,
links_out: Option<Vec<String>>,
links_replace: Vec<(String, String)>,
}
extern {
@ -555,186 +591,293 @@ impl hoedown_buffer {
}
}
pub fn render(w: &mut fmt::Formatter,
s: &str,
print_toc: bool,
html_flags: libc::c_uint) -> fmt::Result {
extern fn block(ob: *mut hoedown_buffer, orig_text: *const hoedown_buffer,
lang: *const hoedown_buffer, data: *const hoedown_renderer_data,
line: libc::size_t) {
unsafe {
if orig_text.is_null() { return }
extern fn hoedown_block(ob: *mut hoedown_buffer, orig_text: *const hoedown_buffer,
lang: *const hoedown_buffer, data: *const hoedown_renderer_data,
line: libc::size_t) {
unsafe {
if orig_text.is_null() { return }
let opaque = (*data).opaque as *mut hoedown_html_renderer_state;
let my_opaque: &MyOpaque = &*((*opaque).opaque as *const MyOpaque);
let text = (*orig_text).as_bytes();
let origtext = str::from_utf8(text).unwrap();
let origtext = origtext.trim_left();
debug!("docblock: ==============\n{:?}\n=======", text);
let mut compile_fail = false;
let mut ignore = false;
let opaque = (*data).opaque as *mut hoedown_html_renderer_state;
let my_opaque: &MyOpaque = &*((*opaque).opaque as *const MyOpaque);
let text = (*orig_text).as_bytes();
let origtext = str::from_utf8(text).unwrap();
let origtext = origtext.trim_left();
debug!("docblock: ==============\n{:?}\n=======", text);
let mut compile_fail = false;
let mut ignore = false;
let rendered = if lang.is_null() || origtext.is_empty() {
false
} else {
let rlang = (*lang).as_bytes();
let rlang = str::from_utf8(rlang).unwrap();
let parse_result = LangString::parse(rlang);
compile_fail = parse_result.compile_fail;
ignore = parse_result.ignore;
if !parse_result.rust {
(my_opaque.dfltblk)(ob, orig_text, lang,
opaque as *const hoedown_renderer_data,
line);
true
} else {
false
}
};
let lines = origtext.lines().filter_map(|l| map_line(l).for_html());
let text = lines.collect::<Vec<&str>>().join("\n");
if rendered { return }
PLAYGROUND.with(|play| {
// insert newline to clearly separate it from the
// previous block so we can shorten the html output
let mut s = String::from("\n");
let playground_button = play.borrow().as_ref().and_then(|&(ref krate, ref url)| {
if url.is_empty() {
return None;
}
let test = origtext.lines()
.map(|l| map_line(l).for_code())
.collect::<Vec<&str>>().join("\n");
let krate = krate.as_ref().map(|s| &**s);
let (test, _) = test::make_test(&test, krate, false,
&Default::default());
let channel = if test.contains("#![feature(") {
"&amp;version=nightly"
} else {
""
};
// These characters don't need to be escaped in a URI.
// FIXME: use a library function for percent encoding.
fn dont_escape(c: u8) -> bool {
(b'a' <= c && c <= b'z') ||
(b'A' <= c && c <= b'Z') ||
(b'0' <= c && c <= b'9') ||
c == b'-' || c == b'_' || c == b'.' ||
c == b'~' || c == b'!' || c == b'\'' ||
c == b'(' || c == b')' || c == b'*'
}
let mut test_escaped = String::new();
for b in test.bytes() {
if dont_escape(b) {
test_escaped.push(char::from(b));
} else {
write!(test_escaped, "%{:02X}", b).unwrap();
}
}
Some(format!(
r#"<a class="test-arrow" target="_blank" href="{}?code={}{}">Run</a>"#,
url, test_escaped, channel
))
});
let tooltip = if ignore {
Some(("This example is not tested", "ignore"))
} else if compile_fail {
Some(("This example deliberately fails to compile", "compile_fail"))
} else {
None
};
s.push_str(&highlight::render_with_highlighting(
&text,
Some(&format!("rust-example-rendered{}",
if ignore { " ignore" }
else if compile_fail { " compile_fail" }
else { "" })),
None,
playground_button.as_ref().map(String::as_str),
tooltip));
hoedown_buffer_put(ob, s.as_ptr(), s.len());
})
}
}
extern fn header(ob: *mut hoedown_buffer, text: *const hoedown_buffer,
level: libc::c_int, data: *const hoedown_renderer_data,
_: libc::size_t) {
// hoedown does this, we may as well too
unsafe { hoedown_buffer_put(ob, "\n".as_ptr(), 1); }
// Extract the text provided
let s = if text.is_null() {
"".to_owned()
let rendered = if lang.is_null() || origtext.is_empty() {
false
} else {
let s = unsafe { (*text).as_bytes() };
str::from_utf8(&s).unwrap().to_owned()
let rlang = (*lang).as_bytes();
let rlang = str::from_utf8(rlang).unwrap();
let parse_result = LangString::parse(rlang);
compile_fail = parse_result.compile_fail;
ignore = parse_result.ignore;
if !parse_result.rust {
(my_opaque.dfltblk)(ob, orig_text, lang,
opaque as *const hoedown_renderer_data,
line);
true
} else {
false
}
};
// Discard '<em>', '<code>' tags and some escaped characters,
// transform the contents of the header into a hyphenated string
// without non-alphanumeric characters other than '-' and '_'.
//
// This is a terrible hack working around how hoedown gives us rendered
// html for text rather than the raw text.
let mut id = s.clone();
let repl_sub = vec!["<em>", "</em>", "<code>", "</code>",
"<strong>", "</strong>",
"&lt;", "&gt;", "&amp;", "&#39;", "&quot;"];
for sub in repl_sub {
id = id.replace(sub, "");
}
let id = id.chars().filter_map(|c| {
if c.is_alphanumeric() || c == '-' || c == '_' {
if c.is_ascii() {
Some(c.to_ascii_lowercase())
} else {
Some(c)
let lines = origtext.lines().filter_map(|l| map_line(l).for_html());
let text = lines.collect::<Vec<&str>>().join("\n");
if rendered { return }
PLAYGROUND.with(|play| {
// insert newline to clearly separate it from the
// previous block so we can shorten the html output
let mut s = String::from("\n");
let playground_button = play.borrow().as_ref().and_then(|&(ref krate, ref url)| {
if url.is_empty() {
return None;
}
} else if c.is_whitespace() && c.is_ascii() {
Some('-')
let test = origtext.lines()
.map(|l| map_line(l).for_code())
.collect::<Vec<&str>>().join("\n");
let krate = krate.as_ref().map(|s| &**s);
let (test, _) = test::make_test(&test, krate, false,
&Default::default());
let channel = if test.contains("#![feature(") {
"&amp;version=nightly"
} else {
""
};
// These characters don't need to be escaped in a URI.
// FIXME: use a library function for percent encoding.
fn dont_escape(c: u8) -> bool {
(b'a' <= c && c <= b'z') ||
(b'A' <= c && c <= b'Z') ||
(b'0' <= c && c <= b'9') ||
c == b'-' || c == b'_' || c == b'.' ||
c == b'~' || c == b'!' || c == b'\'' ||
c == b'(' || c == b')' || c == b'*'
}
let mut test_escaped = String::new();
for b in test.bytes() {
if dont_escape(b) {
test_escaped.push(char::from(b));
} else {
write!(test_escaped, "%{:02X}", b).unwrap();
}
}
Some(format!(
r#"<a class="test-arrow" target="_blank" href="{}?code={}{}">Run</a>"#,
url, test_escaped, channel
))
});
let tooltip = if ignore {
Some(("This example is not tested", "ignore"))
} else if compile_fail {
Some(("This example deliberately fails to compile", "compile_fail"))
} else {
None
};
s.push_str(&highlight::render_with_highlighting(
&text,
Some(&format!("rust-example-rendered{}",
if ignore { " ignore" }
else if compile_fail { " compile_fail" }
else { "" })),
None,
playground_button.as_ref().map(String::as_str),
tooltip));
hoedown_buffer_put(ob, s.as_ptr(), s.len());
})
}
}
extern fn hoedown_header(ob: *mut hoedown_buffer, text: *const hoedown_buffer,
level: libc::c_int, data: *const hoedown_renderer_data,
_: libc::size_t) {
// hoedown does this, we may as well too
unsafe { hoedown_buffer_put(ob, "\n".as_ptr(), 1); }
// Extract the text provided
let s = if text.is_null() {
"".to_owned()
} else {
let s = unsafe { (*text).as_bytes() };
str::from_utf8(&s).unwrap().to_owned()
};
// Discard '<em>', '<code>' tags and some escaped characters,
// transform the contents of the header into a hyphenated string
// without non-alphanumeric characters other than '-' and '_'.
//
// This is a terrible hack working around how hoedown gives us rendered
// html for text rather than the raw text.
let mut id = s.clone();
let repl_sub = vec!["<em>", "</em>", "<code>", "</code>",
"<strong>", "</strong>",
"&lt;", "&gt;", "&amp;", "&#39;", "&quot;"];
for sub in repl_sub {
id = id.replace(sub, "");
}
let id = id.chars().filter_map(|c| {
if c.is_alphanumeric() || c == '-' || c == '_' {
if c.is_ascii() {
Some(c.to_ascii_lowercase())
} else {
Some(c)
}
}).collect::<String>();
} else if c.is_whitespace() && c.is_ascii() {
Some('-')
} else {
None
}
}).collect::<String>();
let opaque = unsafe { (*data).opaque as *mut hoedown_html_renderer_state };
let opaque = unsafe { &mut *((*opaque).opaque as *mut MyOpaque) };
let id = derive_id(id);
let sec = opaque.toc_builder.as_mut().map_or("".to_owned(), |builder| {
format!("{} ", builder.push(level as u32, s.clone(), id.clone()))
});
// Render the HTML
let text = format!("<h{lvl} id='{id}' class='section-header'>\
<a href='#{id}'>{sec}{}</a></h{lvl}>",
s, lvl = level, id = id, sec = sec);
unsafe { hoedown_buffer_put(ob, text.as_ptr(), text.len()); }
}
extern fn hoedown_codespan(
ob: *mut hoedown_buffer,
text: *const hoedown_buffer,
_: *const hoedown_renderer_data,
_: libc::size_t
) -> libc::c_int {
let content = if text.is_null() {
"".to_owned()
} else {
let bytes = unsafe { (*text).as_bytes() };
let s = str::from_utf8(bytes).unwrap();
collapse_whitespace(s)
};
let content = format!("<code>{}</code>", Escape(&content));
unsafe {
hoedown_buffer_put(ob, content.as_ptr(), content.len());
}
// Return anything except 0, which would mean "also print the code span verbatim".
1
}
pub fn render(w: &mut fmt::Formatter,
s: &str,
links: &[(String, String)],
print_toc: bool,
html_flags: libc::c_uint) -> fmt::Result {
// copied from pulldown-cmark (MIT license, Google)
// https://github.com/google/pulldown-cmark
// this is temporary till we remove the hoedown renderer
static HREF_SAFE: [u8; 128] = [
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 1, 0, 1, 1, 1, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1,
1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 0, 1,
1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1,
0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0,
];
static HEX_CHARS: &'static [u8] = b"0123456789ABCDEF";
fn escape_href(ob: &mut String, s: &str) {
let mut mark = 0;
for i in 0..s.len() {
let c = s.as_bytes()[i];
if c >= 0x80 || HREF_SAFE[c as usize] == 0 {
// character needing escape
// write partial substring up to mark
if mark < i {
ob.push_str(&s[mark..i]);
}
match c {
b'&' => {
ob.push_str("&amp;");
},
b'\'' => {
ob.push_str("&#x27;");
},
_ => {
let mut buf = [0u8; 3];
buf[0] = b'%';
buf[1] = HEX_CHARS[((c as usize) >> 4) & 0xF];
buf[2] = HEX_CHARS[(c as usize) & 0xF];
ob.push_str(str::from_utf8(&buf).unwrap());
}
}
mark = i + 1; // all escaped characters are ASCII
}
}
ob.push_str(&s[mark..]);
}
// end code copied from pulldown-cmark
extern fn hoedown_link(
ob: *mut hoedown_buffer,
content: *const hoedown_buffer,
link: *const hoedown_buffer,
title: *const hoedown_buffer,
data: *const hoedown_renderer_data,
_line: libc::size_t
) -> libc::c_int {
if link.is_null() {
return 0;
}
let opaque = unsafe { (*data).opaque as *mut hoedown_html_renderer_state };
let opaque = unsafe { &mut *((*opaque).opaque as *mut MyOpaque) };
let id = derive_id(id);
let sec = opaque.toc_builder.as_mut().map_or("".to_owned(), |builder| {
format!("{} ", builder.push(level as u32, s.clone(), id.clone()))
});
// Render the HTML
let text = format!("<h{lvl} id='{id}' class='section-header'>\
<a href='#{id}'>{sec}{}</a></h{lvl}>",
s, lvl = level, id = id, sec = sec);
unsafe { hoedown_buffer_put(ob, text.as_ptr(), text.len()); }
}
extern fn codespan(
ob: *mut hoedown_buffer,
text: *const hoedown_buffer,
_: *const hoedown_renderer_data,
_: libc::size_t
) -> libc::c_int {
let content = if text.is_null() {
"".to_owned()
} else {
let bytes = unsafe { (*text).as_bytes() };
let s = str::from_utf8(bytes).unwrap();
collapse_whitespace(s)
let link = {
let s = unsafe { (*link).as_bytes() };
str::from_utf8(s).unwrap().to_owned()
};
let content = format!("<code>{}</code>", Escape(&content));
unsafe {
hoedown_buffer_put(ob, content.as_ptr(), content.len());
}
// Return anything except 0, which would mean "also print the code span verbatim".
let link = if let Some(&(_, ref new_target)) = opaque.links_replace
.iter()
.find(|t| &*t.0 == &*link) {
new_target.to_owned()
} else {
link
};
let content = unsafe {
content.as_ref().map(|c| {
let s = c.as_bytes();
str::from_utf8(s).unwrap().to_owned()
})
};
let mut link_buf = String::new();
escape_href(&mut link_buf, &link);
let title = unsafe {
title.as_ref().map(|t| {
let s = t.as_bytes();
str::from_utf8(s).unwrap().to_owned()
})
};
let link_out = format!("<a href=\"{link}\"{title}>{content}</a>",
link = link_buf,
title = title.map_or(String::new(),
|t| format!(" title=\"{}\"", t)),
content = content.unwrap_or(String::new()));
unsafe { hoedown_buffer_put(ob, link_out.as_ptr(), link_out.len()); }
//return "anything but 0" to show we've written the link in
1
}
@ -743,13 +886,16 @@ pub fn render(w: &mut fmt::Formatter,
let renderer = hoedown_html_renderer_new(html_flags, 0);
let mut opaque = MyOpaque {
dfltblk: (*renderer).blockcode.unwrap(),
toc_builder: if print_toc {Some(TocBuilder::new())} else {None}
toc_builder: if print_toc {Some(TocBuilder::new())} else {None},
links_out: None,
links_replace: links.to_vec(),
};
(*((*renderer).opaque as *mut hoedown_html_renderer_state)).opaque
= &mut opaque as *mut _ as *mut libc::c_void;
(*renderer).blockcode = Some(block);
(*renderer).header = Some(header);
(*renderer).codespan = Some(codespan);
(*renderer).blockcode = Some(hoedown_block);
(*renderer).header = Some(hoedown_header);
(*renderer).codespan = Some(hoedown_codespan);
(*renderer).link = Some(hoedown_link);
let document = hoedown_document_new(renderer, HOEDOWN_EXTENSIONS, 16);
hoedown_document_render(document, ob, s.as_ptr(),
@ -993,12 +1139,12 @@ impl LangString {
impl<'a> fmt::Display for Markdown<'a> {
fn fmt(&self, fmt: &mut fmt::Formatter) -> fmt::Result {
let Markdown(md, render_type) = *self;
let Markdown(md, links, render_type) = *self;
// This is actually common enough to special-case
if md.is_empty() { return Ok(()) }
if render_type == RenderType::Hoedown {
render(fmt, md, false, 0)
render(fmt, md, links, false, 0)
} else {
let mut opts = Options::empty();
opts.insert(OPTION_ENABLE_TABLES);
@ -1009,7 +1155,11 @@ impl<'a> fmt::Display for Markdown<'a> {
let mut s = String::with_capacity(md.len() * 3 / 2);
html::push_html(&mut s,
Footnotes::new(CodeBlocks::new(HeadingLinks::new(p, None))));
Footnotes::new(
CodeBlocks::new(
LinkReplacer::new(
HeadingLinks::new(p, None),
links))));
fmt.write_str(&s)
}
@ -1021,7 +1171,7 @@ impl<'a> fmt::Display for MarkdownWithToc<'a> {
let MarkdownWithToc(md, render_type) = *self;
if render_type == RenderType::Hoedown {
render(fmt, md, true, 0)
render(fmt, md, &[], true, 0)
} else {
let mut opts = Options::empty();
opts.insert(OPTION_ENABLE_TABLES);
@ -1050,7 +1200,7 @@ impl<'a> fmt::Display for MarkdownHtml<'a> {
// This is actually common enough to special-case
if md.is_empty() { return Ok(()) }
if render_type == RenderType::Hoedown {
render(fmt, md, false, HOEDOWN_HTML_ESCAPE)
render(fmt, md, &[], false, HOEDOWN_HTML_ESCAPE)
} else {
let mut opts = Options::empty();
opts.insert(OPTION_ENABLE_TABLES);
@ -1076,7 +1226,7 @@ impl<'a> fmt::Display for MarkdownHtml<'a> {
impl<'a> fmt::Display for MarkdownSummaryLine<'a> {
fn fmt(&self, fmt: &mut fmt::Formatter) -> fmt::Result {
let MarkdownSummaryLine(md) = *self;
let MarkdownSummaryLine(md, links) = *self;
// This is actually common enough to special-case
if md.is_empty() { return Ok(()) }
@ -1084,7 +1234,7 @@ impl<'a> fmt::Display for MarkdownSummaryLine<'a> {
let mut s = String::new();
html::push_html(&mut s, SummaryLine::new(p));
html::push_html(&mut s, LinkReplacer::new(SummaryLine::new(p), links));
fmt.write_str(&s)
}
@ -1140,6 +1290,90 @@ pub fn plain_summary_line(md: &str) -> String {
s
}
pub fn markdown_links(md: &str, render_type: RenderType) -> Vec<String> {
if md.is_empty() {
return vec![];
}
match render_type {
RenderType::Hoedown => {
extern fn hoedown_link(
_ob: *mut hoedown_buffer,
_content: *const hoedown_buffer,
link: *const hoedown_buffer,
_title: *const hoedown_buffer,
data: *const hoedown_renderer_data,
_line: libc::size_t
) -> libc::c_int {
if link.is_null() {
return 0;
}
let opaque = unsafe { (*data).opaque as *mut hoedown_html_renderer_state };
let opaque = unsafe { &mut *((*opaque).opaque as *mut MyOpaque) };
if let Some(ref mut links) = opaque.links_out {
let s = unsafe { (*link).as_bytes() };
let s = str::from_utf8(&s).unwrap().to_owned();
debug!("found link: {}", s);
links.push(s);
}
//returning 0 here means "emit the span verbatim", but we're not using the output
//anyway so we don't really care
0
}
unsafe {
let ob = hoedown_buffer_new(DEF_OUNIT);
let renderer = hoedown_html_renderer_new(0, 0);
let mut opaque = MyOpaque {
dfltblk: (*renderer).blockcode.unwrap(),
toc_builder: None,
links_out: Some(vec![]),
links_replace: vec![],
};
(*((*renderer).opaque as *mut hoedown_html_renderer_state)).opaque
= &mut opaque as *mut _ as *mut libc::c_void;
(*renderer).header = Some(hoedown_header);
(*renderer).codespan = Some(hoedown_codespan);
(*renderer).link = Some(hoedown_link);
let document = hoedown_document_new(renderer, HOEDOWN_EXTENSIONS, 16);
hoedown_document_render(document, ob, md.as_ptr(),
md.len() as libc::size_t);
hoedown_document_free(document);
hoedown_html_renderer_free(renderer);
hoedown_buffer_free(ob);
opaque.links_out.unwrap()
}
}
RenderType::Pulldown => {
let mut opts = Options::empty();
opts.insert(OPTION_ENABLE_TABLES);
opts.insert(OPTION_ENABLE_FOOTNOTES);
let p = Parser::new_ext(md, opts);
let iter = Footnotes::new(HeadingLinks::new(p, None));
let mut links = vec![];
for ev in iter {
if let Event::Start(Tag::Link(dest, _)) = ev {
debug!("found link: {}", dest);
links.push(dest.into_owned());
}
}
links
}
}
}
#[cfg(test)]
mod tests {
use super::{LangString, Markdown, MarkdownHtml};
@ -1191,14 +1425,14 @@ mod tests {
#[test]
fn issue_17736() {
let markdown = "# title";
format!("{}", Markdown(markdown, RenderType::Pulldown));
format!("{}", Markdown(markdown, &[], RenderType::Pulldown));
reset_ids(true);
}
#[test]
fn test_header() {
fn t(input: &str, expect: &str) {
let output = format!("{}", Markdown(input, RenderType::Pulldown));
let output = format!("{}", Markdown(input, &[], RenderType::Pulldown));
assert_eq!(output, expect, "original: {}", input);
reset_ids(true);
}
@ -1220,7 +1454,7 @@ mod tests {
#[test]
fn test_header_ids_multiple_blocks() {
fn t(input: &str, expect: &str) {
let output = format!("{}", Markdown(input, RenderType::Pulldown));
let output = format!("{}", Markdown(input, &[], RenderType::Pulldown));
assert_eq!(output, expect, "original: {}", input);
}

31
src/librustdoc/html/render.rs
View file

@ -421,7 +421,7 @@ thread_local!(pub static CURRENT_LOCATION_KEY: RefCell<Vec<String>> =
thread_local!(pub static USED_ID_MAP: RefCell<FxHashMap<String, usize>> =
RefCell::new(init_ids()));
pub fn render_text<F: FnMut(RenderType) -> String>(mut render: F) -> (String, String) {
pub fn render_text<T, F: FnMut(RenderType) -> T>(mut render: F) -> (T, T) {
// Save the state of USED_ID_MAP so it only gets updated once even
// though we're rendering twice.
let orig_used_id_map = USED_ID_MAP.with(|map| map.borrow().clone());
@ -1284,7 +1284,7 @@ impl DocFolder for Cache {
clean::FunctionItem(..) | clean::ModuleItem(..) |
clean::ForeignFunctionItem(..) | clean::ForeignStaticItem(..) |
clean::ConstantItem(..) | clean::StaticItem(..) |
clean::UnionItem(..) | clean::ForeignTypeItem
clean::UnionItem(..) | clean::ForeignTypeItem | clean::MacroItem(..)
if !self.stripped_mod => {
// Re-exported items mean that the same id can show up twice
// in the rustdoc ast that we're looking at. We know,
@ -1861,12 +1861,14 @@ fn document(w: &mut fmt::Formatter, cx: &Context, item: &clean::Item) -> fmt::Re
/// rendering between Pulldown and Hoedown.
fn render_markdown(w: &mut fmt::Formatter,
md_text: &str,
links: Vec<(String, String)>,
span: Span,
render_type: RenderType,
prefix: &str,
scx: &SharedContext)
-> fmt::Result {
let (hoedown_output, pulldown_output) = render_text(|ty| format!("{}", Markdown(md_text, ty)));
let (hoedown_output, pulldown_output) =
render_text(|ty| format!("{}", Markdown(md_text, &links, ty)));
let mut differences = html_diff::get_differences(&pulldown_output, &hoedown_output);
differences.retain(|s| {
match *s {
@ -1898,7 +1900,13 @@ fn document_short(w: &mut fmt::Formatter, item: &clean::Item, link: AssocItemLin
} else {
format!("{}", &plain_summary_line(Some(s)))
};
render_markdown(w, &markdown, item.source.clone(), cx.render_type, prefix, &cx.shared)?;
render_markdown(w,
&markdown,
item.links(),
item.source.clone(),
cx.render_type,
prefix,
&cx.shared)?;
} else if !prefix.is_empty() {
write!(w, "<div class='docblock'>{}</div>", prefix)?;
}
@ -1924,7 +1932,13 @@ fn document_full(w: &mut fmt::Formatter, item: &clean::Item,
cx: &Context, prefix: &str) -> fmt::Result {
if let Some(s) = cx.shared.maybe_collapsed_doc_value(item) {
debug!("Doc block: =====\n{}\n=====", s);
render_markdown(w, &*s, item.source.clone(), cx.render_type, prefix, &cx.shared)?;
render_markdown(w,
&*s,
item.links(),
item.source.clone(),
cx.render_type,
prefix,
&cx.shared)?;
} else if !prefix.is_empty() {
write!(w, "<div class='docblock'>{}</div>", prefix)?;
}
@ -2146,10 +2160,10 @@ fn item_module(w: &mut fmt::Formatter, cx: &Context,
stab_docs = stab_docs,
docs = if cx.render_type == RenderType::Hoedown {
format!("{}",
shorter(Some(&Markdown(doc_value,
shorter(Some(&Markdown(doc_value, &myitem.links(),
RenderType::Hoedown).to_string())))
} else {
format!("{}", MarkdownSummaryLine(doc_value))
format!("{}", MarkdownSummaryLine(doc_value, &myitem.links()))
},
class = myitem.type_(),
stab = myitem.stability_class().unwrap_or("".to_string()),
@ -3338,7 +3352,8 @@ fn render_impl(w: &mut fmt::Formatter, cx: &Context, i: &Impl, link: AssocItemLi
write!(w, "</span>")?;
write!(w, "</h3>\n")?;
if let Some(ref dox) = cx.shared.maybe_collapsed_doc_value(&i.impl_item) {
write!(w, "<div class='docblock'>{}</div>", Markdown(&*dox, cx.render_type))?;
write!(w, "<div class='docblock'>{}</div>",
Markdown(&*dox, &i.impl_item.links(), cx.render_type))?;
}
}

7
src/librustdoc/lib.rs
View file

@ -503,6 +503,11 @@ where R: 'static + Send, F: 'static + Send + FnOnce(Output) -> R {
let crate_name = matches.opt_str("crate-name");
let crate_version = matches.opt_str("crate-version");
let plugin_path = matches.opt_str("plugin-path");
let render_type = if matches.opt_present("disable-commonmark") {
RenderType::Hoedown
} else {
RenderType::Pulldown
};
info!("starting to run rustc");
let display_warnings = matches.opt_present("display-warnings");
@ -517,7 +522,7 @@ where R: 'static + Send, F: 'static + Send + FnOnce(Output) -> R {
let (mut krate, renderinfo) =
core::run_core(paths, cfgs, externs, Input::File(cratefile), triple, maybe_sysroot,
display_warnings, force_unstable_if_unmarked);
display_warnings, force_unstable_if_unmarked, render_type);
info!("finished with rustc");

2
src/librustdoc/markdown.rs
View file

@ -104,7 +104,7 @@ pub fn render(input: &Path, mut output: PathBuf, matches: &getopts::Matches,
} else {
// Save the state of USED_ID_MAP so it only gets updated once even
// though we're rendering twice.
render_text(|ty| format!("{}", Markdown(text, ty)))
render_text(|ty| format!("{}", Markdown(text, &[], ty)))
};
let mut differences = html_diff::get_differences(&pulldown_output, &hoedown_output);

12
src/librustdoc/visit_ast.rs
View file

@ -40,11 +40,11 @@ use doctree::*;
// also, is there some reason that this doesn't use the 'visit'
// framework from syntax?
pub struct RustdocVisitor<'a, 'tcx: 'a> {
cstore: &'tcx CrateStore,
pub struct RustdocVisitor<'a, 'tcx: 'a, 'rcx: 'a> {
cstore: &'a CrateStore,
pub module: Module,
pub attrs: hir::HirVec<ast::Attribute>,
pub cx: &'a core::DocContext<'a, 'tcx>,
pub cx: &'a core::DocContext<'a, 'tcx, 'rcx>,
view_item_stack: FxHashSet<ast::NodeId>,
inlining: bool,
/// Is the current module and all of its parents public?
@ -52,9 +52,9 @@ pub struct RustdocVisitor<'a, 'tcx: 'a> {
reexported_macros: FxHashSet<DefId>,
}
impl<'a, 'tcx> RustdocVisitor<'a, 'tcx> {
pub fn new(cstore: &'tcx CrateStore,
cx: &'a core::DocContext<'a, 'tcx>) -> RustdocVisitor<'a, 'tcx> {
impl<'a, 'tcx, 'rcx> RustdocVisitor<'a, 'tcx, 'rcx> {
pub fn new(cstore: &'a CrateStore,
cx: &'a core::DocContext<'a, 'tcx, 'rcx>) -> RustdocVisitor<'a, 'tcx, 'rcx> {
// If the root is re-exported, terminate all recursion.
let mut stack = FxHashSet();
stack.insert(ast::CRATE_NODE_ID);

8
src/librustdoc/visit_lib.rs
View file

@ -22,8 +22,8 @@ use clean::{AttributesExt, NestedAttributesExt};
/// Similar to `librustc_privacy::EmbargoVisitor`, but also takes
/// specific rustdoc annotations into account (i.e. `doc(hidden)`)
pub struct LibEmbargoVisitor<'a, 'b: 'a, 'tcx: 'b> {
cx: &'a ::core::DocContext<'b, 'tcx>,
pub struct LibEmbargoVisitor<'a, 'tcx: 'a, 'rcx: 'a> {
cx: &'a ::core::DocContext<'a, 'tcx, 'rcx>,
// Accessibility levels for reachable nodes
access_levels: RefMut<'a, AccessLevels<DefId>>,
// Previous accessibility level, None means unreachable
@ -32,8 +32,8 @@ pub struct LibEmbargoVisitor<'a, 'b: 'a, 'tcx: 'b> {
visited_mods: FxHashSet<DefId>,
}
impl<'a, 'b, 'tcx> LibEmbargoVisitor<'a, 'b, 'tcx> {
pub fn new(cx: &'a ::core::DocContext<'b, 'tcx>) -> LibEmbargoVisitor<'a, 'b, 'tcx> {
impl<'a, 'tcx, 'rcx> LibEmbargoVisitor<'a, 'tcx, 'rcx> {
pub fn new(cx: &'a ::core::DocContext<'a, 'tcx, 'rcx>) -> LibEmbargoVisitor<'a, 'tcx, 'rcx> {
LibEmbargoVisitor {
cx,
access_levels: cx.access_levels.borrow_mut(),

60
src/test/rustdoc/intra-links.rs Normal file
View file

@ -0,0 +1,60 @@
// Copyright 2017 The Rust Project Developers. See the COPYRIGHT
// file at the top-level directory of this distribution and at
// http://rust-lang.org/COPYRIGHT.
//
// Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
// http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
// <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your
// option. This file may not be copied, modified, or distributed
// except according to those terms.
// @has intra_links/index.html
// @has - '//a/@href' '../intra_links/struct.ThisType.html'
// @has - '//a/@href' '../intra_links/enum.ThisEnum.html'
// @has - '//a/@href' '../intra_links/trait.ThisTrait.html'
// @has - '//a/@href' '../intra_links/type.ThisAlias.html'
// @has - '//a/@href' '../intra_links/union.ThisUnion.html'
// @has - '//a/@href' '../intra_links/fn.this_function.html'
// @has - '//a/@href' '../intra_links/constant.THIS_CONST.html'
// @has - '//a/@href' '../intra_links/static.THIS_STATIC.html'
// @has - '//a/@href' '../intra_links/macro.this_macro.html'
// @has - '//a/@href' '../intra_links/trait.SoAmbiguous.html'
// @has - '//a/@href' '../intra_links/fn.SoAmbiguous.html'
//! In this crate we would like to link to:
//!
//! * [`ThisType`](ThisType)
//! * [`ThisEnum`](ThisEnum)
//! * [`ThisTrait`](ThisTrait)
//! * [`ThisAlias`](ThisAlias)
//! * [`ThisUnion`](ThisUnion)
//! * [`this_function`](this_function())
//! * [`THIS_CONST`](const@THIS_CONST)
//! * [`THIS_STATIC`](static@THIS_STATIC)
//! * [`this_macro`](this_macro!)
//!
//! In addition, there's some specifics we want to look at. There's [a trait called
//! SoAmbiguous][ambig-trait], but there's also [a function called SoAmbiguous][ambig-fn] too!
//! Whatever shall we do?
//!
//! [ambig-trait]: trait@SoAmbiguous
//! [ambig-fn]: SoAmbiguous()
#[macro_export]
macro_rules! this_macro {
() => {};
}
pub struct ThisType;
pub enum ThisEnum { ThisVariant, }
pub trait ThisTrait {}
pub type ThisAlias = Result<(), ()>;
pub union ThisUnion { this_field: usize, }
pub fn this_function() {}
pub const THIS_CONST: usize = 5usize;
pub static THIS_STATIC: usize = 5usize;
pub trait SoAmbiguous {}
#[allow(bad_style)]
pub fn SoAmbiguous() {}

2
src/tools/error_index_generator/main.rs
View file

@ -100,7 +100,7 @@ impl Formatter for HTMLFormatter {
// Description rendered as markdown.
match info.description {
Some(ref desc) => write!(output, "{}", Markdown(desc, RenderType::Hoedown))?,
Some(ref desc) => write!(output, "{}", Markdown(desc, &[], RenderType::Hoedown))?,
None => write!(output, "<p>No description.</p>\n")?,
}