Pages

Topics

Tools

↑↓navigate↵openescclose
All guides

Rust Macros: macro_rules! Patterns, Hygiene, and Procedural Macros

Matcher fragments, repetition, hygiene and the proc-macro crate. How to read a macro_rules! error and when a derive macro is the right tool.

~4 min readRust
#Rust#Macros#Metaprogramming
3D illustration of token streams expanding into AST syntax trees through a macro hygiene filter

Quick Summary

Rust has two macro systems. Declarative macros (macro_rules!) match token patterns and expand to code at compile time; they need no extra crate and cover most "write this boilerplate for me" cases. Procedural macros are functions that receive a TokenStream and return a TokenStream; they require a separate proc-macro = true crate and are the only way to derive traits or transform items.

macro_rules! is pattern matching over syntax. Each rule is (matcher) => { transcriber }, matchers use fragment specifiers ($x:expr, $name:ident, $t:ty, $($rest:tt)*), and repetition is written $( ... ),*. Hygiene is partial: variables created inside the macro are invisible to the caller, but item names and paths are not, which is why exported macros reference $crate:: instead of crate::.

Procedural macros come in three flavours with distinct attributes: #[proc_macro_derive(Trait)] for derives, #[proc_macro_attribute] for attribute-style wrappers, and #[proc_macro] for function-like invocation. All three work on token streams, so nearly every real implementation pairs them with syn (parsing) and quote (generation).

Basic Syntax & Signature

rust
// Declarative macro: variadic constructor with a trailing comma allowed. macro_rules! map { () => { std::collections::HashMap::new() }; ($($key:expr => $value:expr),* $(,)?) => {{ let mut m = std::collections::HashMap::new(); $( m.insert($key, $value); )* m }}; } // Repetition with an index-free body: builds a Vec of tuples. macro_rules! pairs { ($($k:expr => $v:expr),* $(,)?) => { vec![ $( ($k, $v) ),* ] }; } // A macro that generates an item: item fragment specifiers. macro_rules! make_getter { ($name:ident, $ty:ty) => { pub fn $name(&self) -> &$ty { &self.$name } }; } struct Config { host: String, port: u16 } impl Config { make_getter!(host, String); make_getter!(port, u16); } // Recursive "tt muncher": consumes token trees one at a time. macro_rules! count_idents { () => { 0usize }; ($head:ident $($tail:ident)*) => { 1usize + count_idents!($($tail)*) }; } fn main() { let m = map! { "a" => 1, "b" => 2, }; assert_eq!(m.len(), 2); let p = pairs!(1 => "one", 2 => "two"); assert_eq!(p[1], (2, "two")); let c = Config { host: "localhost".into(), port: 8080 }; assert_eq!(c.host(), "localhost"); assert_eq!(*c.port(), 8080); assert_eq!(count_idents!(a b c d), 4); }

The matcher fragment specifiers, and what they accept:

SpecifierMatchesNotes
$x:exprAn expressionFollow set restricted; cannot be followed by + in older editions
$x:identAn identifier or keywordUsed for names and field access
$x:tyA typeFollow set: {, [, ,, >, =, :, =>, ;, `
$x:pathA pathfoo::bar, crate::x
$x:patA patternpat_param variant when followed by `
$x:stmtA statementRarely needed; hides semicolons
$x:blockA { ... } blockGood for closures and DSL bodies
$x:itemAn itemFunctions, structs, impls
$x:literalA literal tokenStrings, numbers, chars, true
$x:metaAn attribute bodyderive(Debug), cfg(unix)
$x:visA visibilitypub, pub(crate), or empty
$x:ttOne token treeThe workhorse for recursion
$x:lifetime'aFor generic parameters

Repetition syntax: $( ... )* (zero or more), $( ... )+ (one or more), $( ... ),* with a separator, and $(,)? to allow an optional trailing comma. Repeated metavariables must appear in the transcriber with the same nesting: $( $name ),*.

Key Comparison Table

macro_rules!Derive macroAttribute macroFunction-like proc macro
Declared withmacro_rules! name#[proc_macro_derive(T)]#[proc_macro_attribute]#[proc_macro]
LocationAnywhere in the crateproc-macro crate onlyproc-macro crate onlyproc-macro crate only
InputToken patternDeriveInput (parsed)Attribute args + itemRaw TokenStream
RunsExpansion passExpansion passBefore the item is parsedExpansion pass
Typical useShorthand, variadicsImplementing traitsWrapping or rewriting itemsDSLs, codegen
DependencyNonesyn, quotesyn, quotesyn, quote
Can inspect typesNoYes (via syn)YesYes
ErrorsCompile error at match failurecompile_error! / syn::Errorcompile_error!compile_error!
Visible to other cratesOnly with #[macro_export]Yes, if the trait isYesYes

Practical Edge Cases & Deep Dives

1. Hygiene: what is protected and what is not

rust
macro_rules! make_local { () => { let internal_name = 1; internal_name }; } macro_rules! leaky_path { // BAD: resolves `helper` at the CALL site, not the definition site. () => { helper() }; } macro_rules! hygienic_path { // GOOD: `$crate` always points at the defining crate. () => { $crate::helper() }; } fn helper() -> &'static str { "crate helper" } fn main() { let internal_name = 99; // The macro's `internal_name` does not collide with the caller's. assert_eq!(make_local!(), 1); assert_eq!(internal_name, 99); // Callers using the exported macro need no `helper` in scope. assert_eq!(hygienic_path!(), "crate helper"); }

Local variables introduced by macro_rules! carry a syntax context, so they cannot shadow or be shadowed by caller variables. Items, types and paths are not hygienic in the same way — a name inside a macro body resolves where the macro is expanded unless prefixed with $crate::. This is why nearly every published macro crate writes $crate:: in its expansions, and why older crates used #[macro_export(local_inner_macros)].

2. Recursion for a variadic constructor with mixed arms

rust
macro_rules! json_like { // Base case: null. (null) => { serde_value::Value::Null }; // String literal. ($s:literal) => { serde_value::Value::String($s.to_string()) }; // Object: recurse over key: value pairs. ({ $($k:literal : $v:tt),* $(,)? }) => {{ let mut map = std::collections::BTreeMap::new(); $( map.insert($k.to_string(), json_like!($v)); )* serde_value::Value::Map(map) }}; } mod serde_value { #[derive(Debug, PartialEq)] pub enum Value { Null, String(String), Map(std::collections::BTreeMap<String, Value>) } } fn main() { let v = json_like!({ "a": "x", "b": null }); if let serde_value::Value::Map(m) = v { assert_eq!(m.len(), 2); assert_eq!(m["a"], serde_value::Value::String("x".into())); } }

The tt matcher is what makes recursion possible: it accepts exactly one token tree, so a nested { ... } passes through untouched and is re-matched by the object arm. This "tt muncher" pattern is how vec!, println!, and the nom combinators handle arbitrary nesting.

3. A derive macro with syn and quote

rust
// In crate `my_derive`, Cargo.toml: [lib] proc-macro = true use proc_macro::TokenStream; use quote::quote; use syn::{parse_macro_input, DeriveInput}; #[proc_macro_derive(Describe, attributes(describe))] pub fn describe(input: TokenStream) -> TokenStream { let ast = parse_macro_input!(input as DeriveInput); let name = &ast.ident; // Read a helper attribute: #[describe(prefix = "cfg")] let mut prefix = String::new(); for attr in &ast.attrs { if attr.path().is_ident("describe") { let _ = attr.parse_nested_meta(|meta| { if meta.path.is_ident("prefix") { let lit: syn::LitStr = meta.value()?.parse()?; prefix = lit.value(); } Ok(()) }); } } let expanded = quote! { impl #name { pub fn describe(&self) -> String { format!("{} {}", #prefix, stringify!(#name)).trim().to_string() } } }; expanded.into() }
rust
// In the consuming crate: use my_derive::Describe; #[derive(Describe)] #[describe(prefix = "service")] struct Api; fn main() { assert_eq!(Api.describe(), "service Api"); }

quote! interpolates with #var (and #(#items),* for repetition), producing a TokenStream that becomes ordinary tokens in the caller. Note the helper attribute is declared in the derive itself (attributes(describe)); without that, the compiler rejects #[describe(...)] as an unknown attribute.

4. Attribute macro that rewrites an item

rust
use proc_macro::TokenStream; use quote::quote; use syn::{parse_macro_input, ItemFn}; #[proc_macro_attribute] pub fn timed(_attr: TokenStream, item: TokenStream) -> TokenStream { let func = parse_macro_input!(item as ItemFn); let sig = &func.sig; let body = &func.block; let name = &sig.ident; let attrs = &func.attrs; let vis = &func.vis; let expanded = quote! { #(#attrs)* #vis #sig { let __start = std::time::Instant::now(); let __result = (|| #body)(); eprintln!("{} took {:?}", stringify!(#name), __start.elapsed()); __result } }; expanded.into() }
rust
#[timed] fn slow_add(a: u64, b: u64) -> u64 { std::thread::sleep(std::time::Duration::from_millis(5)); a + b } fn main() { assert_eq!(slow_add(1, 2), 3); }

The wrapper must preserve the original signature, visibility, and attributes, otherwise the function silently loses pub or #[inline]. Wrapping the body in a closure keeps ? working inside the body while letting the macro observe the result.

5. Reporting errors from a proc macro instead of panicking

rust
use proc_macro::TokenStream; use quote::quote; use syn::{parse_macro_input, DeriveInput}; #[proc_macro_derive(Validated)] pub fn validated(input: TokenStream) -> TokenStream { let ast = parse_macro_input!(input as DeriveInput); let name = &ast.ident; let ok = matches!(ast.data, syn::Data::Struct(_)); if !ok { // Emit a diagnostic at the item, not a panic. return syn::Error::new_spanned(name, "Validated can only be derived for structs") .to_compile_error() .into(); } quote! { impl #name { pub fn is_valid(&self) -> bool { true } } }.into() }

A panic! inside a proc macro also fails the build, but the message loses span information and points at the macro call rather than the offending field. syn::Error::new_spanned produces a proper clickable diagnostic. For a partially parsed input, parse_macro_input! returns the error as tokens automatically.

6. Debugging macros: cargo expand and stringify!

rust
macro_rules! show { ($($t:tt)*) => { println!("{} = {:?}", stringify!($($t)*), { $($t)* }) }; } fn main() { let x = 1 + 2; show!(x * 3); // prints: x * 3 = 9 }

stringify! turns the tokens back into a string, which is invaluable when a matcher arm is not firing: print the pattern you think matched. cargo expand (install with cargo install cargo-expand) shows the fully expanded code, the fastest way to see why a macro produced code that does not compile. cargo rustc -- -Zunpretty=expanded works on nightly without the extra binary.

Common Compilation/Runtime Mistakes

Error / symptomCauseFix
no rules expected this token in macro callMatcher does not accept the shape givenAdd an arm, or accept tt and recurse
macro expansion ignores token ',' and any followingRepetition separator mismatchUse $( ... ),* matching the transcriber
cannot find macro 'foo' in this scopeMacro defined after use, or not exportedMove the definition up, #[macro_export], or use crate::foo
expected expression, found ';'$x:expr followed by something outside its follow setWrap in a block, or match tt
Cannot use a macro in expression position$x:stmt or item macro used where an expression is requiredRestructure to block or expr
Derive macro not foundproc-macro crate missing, or not in [dependencies]Separate crate with proc-macro = true
Proc macro cannot be used in its own crateCargo forbids self-useMove the derive to a sibling crate
unexpected end of input inside a proc macrosyn parse failure on unfamiliar syntaxReturn syn::Error::to_compile_error()
$crate unresolved in an exported macro2015-edition style pathUse $crate:: and #[macro_export]
Macro expansion yields a huge error cascadeGenerated code lacks spansquote_spanned! with the original span

One habit that prevents most macro bugs: keep the matcher narrow and the transcriber deliberately parenthesized. A transcriber that returns { let a = 1; a } protects callers from operator-precedence surprises; a transcriber that returns a bare expression lets macro!() + 1 associate in ways the author never intended.

Try matching browser tools

All tools

Small utilities that fit Rust work — they run entirely in this browser tab, so nothing you paste leaves your device.

All Rust guides

More in Rust

All Rust guides

The guides filed next to this one in Rust, in library order.