Dynamic Code Generation in Rust

Hi there, fellow Rustaceans! 🦀

Luis Soares

01 Feb 2024 — 8 min read

Hi there, fellow Rustaceans! 🦀

In today’s article, we’ll dive into Rust's mechanisms for code generation, including macros, procedural macros, and code generation during build time.

We’ll see some detailed examples and explanations to help you master code generation in Rust.

Introduction to Macros in Rust

Macros in Rust allow you to write code that writes other code, which is known as metaprogramming. They are a powerful feature used to reduce code repetition and improve maintainability.

Declarative Macros

Declarative macros, defined with the macro_rules! macro, allow pattern matching on the code provided to the macro and are used to perform syntactic manipulations.

Example: Implementing a `vec!` macro

macro_rules! vec { 
    ( $( $x:expr ),* ) => { 
        { 
            let mut temp_vec = Vec::new(); 
            $( 
                temp_vec.push($x); 
            )* 
            temp_vec 
        } 
    }; 
} 
 
fn main() { 
    let a = vec![1, 2, 3]; 
    println!("{:?}", a); // Output: [1, 2, 3] 
}

In this example, $( $x:expr ),* matches zero or more expressions separated by commas. The $(...)* construct is then used to repeat the enclosed block of code for each matched expression.

Procedural Macros

Procedural macros are more powerful and flexible than declarative macros. They allow you to operate on the Rust Abstract Syntax Tree (AST), enabling transformations that are not possible with declarative macros.

Example: Implementing a simple `derive` macro

Let’s create a custom derive macro for a Debug trait that generates a simple debug representation for structs.

Add Dependencies: First, add proc-macro2, quote, and syn to your Cargo.toml for working with Rust's AST and generating code.
Define the Macro: In a new crate of type proc-macro, define your procedural macro.

extern crate proc_macro; 
 
use proc_macro::TokenStream; 
use quote::quote; 
use syn::{parse_macro_input, DeriveInput}; 
 
#[proc_macro_derive(SimpleDebug)] 
pub fn simple_debug_derive(input: TokenStream) -> TokenStream { 
    let ast = parse_macro_input!(input as DeriveInput); 
    let name = &ast.ident; 
    let gen = quote! { 
        impl std::fmt::Debug for #name { 
            fn fmt(&self, f: &mut std::fmt::Formatter) -> std::fmt::Result { 
                write!(f, stringify!(#name)) 
            } 
        } 
    }; 
    gen.into() 
}

In this example, #[proc_macro_derive(SimpleDebug)] defines a derive macro. The quote! macro is used to generate Rust code from the provided tokens, and stringify! converts the struct name into a string.

Code Generation During Build Time

Rust’s build script (build.rs) can be used to generate code during build time. This is useful for including generated files into your source code or for embedding resources.

Example: Generating Code with `build.rs`

Create build.rs: In your project root, create a build.rs file.
Write the Build Script: Generate code and write it to a file that can be included in your project.

use std::env; 
use std::fs::File; 
use std::io::Write; 
use std::path::Path; 
 
fn main() { 
    let out_dir = env::var("OUT_DIR").unwrap(); 
    let dest_path = Path::new(&out_dir).join("hello.rs"); 
    let mut f = File::create(&dest_path).unwrap(); 
    f.write_all(b" 
        pub fn say_hello() { 
            println!(\"Hello, world!\"); 
        } 
    ").unwrap(); 
}

Include the Generated Code: Use the include! macro to include the generated code in your Rust file.

include!(concat!(env!("OUT_DIR"), "/hello.rs")); 
 
fn main() { 
    say_hello(); // This function is generated by build.rs 
}

Advanced Code Generation Techniques in Rust

Taking our exploration of Rust’s code generation capabilities further, we delve into more advanced techniques that cater to complex scenarios and enhance the robustness and efficiency of Rust applications.

Attribute Macros

Attribute macros are similar to procedural macros but are used to annotate functions, structs, enums, or other items. They can inspect, modify, or generate additional code based on the annotated item.

Example: Logging Attribute Macro

Imagine an attribute macro #[log_args] that logs the arguments passed to a function.

Define the Macro: In your proc-macro crate, define the attribute macro.

use proc_macro::TokenStream; 
use quote::quote; 
use syn::{parse_macro_input, ItemFn}; 
 
#[proc_macro_attribute] 
pub fn log_args(_attr: TokenStream, item: TokenStream) -> TokenStream { 
    let input_fn = parse_macro_input!(item as ItemFn); 
    let fn_name = &input_fn.sig.ident; 
    let fn_body = &input_fn.block; 
    let fn_args = input_fn.sig.inputs.iter().map(|arg| { 
        quote! { std::format!("{:?}", #arg) } 
    }); 
 
    let gen = quote! { 
        #input_fn 
        fn #fn_name(#fn_args) { 
            println!("Called with args: {:?}", vec![#(#fn_args),*]); 
            #fn_body 
        } 
    }; 
 
    gen.into() 
}

In this example, the macro parses the input function, and the quote! macro generates a new function that logs its arguments before calling the original function.

Derive Macros for Custom Traits

Derive macros can be extended to implement custom traits for structs or enums, enabling automatic trait implementation based on the struct’s fields.

Example: Custom `ToJson` Trait

Assume you have a custom trait ToJson that converts a struct into a JSON string.

Define the Trait: In your main crate, define the ToJson trait.

pub trait ToJson { 
    fn to_json(&self) -> String; 
}

Implement the Derive Macro: In your proc-macro crate, implement the derive macro for ToJson.

use proc_macro::TokenStream; 
use quote::quote; 
use syn::{parse_macro_input, DeriveInput, Data}; 
 
#[proc_macro_derive(ToJson)] 
pub fn to_json_derive(input: TokenStream) -> TokenStream { 
    let ast = parse_macro_input!(input as DeriveInput); 
    let name = &ast.ident; 
    let fields = if let Data::Struct(data) = &ast.data { 
        data.fields.iter().map(|f| { 
            let name = &f.ident; 
            quote! { format!("\"{}\": \"{:?}\"", stringify!(#name), &self.#name) } 
        }) 
    } else { 
        panic!("ToJson can only be derived for structs"); 
    }; 
 
    let gen = quote! { 
        impl ToJson for #name { 
            fn to_json(&self) -> String { 
                format!("{{ {} }}", vec![#(#fields),*].join(", ")) 
            } 
        } 
    }; 
 
    gen.into() 
}

This macro implementation iterates over struct fields, generating a JSON string representation for each field.

Generating Code with External Tools

In some cases, you might need to generate Rust code from external tools or files, such as protocol buffers (Protobuf) or interface definition languages (IDLs).

Example: Generating Rust Code from Protobuf

Add build.rs and Dependencies: Use prost-build in your build.rs script to compile Protobuf files into Rust code.

// In build.rs 
extern crate prost_build; 
 
fn main() { 
    prost_build::compile_protos(&["src/your.proto"], &["src/"]).unwrap(); 
}

Include Generated Code: Use the include! macro to include the generated code in your Rust application.

mod your_proto { 
    include!(concat!(env!("OUT_DIR"), "/your_proto.rs")); 
} 
 
fn main() { 
    // Use the generated code from your_proto module 
}

This approach ensures your Rust codebase remains synchronized with your Protobuf definitions, enhancing maintainability and reducing the risk of errors.

Advanced Procedural Macros for Domain-Specific Languages (DSLs)

Rust’s procedural macros can be used to create Domain-Specific Languages (DSLs) within Rust, enabling expressive and concise syntax for specific domains.

Example: HTML Templating DSL

Imagine we want to create a DSL for generating HTML content. This DSL will allow users to write HTML-like code in Rust, which gets translated into actual HTML strings at compile time.

Define the DSL Syntax: First, we need to define the syntax for our DSL. Let’s keep it simple for this example:

html! { 
    <div id="main" class="container"> 
        <h1>"Hello, World!"</h1> 
        <p>"This is a paragraph."</p> 
    </div> 
}

Implement the Procedural Macro: In your proc-macro crate, implement the macro that parses this syntax and generates the corresponding HTML string.

use proc_macro::TokenStream; 
use quote::quote; 
use syn::{parse_macro_input, LitStr, Token}; 
use syn::parse::{Parse, ParseStream, Result}; 
 
struct HtmlNode { 
    tag: String, 
    attributes: Vec<(String, String)>, 
    children: Vec<HtmlNode>, 
    content: Option<String>, 
} 
 
impl Parse for HtmlNode { 
    fn parse(input: ParseStream) -> Result<Self> { 
        // Simplified parsing logic for demonstration 
        // In practice, you would need a more robust parser 
    } 
} 
 
#[proc_macro] 
pub fn html(input: TokenStream) -> TokenStream { 
    let root_node = parse_macro_input!(input as HtmlNode); 
 
    // Function to recursively generate HTML from nodes 
    fn generate_html(node: &HtmlNode) -> proc_macro2::TokenStream { 
        let tag = &node.tag; 
        let content = node.content.as_ref().map(|c| quote! { #c }).unwrap_or_default(); 
        // Simplified: Not handling children or attributes for brevity 
        quote! { 
            format!("<{tag}>{content}</{tag}>") 
        } 
    } 
 
    let gen_html = generate_html(&root_node); 
    let gen = quote! { 
        #gen_html 
    }; 
 
    gen.into() 
}

This macro takes the DSL input, parses it into an internal representation (HtmlNode in this example), and then generates the corresponding HTML string.

Compile-time Code Generation for Configuration Files

Rust can also generate code from configuration files at compile time, making it easier to manage application configurations.

Example: Generating Code from JSON Configuration

Suppose we have a JSON file config.json that contains application configuration parameters.

{ 
    "api_url": "https://example.com/api", 
    "retry_attempts": 5 
}

Read and Parse JSON in build.rs: Use serde_json to read and parse the JSON file during the build process.

// In build.rs 
use std::env; 
use std::fs::{self, File}; 
use std::io::Write; 
use std::path::Path; 
 
fn main() { 
    let out_dir = env::var("OUT_DIR").unwrap(); 
    let config_path = Path::new(&out_dir).join("config.rs"); 
    let mut config_file = File::create(&config_path).unwrap(); 
 
    let json_content = fs::read_to_string("config.json").unwrap(); 
    let config: serde_json::Value = serde_json::from_str(&json_content).unwrap(); 
 
    let api_url = config["api_url"].as_str().unwrap(); 
    let retry_attempts = config["retry_attempts"].as_i64().unwrap(); 
 
    write!( 
        config_file, 
        " 
        pub const API_URL: &str = {:?}; 
        pub const RETRY_ATTEMPTS: i64 = {}; 
        ", 
        api_url, retry_attempts 
    ) 
    .unwrap(); 
}

Include the Generated Code: Use the generated constants in your Rust application.

mod config { 
    include!(concat!(env!("OUT_DIR"), "/config.rs")); 
} 
 
fn main() { 
    println!("API URL: {}", config::API_URL); 
    println!("Retry Attempts: {}", config::RETRY_ATTEMPTS); 
}

This setup allows you to maintain your application configuration in a separate JSON file, with changes reflected in the code at compile time.

Custom Derive Macros for Data Validation

Rust’s derive macros can be extended to implement custom validation logic for structs, ensuring data integrity at compile time.

Example: Validate Struct Fields

Suppose we want to ensure that a struct representing a user has valid email and age fields.

Define the Validation Traits: In your main crate, define traits for validation.

pub trait ValidateEmail { 
    fn validate_email(&self) -> bool; 
} 
 
pub trait ValidateAge { 
    fn validate_age(&self) -> bool; 
}

Implement the Derive Macro: In your proc-macro crate, implement the derive macro that generates the validation logic.

use proc_macro::TokenStream; 
use quote::quote; 
use syn::{parse_macro_input, DeriveInput}; 
 
#[proc_macro_derive(Validate)] 
pub fn validate_derive(input: TokenStream) -> TokenStream { 
    let ast = parse_macro_input!(input as DeriveInput); 
    let name = &ast.ident; 
 
    // Simplified: Assumes the struct has `email` and `age` fields 
    let gen = quote! { 
        impl ValidateEmail for #name { 
            fn validate_email(&self) -> bool { 
                // Simplified email validation logic 
                self.email.contains('@') 
            } 
        } 
 
        impl ValidateAge for #name { 
            fn validate_age(&self) -> bool { 
                self.age > 0 && self.age < 130 
            } 
        } 
    }; 
 
    gen.into() 
}

Use the Generated Validation Logic: Apply the custom derive to your structs.

#[derive(Validate)] 
struct User { 
    email: String, 
    age: i32, 
} 
 
fn main() { 
    let user = User { 
        email: String::from("user@example.com"), 
        age: 30, 
    }; 
 
    assert!(user.validate_email()); 
    assert!(user.validate_age()); 
}

This approach enables you to define complex validation logic in a centralized manner, improving code maintainability and reducing the risk of errors.

🚀 Explore More by Luis Soares

📚 Learning Hub: Expand your knowledge in various tech domains, including Rust, Software Development, Cloud Computing, Cyber Security, Blockchain, and Linux, through my extensive resource collection:

Hands-On Tutorials with GitHub Repos: Gain practical skills across different technologies with step-by-step tutorials, complemented by dedicated GitHub repositories. Access Tutorials
In-Depth Guides & Articles: Deep dive into core concepts of Rust, Software Development, Cloud Computing, and more, with detailed guides and articles filled with practical examples. Read More
E-Books Collection: Enhance your understanding of various tech fields with a series of free e-Books, including titles like “Mastering Rust Ownership” and “Application Security Guide” Download eBook
Project Showcases: Discover a range of fully functional projects across different domains, such as an API Gateway, Blockchain Network, Cyber Security Tools, Cloud Services, and more. View Projects
LinkedIn Newsletter: Stay ahead in the fast-evolving tech landscape with regular updates and insights on Rust, Software Development, and emerging technologies by subscribing to my newsletter on LinkedIn. Subscribe Here

🔗 Connect with Me:

Medium: Read my articles on Medium and give claps if you find them helpful. It motivates me to keep writing and sharing Rust content. Follow on Medium
Personal Blog: Discover more on my personal blog, a hub for all my Rust-related content. Visit Blog
LinkedIn: Join my professional network for more insightful discussions and updates. Connect on LinkedIn
Twitter: Follow me on Twitter for quick updates and thoughts on Rust programming. Follow on Twitter

Wanna talk? Leave a comment or drop me a message!

All the best,

Luis Soares
luis.soares@linux.com

Dynamic Code Generation in Rust

Luis Soares

Introduction to Macros in Rust

Declarative Macros

Example: Implementing a `vec!` macro

Procedural Macros

Example: Implementing a simple `derive` macro

Code Generation During Build Time

Example: Generating Code with `build.rs`

Advanced Code Generation Techniques in Rust

Attribute Macros

Example: Logging Attribute Macro

Derive Macros for Custom Traits

Example: Custom `ToJson` Trait

Generating Code with External Tools

Example: Generating Rust Code from Protobuf

Advanced Procedural Macros for Domain-Specific Languages (DSLs)

Example: HTML Templating DSL

Compile-time Code Generation for Configuration Files

Example: Generating Code from JSON Configuration

Custom Derive Macros for Data Validation

Example: Validate Struct Fields

🚀 Explore More by Luis Soares

Read more

Dynamic Linking and Memory Relocations in Rust

Building an Error Correction System in Rust

Rust Lifetimes Made Simple

Rust: From Simple Functions to Advanced Abstractions

Introduction to Macros in Rust

Declarative Macros

Example: Implementing a vec! macro

Procedural Macros

Example: Implementing a simple derive macro

Code Generation During Build Time

Example: Generating Code with build.rs

Advanced Code Generation Techniques in Rust

Attribute Macros

Example: Logging Attribute Macro

Derive Macros for Custom Traits

Example: Custom ToJson Trait

Generating Code with External Tools

Example: Generating Rust Code from Protobuf

Advanced Procedural Macros for Domain-Specific Languages (DSLs)

Example: HTML Templating DSL

Compile-time Code Generation for Configuration Files

Example: Generating Code from JSON Configuration

Custom Derive Macros for Data Validation

Example: Validate Struct Fields

🚀 Explore More by Luis Soares

Read more

Dynamic Linking and Memory Relocations in Rust

Building an Error Correction System in Rust

Rust Lifetimes Made Simple

Rust: From Simple Functions to Advanced Abstractions

Example: Implementing a `vec!` macro

Example: Implementing a simple `derive` macro

Example: Generating Code with `build.rs`

Example: Custom `ToJson` Trait