Metaprogramming & Evolution

Metaprogramming & Evolution in C#

Master advanced C# metaprogramming techniques with free flashcards and spaced repetition practice. This lesson covers reflection, dynamic code generation, expression trees, and source generators—essential concepts for building flexible, evolving software architectures that can inspect and modify themselves at runtime.

Welcome to the World of Code That Writes Code 💻

Imagine writing a program that can examine its own structure, modify its behavior at runtime, or even generate new code on the fly. This isn't science fiction—it's metaprogramming, one of C#'s most powerful advanced features. Whether you're building serialization frameworks, dependency injection containers, or object-relational mappers, metaprogramming is the secret ingredient that makes magic happen.

In this lesson, we'll explore how C# allows your code to transcend its static definition and become truly dynamic and self-aware. 🚀

Core Concepts: The Foundation of Metaprogramming

What is Metaprogramming? 🤔

Metaprogramming is the practice of writing code that treats other code as data. Instead of just executing instructions, metaprograms can:

• Inspect code structure (classes, methods, properties)
• Generate new code programmatically
• Modify behavior at runtime
• Analyze code patterns and relationships

Think of it like this: normal programming is like following a recipe, while metaprogramming is like writing a cookbook generator that creates recipes based on available ingredients.

The Three Pillars of C# Metaprogramming

Technique	Purpose	Performance	Use Case
Reflection	Inspect types at runtime	Slow (runtime overhead)	Serialization, plugins
Expression Trees	Represent code as data	Medium (compilation cost)	ORMs, LINQ providers
Source Generators	Generate code at compile-time	Fast (no runtime cost)	Serializers, DI containers

Reflection: The Runtime Inspector 🔍

Reflection is C#'s built-in system for examining and manipulating type information at runtime. The System.Reflection namespace provides classes like Type, MethodInfo, PropertyInfo, and FieldInfo that let you discover what types exist and what members they contain.

Key capabilities:

• Discover all properties of a class
• Invoke methods by name
• Create instances without knowing the type at compile-time
• Read and write private fields (use cautiously!)

💡 Tip: Reflection is powerful but slow. Cache MethodInfo and PropertyInfo objects when possible, or use expression trees for repeated operations.

Reflection Process Flow:

    Your Code
        |
        ↓
    Type.GetType()
        |
        ↓
    Type Object (metadata)
        |
    ┌───┴───┐
    ↓       ↓
Methods  Properties  ← Discover members
    ↓       ↓
Invoke  Get/Set     ← Perform operations

Real-world scenario: JSON serializers use reflection to discover all properties of an object and convert them to JSON format without knowing the type ahead of time.

Expression Trees: Code as Data Structures 🌳

An expression tree is a data structure that represents code in a tree format where each node is an expression. Instead of executing immediately, the code structure is preserved so you can analyze, modify, or translate it.

Structure of an expression tree:

Expression: x => x * 2 + 1

Tree representation:

        (+)
       /   \
     (*)    (1)
    /   \
  (x)   (2)

Each node is an Expression object:
- BinaryExpression for + and *
- ParameterExpression for x
- ConstantExpression for 1 and 2

Why use expression trees?

• LINQ to SQL/EF: Converts C# queries to SQL
• Dynamic query builders: Construct filters at runtime
• Rule engines: Define business rules as data
• Optimization: Analyze code before execution

💡 Memory trick: Expression trees are like abstract syntax trees (AST) that you can manipulate. Think "Expression = Executable Syntax Tree"

Dynamic Code Generation: Creating Code at Runtime ⚡

C# offers multiple approaches to generating code:

1. Reflection.Emit (IL generation)

• Generate MSIL bytecode directly
• Maximum performance when done right
• Complex and error-prone

2. CodeDOM (legacy approach)

• Generate C# source code as objects
• Compile with Roslyn
• Outdated, use Source Generators instead

3. Source Generators (modern, preferred)

• Roslyn analyzers that run during compilation
• Generate additional source files
• No runtime overhead
• Integrated with IDE (IntelliSense works!)

The dynamic Keyword: Bypassing Type Checking 🔓

The dynamic keyword tells C# to skip compile-time type checking and resolve everything at runtime using the Dynamic Language Runtime (DLR).

When to use dynamic:

• Interoperating with COM objects (Office automation)
• Working with IronPython or other dynamic languages
• Deserializing JSON with unknown structure
• Simplifying reflection code

⚠️ Warning: Overusing dynamic sacrifices type safety and IntelliSense. Use it sparingly and only when necessary!

Compile-time Type System vs. Runtime Dynamic Dispatch

┌──────────────────────────────────────────────┐
│  STATIC (string, int, MyClass)               │
│  ✓ Compile-time checking                     │
│  ✓ IntelliSense                              │
│  ✓ Performance                               │
│  ✗ Must know types ahead                     │
└──────────────────────────────────────────────┘

┌──────────────────────────────────────────────┐
│  DYNAMIC (dynamic)                           │
│  ✗ No compile-time checking                  │
│  ✗ No IntelliSense                           │
│  ✗ Slower (runtime dispatch)                 │
│  ✓ Works with unknown types                  │
└──────────────────────────────────────────────┘

Practical Examples: Metaprogramming in Action

Example 1: Simple Reflection - Property Inspector 🔍

Let's build a utility that displays all properties of any object:

using System;
using System.Reflection;

public class PropertyInspector
{
    public static void DisplayProperties(object obj)
    {
        if (obj == null)
        {
            Console.WriteLine("Object is null");
            return;
        }

        Type type = obj.GetType();
        Console.WriteLine($"Type: {type.Name}\n");

        // Get all public instance properties
        PropertyInfo[] properties = type.GetProperties(
            BindingFlags.Public | BindingFlags.Instance
        );

        foreach (PropertyInfo prop in properties)
        {
            // Get the property value
            object value = prop.GetValue(obj);
            Console.WriteLine($"{prop.Name} ({prop.PropertyType.Name}): {value}");
        }
    }
}

// Usage example
public class Person
{
    public string Name { get; set; }
    public int Age { get; set; }
    public string Email { get; set; }
}

class Program
{
    static void Main()
    {
        var person = new Person 
        { 
            Name = "Alice", 
            Age = 30, 
            Email = "alice@example.com" 
        };

        PropertyInspector.DisplayProperties(person);
        
        /* Output:
        Type: Person
        
        Name (String): Alice
        Age (Int32): 30
        Email (String): alice@example.com
        */
    }
}

How it works:

1. GetType() retrieves the runtime type information
2. GetProperties() returns an array of PropertyInfo objects
3. BindingFlags controls which properties to include (public, private, static, etc.)
4. GetValue(obj) reads the property value from the specific object instance

💡 Real-world use: This pattern powers debugging tools, logging frameworks, and object dump utilities.

Example 2: Expression Trees - Fast Property Accessor 🚀

Reflection is slow for repeated property access. Let's use expression trees to create a fast, compiled property accessor:

using System;
using System.Linq.Expressions;
using System.Reflection;

public static class FastPropertyAccessor
{
    public static Func<T, object> CreateGetter<T>(string propertyName)
    {
        Type type = typeof(T);
        PropertyInfo property = type.GetProperty(propertyName);
        
        if (property == null)
            throw new ArgumentException($"Property {propertyName} not found");

        // Create parameter: T instance
        ParameterExpression parameter = Expression.Parameter(type, "instance");
        
        // Create expression: instance.PropertyName
        MemberExpression propertyAccess = Expression.Property(parameter, property);
        
        // Convert to object if needed
        UnaryExpression conversion = Expression.Convert(propertyAccess, typeof(object));
        
        // Compile into a delegate: (T instance) => (object)instance.PropertyName
        Expression<Func<T, object>> lambda = Expression.Lambda<Func<T, object>>(
            conversion, 
            parameter
        );
        
        return lambda.Compile();
    }
}

// Usage example
class Product
{
    public string Name { get; set; }
    public decimal Price { get; set; }
}

class Program
{
    static void Main()
    {
        // Create the fast accessor once
        var getPrice = FastPropertyAccessor.CreateGetter<Product>("Price");
        
        var product = new Product { Name = "Laptop", Price = 999.99m };
        
        // Now we can access the property very quickly (compiled code!)
        object price = getPrice(product);
        Console.WriteLine($"Price: {price}"); // Price: 999.99
        
        // 10-100x faster than reflection for repeated access!
    }
}

Breaking it down:

Step	Code	What It Creates
1	Expression.Parameter	The input parameter (instance)
2	Expression.Property	Property access expression
3	Expression.Convert	Type conversion to object
4	Expression.Lambda	Complete lambda expression
5	Compile()	Executable delegate

💡 Performance tip: Expression compilation is expensive (milliseconds), but the resulting delegate is as fast as hand-written code. Create once, reuse many times!

Example 3: Source Generators - Auto-implementing ToString 📝

Source generators run during compilation to add new source files to your project. Here's a simple generator that creates ToString() implementations:

using Microsoft.CodeAnalysis;
using Microsoft.CodeAnalysis.CSharp.Syntax;
using Microsoft.CodeAnalysis.Text;
using System.Collections.Generic;
using System.Linq;
using System.Text;

[Generator]
public class ToStringGenerator : ISourceGenerator
{
    public void Initialize(GeneratorInitializationContext context)
    {
        // Register a syntax receiver to find classes with our attribute
        context.RegisterForSyntaxNotifications(() => new ToStringReceiver());
    }

    public void Execute(GeneratorExecutionContext context)
    {
        if (context.SyntaxReceiver is not ToStringReceiver receiver)
            return;

        foreach (var classDeclaration in receiver.CandidateClasses)
        {
            var model = context.Compilation.GetSemanticModel(classDeclaration.SyntaxTree);
            var classSymbol = model.GetDeclaredSymbol(classDeclaration);
            
            if (classSymbol == null)
                continue;

            string namespaceName = classSymbol.ContainingNamespace.ToDisplayString();
            string className = classSymbol.Name;
            
            // Get all public properties
            var properties = classSymbol.GetMembers()
                .OfType<IPropertySymbol>()
                .Where(p => p.DeclaredAccessibility == Accessibility.Public);

            // Generate the ToString method
            var source = GenerateToStringCode(namespaceName, className, properties);
            
            // Add the generated source to the compilation
            context.AddSource($"{className}_ToString.g.cs", 
                SourceText.From(source, Encoding.UTF8));
        }
    }

    private string GenerateToStringCode(
        string namespaceName, 
        string className, 
        IEnumerable<IPropertySymbol> properties)
    {
        var sb = new StringBuilder();
        sb.AppendLine("// Auto-generated code");
        sb.AppendLine($"namespace {namespaceName}");
        sb.AppendLine("{");
        sb.AppendLine($"    public partial class {className}");
        sb.AppendLine("    {");
        sb.AppendLine("        public override string ToString()");
        sb.AppendLine("        {");
        sb.Append("            return $\"");
        sb.Append($"{className} {{ ");
        
        var propList = properties.Select(p => $"{p.Name} = {{{p.Name}}}");
        sb.Append(string.Join(", ", propList));
        
        sb.AppendLine(" }\";" );
        sb.AppendLine("        }");
        sb.AppendLine("    }");
        sb.AppendLine("}");
        
        return sb.ToString();
    }
}

class ToStringReceiver : ISyntaxReceiver
{
    public List<ClassDeclarationSyntax> CandidateClasses { get; } = new();

    public void OnVisitSyntaxNode(SyntaxNode syntaxNode)
    {
        // Find classes marked with [GenerateToString] attribute
        if (syntaxNode is ClassDeclarationSyntax classDecl &&
            classDecl.AttributeLists.Any())
        {
            CandidateClasses.Add(classDecl);
        }
    }
}

Using the generator:

[GenerateToString]
public partial class Customer
{
    public string Name { get; set; }
    public int Id { get; set; }
    public string Email { get; set; }
}

// The generator creates this code automatically:
// public partial class Customer
// {
//     public override string ToString()
//     {
//         return $"Customer { Name = {Name}, Id = {Id}, Email = {Email} }";
//     }
// }

var customer = new Customer { Name = "Bob", Id = 42, Email = "bob@test.com" };
Console.WriteLine(customer);
// Output: Customer { Name = Bob, Id = 42, Email = bob@test.com }

Key advantages of source generators:

• ✅ Zero runtime overhead (code exists at compile-time)
• ✅ IntelliSense works on generated code
• ✅ Easy to debug (can view generated files)
• ✅ Integrates with build process

Example 4: Dynamic Object Creation - Plugin System 🔌

Let's build a simple plugin system that loads and instantiates types at runtime:

using System;
using System.IO;
using System.Reflection;
using System.Linq;

public interface IPlugin
{
    string Name { get; }
    void Execute();
}

public class PluginLoader
{
    public static IPlugin[] LoadPlugins(string pluginDirectory)
    {
        if (!Directory.Exists(pluginDirectory))
            return Array.Empty<IPlugin>();

        // Find all DLL files in the directory
        var dllFiles = Directory.GetFiles(pluginDirectory, "*.dll");
        var plugins = new System.Collections.Generic.List<IPlugin>();

        foreach (var dllPath in dllFiles)
        {
            try
            {
                // Load the assembly
                Assembly assembly = Assembly.LoadFrom(dllPath);
                
                // Find all types that implement IPlugin
                var pluginTypes = assembly.GetTypes()
                    .Where(t => typeof(IPlugin).IsAssignableFrom(t) && 
                               !t.IsInterface && 
                               !t.IsAbstract);

                // Create instances of each plugin type
                foreach (var type in pluginTypes)
                {
                    // Use Activator.CreateInstance for dynamic instantiation
                    var instance = Activator.CreateInstance(type) as IPlugin;
                    
                    if (instance != null)
                    {
                        plugins.Add(instance);
                        Console.WriteLine($"Loaded plugin: {instance.Name}");
                    }
                }
            }
            catch (Exception ex)
            {
                Console.WriteLine($"Error loading {dllPath}: {ex.Message}");
            }
        }

        return plugins.ToArray();
    }
}

// Example plugin implementation (in separate DLL)
public class EmailPlugin : IPlugin
{
    public string Name => "Email Sender";
    
    public void Execute()
    {
        Console.WriteLine("Sending email...");
    }
}

public class LoggingPlugin : IPlugin
{
    public string Name => "File Logger";
    
    public void Execute()
    {
        Console.WriteLine("Writing to log file...");
    }
}

// Main application
class Program
{
    static void Main()
    {
        string pluginPath = @"C:\MyApp\Plugins";
        
        IPlugin[] plugins = PluginLoader.LoadPlugins(pluginPath);
        
        Console.WriteLine($"\nFound {plugins.Length} plugin(s)\n");
        
        foreach (var plugin in plugins)
        {
            Console.WriteLine($"Executing: {plugin.Name}");
            plugin.Execute();
            Console.WriteLine();
        }
    }
}

How the plugin system works:

┌─────────────────────────────────────────────┐
│  Main Application                           │
│  ┌───────────────────────────────────────┐  │
│  │  PluginLoader.LoadPlugins()          │  │
│  └─────────────┬─────────────────────────┘  │
│                ↓                            │
│  ┌─────────────────────────────────────┐   │
│  │ Scan directory for DLL files        │   │
│  └─────────────┬───────────────────────┘   │
│                ↓                            │
│  ┌─────────────────────────────────────┐   │
│  │ Assembly.LoadFrom(dll)              │   │
│  └─────────────┬───────────────────────┘   │
│                ↓                            │
│  ┌─────────────────────────────────────┐   │
│  │ Find types implementing IPlugin     │   │
│  └─────────────┬───────────────────────┘   │
│                ↓                            │
│  ┌─────────────────────────────────────┐   │
│  │ Activator.CreateInstance(type)      │   │
│  └─────────────┬───────────────────────┘   │
│                ↓                            │
│  ┌─────────────────────────────────────┐   │
│  │ Cast to IPlugin and use             │   │
│  └─────────────────────────────────────┘   │
└─────────────────────────────────────────────┘

   Plugin DLLs (EmailPlugin.dll, etc.)
   Created independently, dropped in folder

💡 Did you know? This is how Visual Studio Code loads extensions, web browsers load add-ons, and game engines load mods!

Common Mistakes to Avoid ⚠️

Mistake 1: Forgetting to Cache Reflection Results

❌ WRONG:

for (int i = 0; i < 10000; i++)
{
    // This gets the property info 10,000 times!
    PropertyInfo prop = typeof(MyClass).GetProperty("Name");
    prop.SetValue(obj, $"Name{i}");
}

✅ RIGHT:

// Get it once and reuse
PropertyInfo prop = typeof(MyClass).GetProperty("Name");

for (int i = 0; i < 10000; i++)
{
    prop.SetValue(obj, $"Name{i}");
}

Why it matters: GetProperty() uses expensive lookups. Caching can improve performance 100x or more!

Mistake 2: Not Handling Expression Tree Compilation Costs

❌ WRONG:

public object GetValue<T>(T obj, string propertyName)
{
    // Compiles a new delegate every time!
    var getter = CreateGetter<T>(propertyName);
    return getter(obj);
}

✅ RIGHT:

private static Dictionary<string, Delegate> _getterCache = new();

public object GetValue<T>(T obj, string propertyName)
{
    string key = $"{typeof(T).FullName}.{propertyName}";
    
    if (!_getterCache.TryGetValue(key, out var getter))
    {
        getter = CreateGetter<T>(propertyName);
        _getterCache[key] = getter;
    }
    
    return ((Func<T, object>)getter)(obj);
}

Mistake 3: Overusing dynamic When Static Types Work

❌ WRONG:

dynamic config = GetConfiguration();
string value = config.Database.ConnectionString; // No IntelliSense, runtime errors!

✅ RIGHT:

Configuration config = GetConfiguration();
string value = config.Database.ConnectionString; // Type-safe, compile-time checked

Rule of thumb: Only use dynamic when you genuinely don't know the type at compile-time (JSON deserialization, COM interop).

Mistake 4: Forgetting BindingFlags in Reflection

❌ WRONG:

// This only gets PUBLIC instance properties!
var properties = typeof(MyClass).GetProperties();

✅ RIGHT:

// Be explicit about what you want
var properties = typeof(MyClass).GetProperties(
    BindingFlags.Public | 
    BindingFlags.NonPublic | 
    BindingFlags.Instance | 
    BindingFlags.Static
);

Mistake 5: Not Making Source Generator Classes Partial

❌ WRONG:

[GenerateToString]
public class Customer // Missing 'partial' keyword!
{
    public string Name { get; set; }
}
// Compiler error: Customer already defines ToString()

✅ RIGHT:

[GenerateToString]
public partial class Customer // Now the generator can add members
{
    public string Name { get; set; }
}

Evolution: From Reflection to Source Generators 📈

C# metaprogramming has evolved significantly over the years:

┌──────────────────────────────────────────────────────────┐
│             C# METAPROGRAMMING TIMELINE                  │
├──────────────────────────────────────────────────────────┤
│                                                          │
│  2002  2008  2010  2012  2020                           │
│   │     │     │     │     │                            │
│   ▼     ▼     ▼     ▼     ▼                            │
│  🔍   🌳   💡   ⚡   🚀                                 │
│  C# 1  C# 3  C# 4  Roslyn Source                        │
│  Ref-  Expr- dyna-  APIs   Gener-                       │
│  lect- ession mic          ators                         │
│  ion   Trees                                             │
│                                                          │
│  Slow → Medium → Flexible → Analyzable → Fast           │
│                                                          │
└──────────────────────────────────────────────────────────┘

The progression:

1. Reflection (2002): First way to inspect types at runtime

   • Pros: Simple API, works everywhere
   • Cons: Slow, runtime-only

2. Expression Trees (2008): Code as data structures

   • Pros: Can compile to fast delegates, analyzable
   • Cons: Complex API, compilation overhead

3. Dynamic (2010): Runtime type resolution

   • Pros: Simplifies interop scenarios
   • Cons: No type safety, loses tooling support

4. Roslyn APIs (2012): Full compiler as a service

   • Pros: Complete code analysis and generation
   • Cons: Complex, large API surface

5. Source Generators (2020): Compile-time code generation

   • Pros: Zero runtime cost, full IDE integration
   • Cons: More complex build setup

💡 Modern best practice: Use Source Generators for code generation, Expression Trees for dynamic operations, and Reflection only when absolutely necessary.

Key Takeaways 🎯

📚 Further Study

• Microsoft Docs: Reflection in .NET
• Expression Trees Documentation
• Source Generators Cookbook

---

🎓 Congratulations! You now understand the powerful world of C# metaprogramming. Remember: with great power comes great responsibility—use these techniques wisely, always considering performance and maintainability!