Generating code at runtime with illuminator

Photo by David Gavi on Unsplash

Illuminator is yet another wrapper around ILGenerator, with some interesting features:

1. Fluent, convenient API with functional programming flavor.
2. Tracing the generated code.
3. Transparent abstraction.
4. .netstandard2.0 support.

This library was emerged from another project, which I implemented with code emission, and was field tested in it. That is a library which can generate comparers on runtime for any structure or class.

Fluent functional API

Let imagine we need to generate the following code:

Using vanilla ILGenerator you may write something like this:

So much code for such simple function! When you need to write a more complex thing, it becomes not possible to maintain and understand it.

There are few problems with this code:

1. It’s very verbose, repetitive and hard to read. All this Emit and ObCodes are just unnecessary noise.
2. It’s very hard to write such code. You need to remember the specification for each instruction and keep in mind the state of the evaluation stack. You need to know exactly how many parameters each instruction needs.
3. It’s error prone. It’s very easy to make a mistake, and exceptions that are thrown at runtime are not very informative.

The simplest thing that we can do to improve the situation is to introduce “fluent” API:

Much better this time:

1. We don’t have to memorise all OpCodes and write those endless Emit. Intellisense helps us.
2. It less verbose and much easier to read.

Still, this does not solve all problems. It is still possible to have invalid evaluation stack or misuse short versions for branching instructions ( Brfalse_S).

Lets try one more time with some functional helpers:

You may think: What?! Wait, try to read it again: it checks ( Brfalse_S) the result of comparing ( Ceq) the first argument ( Ldarg_0) and the constant two ( Ldc_I4_2); if they are equal, return 1 ( Ret(Ldc_I4_1())), otherwise return the sum of the argument and three ( Ret(Add(Ldarg_0(), Ldc_I4_3()))).

The code now is much shorter and close to the target C# version. It's nicer to write such code, because all methods have exact amount of parameters that they need, and with output parameters we don't have to break "fluent flow" to create labels or locals ( out var label).

How does it work

Lets look at this line: Add(Ldarg_0(), Ldc_I4_3()).

Add is the static function from the Functions class:

We can use it directly thanks to using static directive feature. That why we need to include using static Illuminator.Functions; at the beginning.

This function uses the extension for ILEmitter class which calls ILEmitterFunc functions before it calls the actual Add methods:

ILEmitterFunc functions are responsible for preparing values in the stack. And as you may guess, it can be much complex list of constructions than simple Ldc_I4.

As the result we get the fluent, convenient API with functional programming flavor:

1. It uses the original naming of MSIL instructions, so you don't need to guess what Ldc_I4_2 for example does.
   It does exactly generator.Emit(OpCodes.Ldc_I4_2);.
2. All methods have helpers with exact amount of ILEmitterFunc parameters that they need to execute.
3. We can use output parameters to not break fluent flow.
4. A flow of instructions can be reused many times as it is a first class function now.

How it is implemented

Writing all those functional extensions and wrappers manually is really boring and takes a lot of time.

To make things fun and make fewer mistakes I created a complementary tool to generate most of the library’s code using F#, Scriban library as a template engine, documentation and reflection as a basic information.

I’ve used documentation to create opcodes.json file to define what exact parameters we need to run an instruction. So I can generate methods like this:

And using information in the System.Reflection.Emit.OpCodes class I know how much items an instruction are needed to generate functional extensions. For example OpCodes.Beq_S needs 2 values in the evaluation stack, to it creates the following methods:

Custom extensions

We can improve the readability of the code by creating our own extensions. Lets look at this one:

This extension emits the equivalent of the following code:

Using this extension we can rewrite our previous attempt like this:

At this time, the code is now even easier and more convenient for reading!

Of course, it can take a while for you to adapt to the functional style, but it doesn’t have to. You can still use fluent syntax. In my code I usually mix fluent and functional approaches depending on a situation.

Tracing

In complex cases you will want to see the list of instructions that your code generates. To do it you can use enableTraceLogger parameter, you may notice it in the last example.

It uses System.Diagnostics.Trace and outputs such result:

Final thoughts

The main goal of this project is to have simple, transparent library, which does exactly what you tell it.

No overengineering, no custom names, no complex rules to follow.

Very often different libraries try to hide the underlying technology which they use. For example, Entity Framework tries to hide SQL from programmers. But it does a disservice eventually, because in order to write efficient code and be able to understand what your code really does, you have to learn EF itself, know what SQL it generates and all caveats, and master SQL anyway.

Developers who generate code using MSIL have to know instructions. Doing magical tweaks and hiding Ldc_I4 for example behind nice LoadInteger name does not make things easier. A programmer is forced to read the documentation or source code to find out what such methods actually do.

It does not mean that it’s not allowed to create your own magical helpers. You can and you should to make your own solution more maintainable. But this is not the responsibility of the library.