Blacksmith is a data generation framework for Elixir which helps you to create sample data in tests. If you are a Ruby developer, you can think of it as an equivalent to factory_girl. I will explain a part of its functionality within the post. For further information about the package you should read its README as well as this blog post written by its creator. I’ve chosen this library for my code reading article because it makes use of OTP and metaprogramming while being short enough to grasp it completely (at the time of writing, the whole package consists of more than 200 LOC). Since no version was released yet, I will refer to a specific commit within its git repository.
If you want to try the code samples in this post yourself, you can grab the source code from Github; make sure to refer to the same commit as I do:
Code analysis: OTP Structure
I will start analysing the OTP structure of the package by looking into which processes are spawned. This helps me understand which parts are done concurrently. Erlang comes with a graphical tool called observer that allows to look into a running erlang system and displays its supervisor trees, processes and more. You can start it from iex:
In the Applications tab you can see that the blacksmith application started one process named “Elixir.Blacksmith.Sequence”.
The Processes gives more detailed information about the processes. Here we can see that the Sequence process is a gen_server process.
You can find the complete source code of the Sequence Server under lib/blacksmith/sequence.ex, the following gist shows an excerpt:
The start_link function starts an Agent process under the name Blacksmith.Sequence that holds an empty HashDict as its internal state. This is the process we saw in the observer, but as Agent is just an abstraction of GenServer, the Erlang observer displayed as a gen_server server process. The next/1 function takes a sequence name, returns the current value of the sequence and increases the stored value. As a convenience, a next/0 function is added which refers to a sequence named :default.
Knowing that the sequence runs under a named Agent process, we can use Agent.get/2 to look into the internal state of the process in iex:
Code Analysis: The core functionality
The core functionality of blacksmith is to build records with default values. In order to play with the basic usage, add a file “forge.exs” in the project’s root folder and compile it in iex:
Let’s have a look into how this is done internally. In order to do so, I will again use only an excerpt of the source code. I omitted quite a lot and also changed the implementation of new/4 as I don’t need some functionality for the example. You can find the complete code at lib/blacksmith.ex.
The module consists of three parts:
- The __using__ Macro enables the including module to call the register Macro directly in its source code and sets default value for the module attribute @new_function.
- The register Macro adds a method to the calling macro each time it is called.
- The new and to_map are normal functions of the Blacksmith module
The most important functionality happens in the register Macro. In order to analyze what it does exactly, Macro.to_string/2 is a great helper. This allows to look at the source code for an abstract syntax tree that is returned by the quote block of the macro. I will alter the source code of the macro and save the result of the block in a variable called ast and output the code representation before returning the ast:
After recompiling the code, I can now see the result of the register :user call in our Forge:
The macro generates a user method within the Forge module which has the default values saved in its function body. You can also see that it calls the anonymous method that is stored in @new_function that was set to &Blacksmith.new/4 as a default. The reason for this indirection is that you could overwrite @new_function before calling register. TheBlacksmith.new now does the real work of building the record by merging the default values with the overriding values.
What I left out
As previously mentioned, I just used excerpts from the real source code, so feel free to dig into the other functionality:
- Saving the generated records in a repository
- Generating lists of records
- Using common data elements through the having macro (which would be worth a blog post of its own)
Whenever you read someone else’s source code, you find things you would have done differently. The one thing I had expected to be implemented differently was the method generation. As an alternative to generating a method per register call, you could also generate different implementations of the same method, a pattern that is used in the Unicode handling of Elixir itself and makes use of pattern matching of the Erlang VM. The interface would then change:
I hope you liked my approach of delving into the internals of blacksmith and learned as much as I did along the way. If you want to continue on this path, I suggest the following sources:
- Metaprogramming Elixir by Chris McCord explains the Elixir metaprogramming possibilities far more detailed than I did here.
- Xavier Shay’s blog series “code safari” was my inspiration of writing this kind of article.
- The ruby rogues also did some code reading and recorded it: http://youtu.be/lKmY_0uY86s.