We will now analyze how we can develop code that writes code for us, before being finally transformed into BEAM bytecode. This will let us extend Elixir, inject new code into existing modules, and even write a domain-specific language to simplify our media pipeline definitions.
What you will see through this chapter is only possible due to the incredible tools Elixir gives us to manipulate the abstract syntax tree just before it is turned into bytecode. Let's jump right into it!
You may have already heard about Abstract Syntax Trees (ASTs) in other languages. As the name indicates, these are tree-like data structures that represent the code syntax. In Elixir, we call these representations quoted expressions.
If we try to obtain the quoted expression of simple expressions, such as single atoms, strings, integers or floats, lists or two element tuples, we'll see their quoted representation doesn't change when compared to their normal representation. These elements are called literals because we get the same value after quoting them. Take a look at the following code:
iex> quote do: :"Funky.Atom"
:"Funky.Atom"
iex> quote do: ["a", "b", "c", "z"]
["a", "b", "c", "z"]
iex> quote do: 1.88
1.88
iex> quote do: "really big string but still simple"
"really big string but still simple"
iex> {:elixir, :rocks} == quote do: {:elixir, :rocks}
true
The tree form of the quoted expressions is created by nesting three-element tuples and can be seen for complex snippets of code, which are composed by more than just Elixir literals:
iex> quoted_case = quote do
...> case 1 == 2 do
...> true -> "it seems 1 == 2 is true"
...> _ -> IO.puts "1 == 2 isn't true after all"
...> end
...> end
{:case, [],
[
{:==, [context: Elixir, import: Kernel], [1, 2]},
[
do: [
{:->, [], [[true], "it seems 1 == 2 is true"]},
{:->, [],
[
[{:_, [], Elixir}],
{{:., [], [{:__aliases__, [alias: false], [:IO]}, :puts]}, [],
["1 == 2 isn't true after all"]}
]}
]
]
]}
In the preceding example, we are obtaining the quoted representation of a case statement. Each three-element tuple is usually composed by an atom (or another three-element tuple) for the function name, a list with metadata, and an arguments list. If the tuple represents a variable, the last element of the tuple will instead be an atom.
To evaluate a quoted expression, we can use the Code.eval_quoted/3 function. If we evaluate the previous quoted_case representation, we will get a two-element tuple, with the evaluation result and the value of the passed bindings after evaluation (the :ok atom is the return value of calling IO.puts/1):
iex> Code.eval_quoted quoted_case
1 == 2 isn't true after all
{:ok, []}
Our quoted case expression didn't have any variables, so we weren't able to observe how bindings work. Let's now see what bindings are for with the following quoted expression using a variable x:
iex> quoted_case_with_vars = quote do
...> case x == 2 do
...> true -> "it seems x == 2"
...> _ -> IO.puts "x == 2 isn't true after all"
...> end
...> end
{:case, [],
[
{:==, [context: Elixir, import: Kernel], [{:x, [], Elixir}, 2]},
[
do: [
{:->, [], [[true], "it seems x == 2"]},
{:->, [],
[
[{:_, [], Elixir}],
{{:., [], [{:__aliases__, [alias: false], [:IO]}, :puts]}, [],
["x == 2 isn't true after all"]}
]}
]
]
]}
If you compare this quoted expression with the previous one, besides the minimal changes to the strings we're using, you can observe that the only change is, instead of having the 1 on the case equality comparison, we have the quoted representation of getting the value of x.
However, if we evaluate the quoted_case_with_vars expression with an explicit [x: 3] binding, it won't yield the expected result:
iex> Code.eval_quoted quoted_case_with_vars, [x: 3]
warning: variable "x" does not exist and is being expanded to "x()", please use parentheses to remove the ambiguity or change the variable name
nofile:1
** (CompileError) nofile:1: undefined function x/0
(stdlib) lists.erl:1354: :lists.mapfoldl/3
The behavior we're seeing here is deliberate, and the main reason for it is to spare ourselves from headaches further down the road; by default, these expressions are evaluated on a separate context and aren't able to access external variables (even if in this case we're passing x). If we want our expressions to access an outer value, like x, we have to wrap the x with a call to var!/1:
iex> quoted_case_with_external_vars = quote do
...> case var!(x) == 2 do
...> true -> "it seems x == 2"
...> _ -> IO.puts "x == 2 isn't true after all"
...> end
...> end
{:case, [],
[
{:==, [context: Elixir, import: Kernel],
[{:var!, [context: Elixir, import: Kernel], [{:x, [], Elixir}]}, 2]},
[
do: [
{:->, [], [[true], "it seems x == 2"]},
{:->, [],
[
[{:_, [], Elixir}],
{{:., [], [{:__aliases__, [alias: false], [:IO]}, :puts]}, [],
["x == 2 isn't true after all"]}
]}
]
]
]}
This way, our case expression finally has access to the bindings we set on the Code.eval_quoted/2 call:
iex> Code.eval_quoted quoted_case_with_external_vars, [x: 2]
{"it seems x == 2", [x: 2]}
The previous examples highlighted a very important aspect of metaprogramming in Elixir: the final quoted expressions generated by us, are only able to access variables outside their context if we explicitly said so through var!/1.
This inability to access the outer context by default is called macro hygiene and is the safeguard that keeps things separate, without leaking into the context of the caller. The decision of accessing (and possibly changing) the outer context should be fully considered by the developer, and this separation by default rule helps keeps things sane.
After lo...
