Using atoms from declared structs in elixir works on repl but not in application

Question

I want to use String.to_existing_atom in elixir to avoid memory leaks.

This 100% works on the REPL:

iex(1)> defmodule MyModule do
...(1)> defstruct my_crazy_atom: nil
...(1)> end
{:module, MyModule,
 <<70, 79, 82, ...>>,
 %MyModule{my_crazy_atom: nil}}

So now the atom my_crazy_atom exists. I can verify this:

iex(2)> String.to_existing_atom "my_crazy_atom"
:my_crazy_atom

Compared to:

iex(3)> String.to_existing_atom "my_crazy_atom2"
** (ArgumentError) argument error
    :erlang.binary_to_existing_atom("my_crazy_atom2", :utf8)

But I have some code that looks like this:

defmodule Broadcast.Config.File do
  defstruct channel_id: nil, parser: nil
end

From a method call after starting a GenServer process, I can decode with Poison's

keys: :atoms!

or even just call

String.to_existing_atom("parser")

in the same place in the code and I get an error:

** (Mix) Could not start application broadcast: exited in: 
Broadcast.Application.start(:normal, [])
    ** (EXIT) an exception was raised:
        ** (ArgumentError) argument error
            :erlang.binary_to_existing_atom("parser", :utf8)

Oddly, if I instantiate the struct and inspect it, then the issue goes away!

IO.puts inspect %Broadcast.Config.File{}
String.to_existing_atom("parser")

What's going on here? Is this some kind of ordering thing?

Can you post the complete code which reproduces the error? I'm unable to reproduce the error by copying your MyModule and then doing String.to_existing_atom("my_crazy_atom") from another function in another module. — Dogbert
– Dogbert, Commented Sep 20, 2017 at 6:21
I've put the code in a mix project. Where should I put the String.to_existing_atom call? If I put it in a function and then call it, it works without me needing to create an instance of the struct. (I don't think mudasobwa's answer is right (or maybe I've misunderstood the question.)) — Dogbert
– Dogbert, Commented Sep 20, 2017 at 16:13
@Dogbert please see an update in my answer for the steps to reproduce. — Aleksei Matiushkin
– Aleksei Matiushkin, Commented Sep 20, 2017 at 17:04

Dogbert · Accepted Answer · 2017-09-20 18:55:08Z

5

This happens because Elixir by default lazy loads modules from the compiled .beam files when they're first used. (Your code will work if start_permanent is set to true in mix.exs, which is set to true by default in the :prod environment, because then Elixir eagerly loads all modules of the package.)

In the code below, the atom :my_crazy_atom will be present in the Blah module's code but it's not present in Foo. If you start a REPL session and run Foo.to_existing_atom, the Blah module is not loaded which causes String.to_existing_atom("my_crazy_atom") to fail.

# Credits: @mudasobwa
defmodule Blah do
  defstruct my_crazy_atom: nil
end

defmodule Foo do
  def to_existing_atom, do: String.to_existing_atom("my_crazy_atom")
end

As you've observed, if you create a struct manually once, all subsequent calls to String.to_existing_atom("my_crazy_atom") return the correct atom. This is because when you create a struct, Elixir will load the .beam file of that module which will also load all the atoms that are used by that module.

A better way to load a module (as compared to creating a struct) is to use Code.ensure_loaded/1 to load the module:

{:module, _} = Code.ensure_loaded(Blah)

answered Sep 20, 2017 at 18:55

Dogbert

224k43 gold badges419 silver badges416 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

JasonG Over a year ago

Thanks Dogbert. I had to give it to @mudasobwa as he answered first :) Appreciate the answer and like the solution w/ Code.ensure_loaded

Dogbert Over a year ago

I think mudasobwa's answer missing the exact reason why this happens: lazy loading of modules. If you start Elixir with start_permanent: true (the default for prod env) you'll see that everything works even if you don't create an instance of the struct. The only reason why your workaround (of creating an instance of the struct) worked is because when a struct is created, Elixir loads that module which causes all atoms used by that module to be created.

Aleksei Matiushkin Over a year ago

@JasonG FWIW, I upvoted this answer myself and I think you are to mark this as a correct one.

Aleksei Matiushkin · Accepted Answer · 2017-09-20 17:14:11Z

1

The difference between REPL and your application is that in REPL the compilation process happens immediately. That said, the VM, as it sees

iex(1)> defmodule MyModule do
...(1)>   defstruct my_crazy_atom: nil
...(1)> end

has it immediately compiled. During compilation stage, the atom is being created and everything works.

In your application, OTOH, the compilation process is being performed in advance, by different invocation of the VM. Hence unless the structure is explicitly used, the atom would not be created.

One could think about it as about class declaration vs. instantiation in OOP: the class definition existence does not guarantee there are instances of this class present.

To see, what and when actually happens, try to put IO.puts "I AM HERE" inside the module declaration, immediately before defstruct. In REPL you’ll see this line printed immediately. In your application, you’ll see it during compilation and won’t see it while running the application normally.

Steps to reproduce:

$ mix new blah && cd blah
$ cat lib/blah.ex
defmodule Blah do
  defstruct my_crazy_atom: nil
end

defmodule Foo do
  def to_existing_atom, do: String.to_existing_atom("my_crazy_atom")
end
$ mix compile
$ iex -S mix
iex|1 ▶ Foo.to_existing_atom
** (ArgumentError) argument error
    :erlang.binary_to_existing_atom("my_crazy_atom", :utf8)
    (blah) lib/blah.ex:6: Foo.to_existing_atom/0
iex|1 ▶ %Blah{}
%Blah{my_crazy_atom: nil}
iex|2 ▶ Foo.to_existing_atom
:my_crazy_atom

I will put it here for the sake of comprehensiveness (adopted from this brilliant answer):

defmodule AtomLookUp do
  defp atom_by_number(n),
    do: :erlang.binary_to_term(<<131, 75, n::24>>)

  def atoms(n \\ 0) do
    try do
      [atom_by_number(n) | atoms(n + 1)]
    rescue
      _ -> []
    end
  end
  def atom?(value) when is_binary(value) do
    result = atoms()
             |> Enum.map(&Atom.to_string/1)
             |> Enum.find(& &1 == value)
    if result, do: String.to_existing_atom(result)
  end
end

iex|1 ▶ AtomLookUp.atom? "my_crazy_atom"
nil
iex|2 ▶ %Blah{}
%Blah{my_crazy_atom: nil}
iex|3 ▶ AtomLookUp.atom? "my_crazy_atom"
:my_crazy_atom

edited Sep 20, 2017 at 17:14

answered Sep 20, 2017 at 6:06

Aleksei Matiushkin

121k12 gold badges109 silver badges174 bronze badges

2 Comments

JasonG Over a year ago

Thanks, so what's the idiomatic approach here - I'm just invoking the structs once and throwing them away to ensure the atoms are created which seems wrong.

Aleksei Matiushkin Over a year ago

Honestly, I have no idea. I would probably go with declaring a list of permitted atoms (that are usually hardly strictly equal to the fields of all structs over the project) and executing smth like MyAtoms.permitted before the application starts.

Collectives™ on Stack Overflow

Using atoms from declared structs in elixir works on repl but not in application

2 Answers 2

3 Comments

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related