> I hadn't really thought about GC latency for a single threaded app. Is the erlang GC any worse than e.g. the JVM for this case?
It is a lot, lot worse yes.
Because of the purpose of Erlang / Elixir, and the way the runtime functions, the GC is rather simple (though not trivial): it's a stop-the-world generational (2) semispace collector. So on a GC run, the execution stops, the GC acquires a new empty heap, scans the stack (the "root-set"), traverses the tree of heap object, and each heap object it finds is copied to the new heap (the actual process is a bit different but that's the idea).
That works well, and the generational hypothesis applies nicely because Erlang only has immutable data structures so unlike Java and friends an Erlang object can not refer to something younger than it is. However it generates a lot of garbage as an "update" requires creating a new object.
It's quite simple, and has good (though not amazing) throughput, but it has horrible latency.
The trick is, the GC works per process. So the "world" it stops is a single process, and all the stack scanning and faffing about is per-process, meaning on a stop it might have to deal with kilobytes of data, megabytes at the absolute worst. It doesn't need to scan the entire runtime and go through gigabytes of heap as Java commonly does, because Erlang processes are shared-nothing (aside from the global ref-counted "shared heap" but that's a bit of a special case). This means with "normal" usages (normal for BEAM) a given GC run has very little memory to scan and not too much work to do per collection, it's essentially leveraging the other characteristics of the language to create an emergent concurrent low-latency collector, the concurrency and latency are not part of the design of the GC, but instead part of the system the GC is used in.
All of that falls over when you start moving away from lots of small actors, and towards few big actors. Then the size of the stack increases, the amount of garbage explodes, and the concurrency drops precipitously, because the GC's "world" covers a larger and larger amount of the program's surface.
It is a lot, lot worse yes.
Because of the purpose of Erlang / Elixir, and the way the runtime functions, the GC is rather simple (though not trivial): it's a stop-the-world generational (2) semispace collector. So on a GC run, the execution stops, the GC acquires a new empty heap, scans the stack (the "root-set"), traverses the tree of heap object, and each heap object it finds is copied to the new heap (the actual process is a bit different but that's the idea).
That works well, and the generational hypothesis applies nicely because Erlang only has immutable data structures so unlike Java and friends an Erlang object can not refer to something younger than it is. However it generates a lot of garbage as an "update" requires creating a new object.
It's quite simple, and has good (though not amazing) throughput, but it has horrible latency.
The trick is, the GC works per process. So the "world" it stops is a single process, and all the stack scanning and faffing about is per-process, meaning on a stop it might have to deal with kilobytes of data, megabytes at the absolute worst. It doesn't need to scan the entire runtime and go through gigabytes of heap as Java commonly does, because Erlang processes are shared-nothing (aside from the global ref-counted "shared heap" but that's a bit of a special case). This means with "normal" usages (normal for BEAM) a given GC run has very little memory to scan and not too much work to do per collection, it's essentially leveraging the other characteristics of the language to create an emergent concurrent low-latency collector, the concurrency and latency are not part of the design of the GC, but instead part of the system the GC is used in.
All of that falls over when you start moving away from lots of small actors, and towards few big actors. Then the size of the stack increases, the amount of garbage explodes, and the concurrency drops precipitously, because the GC's "world" covers a larger and larger amount of the program's surface.