Save your memory when compiling Scala
Too long; didn't read
My work computer has 32 GB of RAM memory. When I heard the number, I immediately thought that size would once have been worth the capacity of a server!
So, happy as a child with a new toy, I have been firing applications in total freedom. And everything went well until I discovered the wonders of Scala Metals. This tool is amazing for Scala development and you can find how to set it up here.
Anyway, the other day I had to take a video call and... everything froze. Oh no! Hard reset?!? What happened!?
My RAM was full! 32 GB! My own personal server, aarrrgh!!
It is a problem indeed
We are in 2021, and although I believe in Moore's law, I reject the idea that any software can fill up all of my humongous RAM memory. Removed from the list of culprit Microsoft Teams, I pointed my finger at the K8S cluster I was running: surely that was the memory drainer. For a month or two I kept assuming that. Tip: never ever ever assume (and point fingers)!
Restart after restart, I started noticing the pattern: I open a Scala project, run sbt, and... hard reset again.
My finger slowly start dropping. Time to invest some time in a proper
investigation. Launch a terminal, write
top, click shift-m, and...
tada: the three knights of the RAM Apocalypse are there: metal's sbt
13GB, bloop 10GB, and my sbt 5GB. I am an sbt instance or two away
from hard resetting again.
What's going on?! Isn't this 2021? Aren't we planning to send people on Mars? Oh my!
And there is a solution
Apparently the default configuration of the JVM and Sbt lets each instance of these tools absorb 1/4 of the available RAM memory. This does not really scale, right? By the way, never fully trust default configurations: oftentimes they are real traps (see how too many ElasticSearch instances are publicly available for the happiness of Shodan's curious).
So you need to establish boundaries with your tools. Here the settings you need:
(setq sbt:program-options (list "-Dsbt.supershell=false" "-J-XX:+UseParallelGC" "-J-Xmx4g" "-J-Xms100m" "-J-Xss4m"))
With this line you can set your sbt-mode to reduce the memory consumption of your JVM. The relevant settings are in order:
- use parallel garbage collector, which is apparently faster than the default (and safer) choice
- fix memory maximum at 4GB of consumption (you can pick 2GB if your Scala projects are good)
- some extension of memory available for the stack, just in case
This works for Java 9, if you are using older version drop the "GC" settings.
The same settings are necessary for your setup of Metals:
curl -L -o coursier https://git.io/coursier-cli chmod +x coursier ./coursier bootstrap \ --java-opt -Xss4m \ --java-opt -Xms100m \ --java-opt -Xmx4g \ --java-opt -XX:+UseParallelGC \ --java-opt -Dmetals.client=emacs \ org.scalameta:metals_2.12:9.1.0 \ -r bintray:scalacenter/releases \ -r sonatype:snapshots \ -o /usr/local/bin/metals-emacs -f
Now with the extra JVM options all the Metals and Bloop processes should be bound to those memory sizes.
If you want to know more checkout
I hope this saves you some unnecessary hard reset! Always configure your defaults!!