Most of the time it helps by closing your modeler and reopen it again and then do a recompile. Because then the modeler has a clean memory. Just closing the project and then reopen does not always help. Only restarting of the modeler. Out of memory are soon a thing of the past when Mendix will post a 64 bit version of the modeller in the very near future.
To check you Java Heap size goto Project / Settings / Configurations and choose edit. You will see this:
Here you can set the Java Heap. Try to set it higher to see if it helps. Note that this only influance your local heap. Not the settings in the cloud.
that sounds like it would solve the problem, but unfortunately it doesn't change anything!
I tried increasing the max java heap size, even to 4gb, but I still get the same Java compile error...
Perhaps the heap space configured here, is only for the server and it is not used for compiling?
I have the same problem with the Mendix 6.10.2 modeler on my 32bit windows 10 virtual machine.
The problem is during building locally, not running/starting the app locally.
The userlib folder is around 75MB.
Does anybody know how to increase the java heap size of the builder tool?