Mac OS X: The file for NetLogo will be located at:
/Applications/NetLogo 6.3.0/NetLogo 6.3.0.app/Contents/app/NetLogo 6.3.0.cfg
Change -Xmx1024m to -Xmx4096m in [JavaOptions].
Your model is too large to run in the amount of memory available to NetLogo! For more information consult the “How big can my model be?” section of the FAQ in the user manual.
Warning: you may see strange results if you continue running the current model without restarting NetLogo.