Choosing the optimal number of processors is heavily dependent on many things:
- number of atoms (more atoms tends to allow more processors to be used)
- chosen algorithm (algorithms with more numerics can sometimes be distributed to a greater extend)
- implementation specific details (some codes are very good at distributing work, which might scale better than other codes)
- compilation optimizations, poor optimizations can lead to better performance scaling, but slower code altogether.
- the hardware you are using, your rule of thumb for your current nodes, might not be valid when you get new hardware (this is especially true for GPU's).
- most hardware nodes in nowadays HPC facilities uses NUMA (Non-Uniform-Memory-Access) nodes. Here the distribution of your cores on the node also decides the performance (memory location, affinity)
I don't know of any rules of thumb for LAMMPS. But only a proper benchmark on your test system and number of cores would actually do the job.
So I would suggest you to take a small system of yours, tune the number of iterations so it will run in ~ 10 minutes (no more than an hour).
Run the calculation on the same hardware, for 1, 2, 4, 8, 16, 24, 32
cores
and plot the performance. If it scales nicely up to 32 cores, then great! :) You can go ahead and play with a 2nd node.
Now try with 4 cores on each node. And increase number of cores on each node simultaneously.
When the performance seems to flatten out, then it might be better to use fewer cores. :)
There is no need to add more cores if you only get 1-5% of performance of what you would expect (an ideal scaling code would be twice as fast, if you doubled the number of cores).