I'm a scientist looking hard at building a Threadripper machine. I build and use simulation software. It may be a very long time before I get into parallelizing the software, but I now put multiple processors to work by starting more than one independent instance (on the four core machine I use now). A Threadripper could really speed things along, if I use it well. I would appreciate any comment or tips on my workflow.
Right now I do this work with brute force. I create multiple drives, copy the programs and files to all the drives, and in a command line kick off a process for each drive. It may then crank for minutes, hours, or weeks. The reason for multiple drives ... the reason for complications in general ... is to avoid conflict when producing output files; I can't have one process writing to a file belonging to another process.
I can dream up pathways to avoid the multiple drives (such as providing each instance with a long random suffix to add to each file), but it seems awkward.
Any tips on best practices?