Updated content from discourse for term slurm

[manbat]

I’m running a Sage MATH script I wrote with GNU parallel to get local parallelization, however I am currently looking to scale up to a cluster that uses Slurm as a workload manager.
I am using a simple GNU parallel script locally:

#!/bin/bash
time parallel --timeout 10 -j$(nproc) -N0 …/sage ./loader.sage.py ::: {1…4000} --progress echo {} >/tmp/out

I’m not sure how to go about recreating this with Slurm, I would appreciate any and all suggestions of resources and guides to look at. I am also looking to implement an output file that uses a first-in-first-out semaphore, and ideally an input file to read from. If any of this doesn’t make sense, I’m happy to clarify.

See Full Post