You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Now that #72 is beggining to handle full project organisation with subject > session > run levels, it is worth thinking about parallelisation.
The top level will always be the subject. However, if running 3 subjects sorting separately, we will want to parallalise. This will be over nodes using SLURM if GPU required, or other cores otherwise.
For now, keep the code structured as for loops over subject, runs. This will be the easiest to play around with parallelisation strategies. Following this, if the jobs of per-session processing and per-run processing are overlapping, create class scheme.
The text was updated successfully, but these errors were encountered:
This is very relevant to #64 because SI also implements dask / multiprocessing backends for sorting which it would be great to leverage, and determine how these will play if we are already parallelising over subject / runs.
Now that #72 is beggining to handle full project organisation with subject > session > run levels, it is worth thinking about parallelisation.
The top level will always be the subject. However, if running 3 subjects sorting separately, we will want to parallalise. This will be over nodes using SLURM if GPU required, or other cores otherwise.
For now, keep the code structured as for loops over subject, runs. This will be the easiest to play around with parallelisation strategies. Following this, if the jobs of per-session processing and per-run processing are overlapping, create class scheme.
The text was updated successfully, but these errors were encountered: