[Feature] Improved template script for atlas integration

[PolarBean]

Is your feature request related to a problem? Please describe.
I feel at the moment atlas integration is a bit complex. It seems like the atlas integration scripts can become a bit unwieldy, with unclear separation between each stage.

Describe the solution you'd like
Have a simplified template script for atlas integrators to use. The basic idea would be set it up such that there are several functions that a user must fill out where the output values are clearly defined. here is a basic untested sketch of the idea which needs more work. We could as well move a lot of mesh creation logic into helper functions. Let me know your thoughts.

###Metadata
__version__ = 1 # The version of the atlas in the brainglobe_atlasapi, this is internal, if this is the first time this atlas has been added the value should be 1
ATLAS_NAME = None # The expected format is FirstAuthor_SpeciesCommonName, ie; kleven_rat
CITATION = None # DOI of the most relevant citable document
SPECIES = None # The scientific name of the species, ie; Rattus norvegicus
ATLAS_LINK = None #The URL for the data files
ORIENTATION = None #The orientation of the atlas, for more information on how to determine this click here: ........
ROOT_ID = None # The id of the highest level of the atlas. This is commonly called root or brain. Include some information on what to do if your atlas is not hierarchical
RESOLUTION = None # the resolution of your volume in microns. details on how to format this parameter for non isotropic datasets or datasets with multiple resolutions. 

def download_resources():
    """
    Download the necessary resources for the atlas.
    
    If possible, please use the Pooch library to retrieve any resources.
    """
    pass

def retrieve_template_and_reference():
    """
    Retrieve the desired template and reference as two numpy arrays.

    Returns:
        tuple: A tuple containing two numpy arrays. The first array is the template volume, and the second array is the reference volume.
    """
    template = None
    reference = None
    return template, reference

def retrieve_hemisphere_map():
    """
    Retrieve a hemisphere map for the atlas.

    If your atlas is asymmetrical, you may want to use a hemisphere map. This is an array in the same shape as your template, 
    with 0's marking the left hemisphere, and 1's marking the right.

    If your atlas is symmetrical, ignore this function.

    Returns:
        numpy.array or None: A numpy array representing the hemisphere map, or None if the atlas is symmetrical.
    """
    return None

def retrieve_structure_information():
    """
    This function should return a pandas DataFrame with information about your atlas.

    The DataFrame should be in the following format:

    ╭─────────────┬───────────────────────────────────┬─────────┬───────────────────────┬───────────────────╮
    | id          | name                              | acronym | structure_id_path     | rgb_triplet       |
    |             |                                   |         |                       |                   |
    ├─────────────┼───────────────────────────────────┼─────────┼───────────────────────┼───────────────────┤
    | 997         | root                              | root    | []                    | [255, 255, 255]   |
    ├─────────────┼───────────────────────────────────┼─────────┼───────────────────────┼───────────────────┤
    | 8           | Basic cell groups and regions     | grey    | [997]                 | [191, 218, 227]   |
    ├─────────────┼───────────────────────────────────┼─────────┼───────────────────────┼───────────────────┤
    | 567         | Cerebrum                          | CH      | [997, 8]              | [176, 240, 255]   |
    ╰─────────────┴───────────────────────────────────┴─────────┴───────────────────────┴───────────────────╯

    Returns:
        pandas.DataFrame: A DataFrame containing the atlas information.
    """
    return None
   
### If the code above this line has been filled correctly, nothing needs to be edited below. 

download_resources()
template_volume, reference_volume = retrieve_template_and_reference()
hemispheres_stack = retrieve_hemisphere_map()
structures = retrieve_structure_information()

output_filename = wrapup_atlas_from_data(
      atlas_name=ATLAS_NAME,
      atlas_minor_version=__version__,
      citation=CITATION,
      atlas_link=ATLAS_LINK,
      species=SPECIES,
      resolution=(RESOLUTION,) * 3,
      orientation=ORIENTATION,
      root_id=ROOT_ID,
      reference_stack=template_volume,
      annotation_stack=annotated_volume,
      structures_list=structures_with_mesh,
      meshes_dict=meshes_dict,
      working_dir=working_dir,
      hemispheres_stack=None,
      cleanup_files=False,
      compress=True,
      scale_meshes=True,
  )

[PolarBean]

I see this is a duplicate of #242 however the script that was created there seems to no longer exist. there is example mouse however I still feel implementation of the above would constitute an improvement.

[alessandrofelder]

I think this is a good idea - we've had similar discussions internally before (that I failed to document 😅 - thanks for opening this issue): The alternative is that we refactor (#219 and related issues) and then have an actual working script (we need to identify the simplest, most emblematic one) that we point to in the developer docs as the "poster child" of packaging scripts, and therefore as the pattern to follow.

The alternative is what was slightly preferred in those discussions, because

• it's less code to maintain
• it's more concrete: it's code that can be (and is) actually run "in-the-wild".

What do you think @PolarBean ?

[PolarBean]

I think the danger of this is that you will always have atlas specific code. beginners will struggle to know what is strictly required and what was a quirk of that atlas. It's also just nice to point someone to a form like script, and say fill this out. I think both could be nice, one template script with code separated into relatively self contained functions, and another poster child which uses that template. I'm happy to keep working on this and create an implementation of the above template!

[adamltyson]

I used to argue against a template script, because we had a functional script that wasn't (IMO) a sufficient abstraction to allow new contributors to learn what was needed. However, I like @PolarBean's version, as it's as much of a teaching tool as it is a template. It also shouldn't need much maintenance.

I would vote for:

• Very high level template such as this
• Converting existing scripts to follow this template to serve as examples

[alessandrofelder]

I would vote for:

• Very high level template such as this
• Converting existing scripts to follow this template to serve as examples

Happy with this too. cc @viktorpm so we coordinate the refactoring of the packaging scripts well 🙂