Future of Alexandria

[jggatter]

Future of Alexandria

As a part of my resignation, I have been asked to write what would have been done for the Alexandria project for the foreseeable future.

Workflows

I'll first say that the workflows side of Alexandria is nebulous. A lot of work was dedicated to wrapping Cumulus Team workflows for hosting on/compatibility with the Single Cell Portal, although the pay off for these efforts seemed pretty low as time went on for a few reasons:

• Source control: an efficient system for updating workflows hosted on SCP never came to fruition. The procedure developed was onerous for both SCP and Alexandria.
• Niche: Terra seems to be a much friendlier place for running a diverse array of workflows (despite not allowing for workflow input descriptions). SCP users seem more interested in exploring already processed data.
• User Needs: Similarly to the point above, as time went on it seemed like barely any users would need to run workflows on SCP, but even in the case there was a demand how could we be sure that our workflows would fulfill their needs?

The future of workflows need to be reevaluated. SCP already utilizes Terra as the backend for its studies and is currently improving the user journey for traveling to Terra to run notebooks. In my opinion it would be best if new SCP studies were simply preset to contain all Cumulus workflows (and Alexandria notebooks). Really the only burden on the Alexandria team needs to be documenting how to use workflows (much of this is already done by the Cumulus Team) and how to take outputs and resync them with the SCP study.

Notebooks

Notebooks should be the focus of the Alexandria project for the foreseeable future. This is the area with the highest pay off. Providing bite-sized analysis and exploration notebooks is what SCP users are really looking for. The Score Genes notebook recently developed fits the template for these sorts of notebooks we envision. It has importing/exporting of data from Terra/SCP/GCP to the notebook VM, a description of the notebook, examples of file formats, and other instructions for aiding the user in completing the notebook.

Some notebooks that must be delivered:

• Gene-Gene correlation + visualization
• Batch correction
• Perhaps more for gene set scoring + visualization

It's important to recognize that we're not the first to do template notebooks for single-cell analysis. It's worth studying the work of other projects to determine what other notebooks Alexandria could provide.

Additionally, SCP has been doing some fantastic work for visualization of data on the portal. It's worth checking in with them to make sure notebooks are not redundant with this work and to gain clarity on what advanced analyses/exploration Alexandria notebooks can afford.

Metadata Convention

The Alexandria Metadata Convention has recently seen uptake by SCP users outside of Alexandria/The Shalek Lab. Given its potential, it's worth continuing to refine the metadata convention. Perhaps efforts could be made for user outreach/feedback. It is also worth developing a metadata validation script that is portable. The current one used by SCP is ingrained in their systems. It may also be worth creating an interactive notebook or program that assists researchers in annotating their cells with metadata.