2023/06/22 Data Architecture Meeting

[jkbhagatio]

• Multi-animal tracking
  • Acute multianimal tattooed tail sessions aeon_experiments#260
  • Add pose reader classes supporting ingestion of SLEAP tracking data #206
• 'aeon_experiments' codebase structure for future experiments
  • Codebase for multiple experiments aeon_experiments#196
  • Handling data from test/dev epochs aeon_experiments#197
  • Only copy data into CEPH from clean branches aeon_experiments#186
• Consider renaming Harp stream file naming conventions aeon_experiments#262
• SciViz Video Streamer updates
• Specify foraging experiments data contract using formal yaml schema aeon_experiments#175

[ttngu207]

Ingest data for the current, ongoing presocial experiments. We will discard these data later because these are primarily for testing the system, but let's make sure the ingestion is running as expected as well.
@JaerongA

[JaerongA]

Following up on this. 226 epochs and 2727 chunks have been ingested so far from presocial0.1 from AEON2 and AEON3 (I don't see any data in AEON4 yet).

Are you running any analysis on these datasets @jkbhagatio? I'm trying to see if there's any way I can validate the ingested data.

[ttngu207]

Dario is planning for a new 24/7 experiment

• 2 animals
• Similar to the foraging experiment
• First week of july 2023
• May not need to ingest

[glopesdev]

Proposal for changing metadata into a more distributed format ("per-stream"):

• Have one metadata file per stream describing the properties of the device/stream
• In the future we can create links between external metadata files (e.g. Harp devices have their own schemas available publicly which could tell us the metadata of Harp device streams, column names, etc)