2024-02-07 Experiment Control Meeting

[glopesdev]

• Double 0's still show up in BlockState data #497
• Add timeout notifications for all camera streams #496
• #500

Outstanding questions

Should we introduce a new data stream saving the list of subjects present in the block?

Currently it is cumbersome and error prone to query historical data over irregularly sampled streams with the low-level API. The issue stems from the irregular chunking of files and time-based queries, e.g. there may be no chunk files intersecting the query time range of interest, but still there may be state we care about in historical data from transitions happening before the start of the time range.

The only way we have to handle this now is to either load the entire data for that stream and then filter in-memory, or use the database after ingestion to query these historical streams. For ingestion purposes it might be necessary to ingest irregular streams before computing other tables requiring such time ranges.

Later we will want to find a general and efficient way to query historical data at runtime and analysis time.