Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Metadata that needs fixing #307

Open
10 tasks
saskiad opened this issue Sep 27, 2024 · 3 comments
Open
10 tasks

Metadata that needs fixing #307

saskiad opened this issue Sep 27, 2024 · 3 comments

Comments

@saskiad
Copy link

saskiad commented Sep 27, 2024

  • Add project_name to data_description

In Session:

  • "Lick sensor left" and "Lick sensor right" are not daq_names for your rig and thus can't be listed as under daq_names.
  • Camera names don't match the names in the rig. They must match. There is a utility function you can use to make sure that the names align between rig and session.
  • Reward delivery cannot be null. How are you delivery rewards without it?
  • Your session_end_time is before the end of one of your data streams. The end time should be the last time of all data streams and stimulus epochs.

Stimulus Epoch

  • Missing stimulus_device_name
  • These output parameters and task parameters duplicate almost the entire metadata schema. Can we please please please not do this? Very seriously, this is going to make our DocDB painful and it ultimately undermines having the schema in the first place.
  • eg. calibrations should be in the calibration field
  • frame rate should be in the device (also, check the version of the schema you're using. The cameras in your device class have a max_frame_rate which was changed a while ago)
  • Your frame_num item specifically is blocking upload to DocDB. You can't have fields with ".csv" in the name. This information should live in the quality_control class (which was only recently added to the schema, but that's where it should live)
@bruno-f-cruz
Copy link
Collaborator

Thanks for submitting this @saskiad!
On a closer look, i am starting to wonder if this session was uploaded by this repository library

Could you attach the session.json? The reason for this is:

  1. we don't have left and right lickometers
  2. we do not generate a rig.json
  3. RewardDeliveryConfig is automatically generated (https://github.com/AllenNeuralDynamics/Aind.Behavior.Services/blob/b952f498c9aedcd7878e824721e913b4b89c6b71/src/aind_behavior_services/aind_services/data_mapper.py#L145)

I will look into the other problems just in case. Also, keep in mind that the goal of this repo is to fulfill the agreement of generating a "valid" session.json such that the datasets can be uploaded. As discussed a few months ago, until further discussion, we will be faking session.json fields and NOT generating any rig.json.

@saskiad
Copy link
Author

saskiad commented Oct 3, 2024

Sorry, are you saying the session.json contains fake information? Why are we generating fake data? Or do you mean the links to the rig are fake?
We need to fix your issues with the rig because this isn't sustainable.

@bruno-f-cruz bruno-f-cruz transferred this issue from AllenNeuralDynamics/Aind.Behavior.Services Oct 17, 2024
@bruno-f-cruz
Copy link
Collaborator

bruno-f-cruz commented Oct 17, 2024

Adding @saskiad feedback from teams here since it appears the previous report was from a different task asset:

  • Need experimenter name (either real name or an anonymized version, your team decides how you want to do it)
  • need session_end_time - this is surprising to me
  • Need iacuc protocol. There much be an iacuc protocol for these experiments
  • Olfactometer Channel Config does not belong in the calibration, it lives in the stimulus
  • The WHEEL information also doesn’t belong in the calibration but in the mouse platform device. (Also, this is a wheel. There is a separate treadmill class. This really needs to be consistent and clear - and this applies to the parameters things as well. Interchanging wheel/disc/treadmill is going to breed confusion across our data if we let it slide)
  • Watervalve calibration would benefit from some unit definitions
  • Some of the things on your daq list are not daq devices. They might be connected to a daq, but they themselves are not daq. Eg screen. Manipulator.
  • Given the stimulus modalities you list, I expect your list of stimulus devices to include an olfactometer, monitor, speaker, reward delivery,
  • Given the presence of a speaker, I expect speaker config

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants