You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Sleap networks may have different input sizes than the videos they were trained on. This is done to accommodate two requirements:
The image size is divisible by the max stride of the model (usually 16 or 32) so we don't get mismatched tensor sizes across skip connections due to shape rounding/truncation
To ensure that all images in the same batch are of the same size so that we can handle variable image sizes.
This pre-processing step is automatically handled in the SLEAP API but not currently in Bonsai. They do it by padding, with zeros, at the bottom, and right of the image (which means coordinates should not change).
This pre-process step can be easily done in Bonsai by using the ResizeCanvas operator.
For the sake of usability, we could add this optional functionality to the predict nodes. Unfortunately, the training_config.json we are currently loading in bonsai with all the metadata does not have the network input size information.
Few possible solutions:
Load the info.json that results from the export sleap functionality. This would require loading and parsing an additional JSON file.
An alternative to 1. would be to simply merge the two json files in a single output file during the export stage, which would save users the bother to keep track of what metafiles correspond to what networks.
It might be possible to infer the network input size form the deserialized frozen network. Must investigate if this is a valid approach.
The text was updated successfully, but these errors were encountered:
bruno-f-cruz
changed the title
Add option to attempt to automatically adjust input image size
Attempt to automatically adjust input image size
Nov 11, 2022
Sleap networks may have different input sizes than the videos they were trained on. This is done to accommodate two requirements:
This pre-processing step is automatically handled in the SLEAP API but not currently in Bonsai. They do it by padding, with zeros, at the bottom, and right of the image (which means coordinates should not change).
This pre-process step can be easily done in Bonsai by using the
ResizeCanvas
operator.For the sake of usability, we could add this optional functionality to the predict nodes. Unfortunately, the
training_config.json
we are currently loading in bonsai with all the metadata does not have the network input size information.Few possible solutions:
info.json
that results from the export sleap functionality. This would require loading and parsing an additional JSON file.The text was updated successfully, but these errors were encountered: