Issues with Bonsai Displaying Points from Multi-Animal DeepLabCut Model

[Bioarti]

Hello everyone,

I am facing a challenge with integrating a DeepLabCut multi-animal model into Bonsai for real-time tracking. Specifically, the model correctly identifies and tracks all points for multiple animals in DeepLabCut, but when I open the model in Bonsai, it only displays points for one animal.

Details of the Setup:
DeepLabCut Version: [DLC 2.3.9]
Model Details: The model is trained to track two mice, with 9 points per mouse, making a total of 18 points.
Issue:
In Bonsai, only 9 points (for one mouse) are displayed, even though the model is supposed to track 18 points for two mice.
Questions:
How can I modify the Bonsai settings to ensure it displays all points for both animals?
Is there a specific configuration or workaround that I need to implement to correct this issue?
Has anyone else encountered this problem, and if so, how did you resolve it?
Are there any tutorials or documentation available that cover multi-animal tracking in Bonsai using a DeepLabCut model?
Any guidance or suggestions from the community would be greatly appreciated. Thank you!

Best regards,
Sam You

[Bioarti]

@bruno-f-cruz Hi, Sorry to bother you, but I am very eager to know about the progress of maDL-live and the possibility of integrating it with Bonsai。If you could provide me with some progress updates, I would be very grateful。

[bruno-f-cruz]

I never used multi animal tracking with dlc. I am using the Sleap package + with great results tho.

[Bioarti]

I appreciate your quick reply. I'll try SLEAP for real-time multi-animal tracking instead of DLC and hope for positive results.
Thanks a lot.

[Bioarti]

@bruno-f-cruz
Thank you very much for your suggestion. I have used the Bonsai-Sleap package for real-time tracking of two animals, but I am encountering two issues during operation:

The frame rate gradually drops from 30 to 10 frames per second, possibly due to insufficient GPU power.
There are errors when the input video size does not match the size used for training the model.

Do you have any relevant solutions for these issues?

[Bioarti]

Thanks for your suggestions!
I have another issue about multi-animals pose-prediction. I can't locate or extract the coordinate data for specific body parts within the predict posed or any other nodes in bonsai.sleap. I even tried pythontransform,still got nothing.
I would greatly appreciate any advice or help you can provide.

[bruno-f-cruz]

You should use "GetBodyPart" with the name of the body parts defined in your config file.

[Bioarti]

Thank you for your reply. But in my project, "GetBodyPart" could only be added after "Predict SinglePose", not "Predict poses".

[bruno-f-cruz]

Have you tried going through the examples in the docs? Sounds like they address all these points.

https://bonsai-rx.org/sleap/articles/manual.html

[Bioarti]

I'm really sorry, I only looked at the environment setup and installation files for Bonsai.Sleap and didn't see this manual. I will read it carefully. Thank you for your patience.