Skip to content
Snippets Groups Projects
Select Git revision
  • development
  • master default protected
  • chenyao
  • example-config
4 results
Compare
Cribellier, Antoine's avatar
Merge branch 'development' into 'master'
Cribellier, Antoine authored 
Improved architecture, documentation and flexibility, added working tests for preprocessing of images

See merge request cribe001/dlc_flytracker!7
313d894b
History
Cribellier, Antoine's avatar 313d894b
History
Name Last commit Last update
camera
data
images
point_tracker
process
scripts
skeleton_fitter
tests
.gitignore
LICENCE.txt
README.md
main_process_mating.py
processes.yaml
requirements.txt
selection_mating.csv
setup.py
README.md

dlc_flytracker

Python package for the tracking the 3d kinematics of animals (currently only flies) build around DeepLabCut

???image

Introduction

The package allow the fitting of an animal (chosen) 3d skeleton to the tracking results from DeepLabCut.

The usual steps:

  • Post-processing of video recordings (at least 2 cameras): image enhancement, dynamic cropping around moving animal and the stitching of multiple camera views (to improve tracking with DeepLabCut)
  • Define the joints of the animal skeleton that will be used for the tracking. You can use a previously made skeleton (e.g. flies), or make your own from body modules (see the section ??? below to see an example).
  • (Outside this package) Training of a DeepLabCut (DLC) network: usually done using the DLC user interface. Require to manually label 100-200 images. See the section ??? below to see an example, and how DLC settings can be tweaked in order to get good tracking results with stitched images.
  • Run DLC tracking on the full dataset
  • Load the DLC results in dlc_flytracker
  • Reconstruct 3d coordinates of the skeleton joints from their 2d coordinates tracked with DeepLabCut
  • Optimize fit the 3d skeleton to the 2d/3d coordinates of the joints
  • Post-processing of the kinematics parameters

Citation

???

Getting Started

Prerequisite

dlc_flytracker requires deeplabcut, tensorflow and other common packages like numpy, scipy, matplotlib. For a full list, you can check the file requirements.txt

Installing

???

Usage

Here the main steps introduced earlier are explained in details with examples.

Post-processing of video recordings

??? (at least 2 cameras):

  • Image enhancement: ???
  • Dynamic cropping around moving animal: ???
  • Stitching of multiple camera views: ??? (to improve tracking with DeepLabCut)

???figure showing post-processing steps

Build animal skeleton

???

in DeepLabCut

???

in dlc_flytracker

???

Training a network in DeepLabCut

  • ??? using the DLC user interface. Require to manually label 100-200 images. See the section ??? below to see an example, and how DLC settings can be tweaked in order to get good tracking results with stitched images.

Run DLC tracking on the full dataset

???

Load the DLC results in dlc_flytracker

???

Reconstruct 3d coordinates of the skeleton joints from their 2d coordinates tracked with DeepLabCut

???

Optimize fit the 3d skeleton to the 2d/3d coordinates of the joints

???

Post-processing of the kinematics parameters

???

Notes

Running the tests

To run tests locally, type:

> pytest dlc_flytracker

License

This project utilizes the LGPL LICENSE. Allows developers and companies to use and integrate a software component released under the LGPL into their own (even proprietary) software without being required by the terms of a strong copyleft license to release the source code of their own components. However, any developer who modifies an LGPL-covered component is required to make their modified version available under the same LGPL license.