These files contain all the functions needed to read, preprocess and combine both the UWB data and the acceleration data with the cow ids. It translates the
raw activity measures to interpretable numbers based on the dynamic range (individual per sensor, but technically all should be 8, not allways the case, though).
Besides linking the IDS, it also corrects for the time-errors (the sensor systems records on UTF time, causing a difference of 1 or 2 hours
...
...
@@ -79,13 +80,38 @@ The functions can be called with the script "data_preparation.py" and in the qua
**2. Tool to verify quality control of the uwb data**
Built into the pipelines of Dairy Campus, these scripts produce tables and figures with which the quality of the data of the UWB tags can be verified and visualised.
The "parquet_cmd" file checks the .parquet raw files without loading them, e.g. the number of lines in each file, and writes the results of the summary to an excel document on the W:/.
The "per_day_cmd" file reads the data of a prespecified number of days, and adds cow ids to them. Next, the raw data are summarized per day: number of records, tags, summary statistics with e.g.
the range and number of NaNs. This is handy for example to verify which accelerometer does not function well or is not calibrated. It also summarizes the gaps in the data, and makes some basic plots.
Upon errors, an e-mail is automatically sent to the people that need to check the data.
Bin contains different previous versions of the files, e.g. for development. The _cmd files are run via the command line, not via an IDE.
**3. Tools to read, condense, preprocess and interpret positions**
Bouts contains information on individual bouts per behaviour, one line per bout. It is calculated based on 2 thresholds: min_length = minimum duration to call it a “bout” e.g. feeding of less than 20 seconds is no feeding bout. Interval = time between two bouts of the same behaviour to be considered as separate bouts. This is dependent on the behaviour. For now, I used the following:
uwbfunctions contains the necessary functions to preprocess and interpret the uwb data. It uses information from the W:// drive (barn_areas.xlsx) with the barn locations for interpreting the behaviours.
When the script preprocessing_all_behaviours is ran, it produces several files with preprocessed and ready-to-work-on data, among which "allday", "summary", "bouts", "activity". These contain information of all
cows, starting from the raw .parquet files. The detailed contents and their meaning are listed below.
"Exploration" contains functions to visualise and plot the data (over time). For example, it allows to plot the heatmap with the zones in barn_areas, and the ethograms. This is handy when looking into the order,
amount and distribution of the behaviours over time.
Files not named above are integrated in the named files.
Bouts contains information on individual bouts per behaviour, one line per bout. It is calculated based on 2 thresholds: min_length = minimum duration to call it a “bout” e.g. feeding
of less than 20 seconds is no feeding bout. Interval = time between two bouts of the same behaviour to be considered as separate bouts. This is dependent on the behaviour. For now, I used the following:
|**Behaviour** |**min_length**|**interval**|
|-----------|:---------:|:---------:|
|Feed | 30s |5*60s|
|Drink| 5s| 5*60s|
...
...
@@ -130,5 +156,3 @@ _Activity_ contains a “summary” of the activity / different views on active/
