New zonation based on ADM0 - AEZ - DOMSOIL_IPCC(mineral and Organic) - Validation with EUROSTAT and FAO. Data transformation. Different types of agricultural land extractions

Function, defining the different surogate crop species for aggregated groups.

Peat done (except of merging the yield and area of grass >> still some issues). Single files are fine for crop and grassland area now with peat and nonpeat.
SPAM extraction from GEOTIFF file >> New zonal statistics function > much more efficient now. Replaced left_join with bind function. Now much less memory demanding.
Extracted factors from columnames.

Extraction of SPAM geotiff files based on ULrikes recommendation. Then I can use GAUL as a consistent dataset.

Deleted all old carbage from the script.

Started with the creation of the EU crop area validation data >> still needs finishing (see issues on git)

> No NA's > Only the **ratio of harvested area** for disaggregation. Like this, there is more consistency with the presence of nuts/veggies in adm1 zones. Otherwise, it happened that there is area but no production vice versa. Then you divide by 0 = inf.
> No inf values: The inf values came from the division of P/H=Y calculation. The P was in all cases very low so I decided to simply recode them to 0. Then Yields would be also 0.
> rounding fixed > 1 decimal like in the original dataset so we do not lose the level of detail.
> Yield conversion (production in mt) checked and correct.
> Aggregation to adm1 checked and correct.

> No NA's > Only the **ratio of harvested area** for disaggregation. Like this, there is more consistency with the presence of nuts/veggies in adm1 zones. Otherwise, it happened that there is area but no production vice versa. Then you divide by 0 = inf.
> No inf values: The inf values came from the division of P/H=Y calculation. The P was in all cases very low so I decided to simply recode them to 0. Then Yields would be also 0.
> rounding fixed > 1 decimal like in the original dataset so we do not lose the level of detail.
> Yield conversion (production in mt) checked and correct.
> Aggregation to adm1 checked and correct.

I fixed some computation problems. When I directly combined the SPAM datasets before running all the aggregation functions on them, my computer with 100 GB ram could not process it. So I changed it back to the old version with single files and then at last combine the datasets. Now it worked. Issue: No difference when disaggregating with harv and prod propotions on final values. Check why!

Veggies extraction from monfreda are now updated and code is much more concise with regex. Worked up to the adm1 aggregation of SPAM crop data line 652.

updated and validated the treenut script. Also the R.script now contains all the script needed to run the adm1 aggregation, treenut extraction. Before this was seperated in the treenuts script.

Changed the data loading to a more consice approach (based on Antoine from PPS) using regular expressions.

This version contains all code up to EU27 adm1 aggregation. There is some bugs still with the encoding (test adm1 unit is "sverige ..." ) as this is wrong when encoding did not work.

Treenuts and Vegetables were disaggregated today. The actual disaggregation is in the Treenuts_GGS_SPAM2010.R script. But then I worked it into the final datasets in the SPAM2010_aggregationAM1.R. So it is a bit redundant. The disaggregation looks good and I validated it. Was all fine!

Initializing the new Treenut script. Aim to refine the treenuts definition and run the zonal statistics for the crops for the new actual GSS shapfile.

ADM1 aggregation functions done and validated with rounding to one decimal position. Also I changed the aggregation fucntion to work with summarize as this does not mess up the encoding and keeps the special names as they should be. Further, I added a function where the adm1 units present in SPAM but not in GGS, are merged so that the production and land are correct on country level (relevant for England, Malawi, Cze, Hnd, etc. -> check documetnation for more detials.

I finished the ADM1 zones based on SPAM and the GGS from Ulrike (SPAM operator). The Final adm1 zone file is exported as csv and shp. Please mind the encoding as otherwise the names from nordic and slavic and french places might be distorted.

Changed the pipe to not using assigments --> like this it prevents from using too much memory and writes it directly to the drive. I also added the physical land and assumed the tnuts share for the harvested area to be the same for phys area. Yields are excluded.