#### Note that it is expected that your working directory is currently the `exec` folder!
The preprocessing pipeline has several options for estimating CO2 emissions. Different pipelines are specified in `da/pipelines/nrtpipeline.py`.
Current options include:
- Poor mans inversion global
- Statistical fit global
- High-resolution over Europe
These options can be combined at will.
The individual building blocks of the pipelines are specified in `da/preprocessing/inputs.py`. These building blocks are functions that initialise specific classes that create the specific fluxes (e.g. Ocean for oceanic fluxes).
The classes are created in their own files (e.g. the Ocean class has a file `ocean.py`). Here, both a global and regional version of the class is specified. Also different flavours can be added, for example using differnt meteorological models. All classes inherit from the base classes `Regional` or `Global`, which are created in `classes.py`. In these baseclasses, the spatial and temporal domains are stored, and different functions for handling the data (e.g. writing, or comining regional with global data). For the fluxes, the following files exist:
- Biosphere: `upscale_bio.py` (both high-res and statistical fit biosphere fluxes); `poormans.py` (poormans inversion fluxes only)
- FF: `emissionmodel.py`
- Ocean: `ocean.py`
- Fire: `fire.py`
The `da/preprocessing/` folder also contains python files for downloading and preprocessing meteorological data (ERA, MERRA, GPCP(outdated)) and running SiB4. Currently, these files are functional, so no classes. Furthermore, the folder contains the additional files `gpcp_targetgrid.txt`, which specifies a grid to regrid GPCP data to (outdated), `europe.grid`, which specifies a grid to regrid any data source to the regional grid (used for meteo data), and `category_info.py`, which gives specific information on different categories to use in the emission model.
## Usage of the preprocessing pipeline
1. Setup the correct pipeline in `da/pipelines/nrtpipeline.py`. This pipeline can be elaborate in `da/preprocessing/inputs.py`
1. Check the run-control files. These are `da/rc/nrt.rc`, in which infomration on the run is stored, and `near-real-time.rc`, in which the time of the run is specified. Note that the variables in `da/rc/nrt.rc` do not neccessarily need to be all present for all runs and functionalities.
1. Import the correct pipeline in `near-real-time.py` and run the python script (or the jobscript), specify the correct rc file
1. If all worked, the output should be stored in `../output/{yyyymmdd}/` as nc files.
