Commit bef7478d authored by Jorge S. Mendes de Jesus's avatar Jorge S. Mendes de Jesus
Browse files

update docs and Dockerfile

parent 27b0b8a5
Pipeline #25319 passed with stage
in 1 minute and 20 seconds
......@@ -10,13 +10,6 @@ LABEL "maintainer"=""
ENV DEBIAN_FRONTEND noninteractive
ENV DB_USER hass005
ENV MONOLITH_PATH /data/monoliths
RUN apt-get update \
&& apt-get install -y --no-install-recommends postgresql-client build-essential libpq-dev tini
World Soil Museum - REST API
App structure
## App structure
REST APP is on [](./ and then calling modules like `resources`. This app is based on:
......@@ -25,18 +23,23 @@ App accesses ISRIC database on ``.
**NOTE:** To copy files check section below [Monolith picture location](#monolith-picture-location)
### DB access configuration / Data path
### DB access configuration
DB options and monolith path location is picked up from env variables on script: [common/](./common/
DB options are picked up from file [common/](./common/ This file needs to be changed if a new DB is used.
NOTE: Jorge did a bad job on leaving DB password on the repository. Password should be on envs.
### DB queries
In the code we have the following queries:
| code path | method/function | SQL |
| common/ | filter_classification()|select * from web_isis.vw_isis_profiles_with_classification_geo_img_rest where classification ilike %s"|
......@@ -44,40 +47,35 @@ NOTE: Jorge did a bad job on leaving DB password on the repository. Password sho
**NOTE:** Views need to be refreshed when new data is added to the upstream tables
Monolith picture location
### Monolith picture location
Monoliths used by REST are stored on ISRIC data volume, the access to this volume is very restricted and based on IP. Rserver (, on this server users haas005, turku001 and ruipe001 have a symbolic link to the monoliths.
There the monolith pictures can be updated or modified
Monolith file copy
### Monolith file copy
On the Rserver is it better to copy the monoliths from workspace scratch into `ISRIC_Monoliths`. On the home folder of user the command would look like:
turdu001@scomp1457:~$ cp -rv ./ISRIC_Workspace/scratch/Upload_ToRest/combined_processed/* ./ISRIC_Monoliths/
The FB-IT mounts are not reliable, check if "ISRIC_Monolith" is accessible, it not mount it:
# mount all volumes from /etc/fstab
sudo mount -a
In case of batch file permission it is advisable (NOTE: Only sudo can do it):
find /mnt/ISRIC_Monoliths -type d -exec chmod 777 {} \; #for directories
find /mnt/ISRIC_Monoliths -type f -exec chmod 666 {} \; #for files
Monolith image update and implementation
## Monolith image update and implementation
To update monolith images, it is necessary:
......@@ -89,8 +87,7 @@ It is advisable to use a rsync software when moving files around.
REST API doesn't make any assumptions on files and will crawl the fold structure for the pictures. It maybe necessary some code changes if there are to many changes.
Local Deployment
## Local Deployment
This API can be deployed using a docker container. The code resides in the
host system, together with the monoliths images to be served. These images
......@@ -100,38 +97,45 @@ as this file). This code folder is then mapped into the container at startup.
The Dockerfile present in this project sets up the API with Gunicorn and nginx.
To build the container image change to the project folder and run:
`docker build -t ldesousa/ .`
`docker build -t .`
The resulting image is named `wsm/rest`.
The resulting image is named `ldesousa/`.
To run the image it is necessary to map the right environment variables to connect to the database as well as mount the source code folder and bind port 80 (where nginx listens):
To run the image it is necessary to map the right environment variables to connect to the database as well as mount the source code folder and bind port 8000 (where gunicorn listens):
Docker run command with envfile:
docker run -dit --env-file /path/to/envfile -v /data/monoliths:/data/monoliths --name -p 8081:80 ldesousa/
docker run -dit \
--env-file /path/to/envfile \
-v /data/monoliths:/data/monoliths \
--name -p 8000:8000
Docker run command with variables inline:
docker run -dit -e -e DB_NAME=isric -e DB_USER=haas005 -e DB_PASSWORD=wur -e DB_PORT=5479 -v /data/monoliths:/data/monoliths --name -p 8081:80 ldesousa/
docker run -dit \
-e \
-e DB_NAME=isric \
-e DB_USER=haas005 \
-e MONOLITH_PATH=/data/monolith
-e DB_PORT=5479 \
-v /data/monoliths:/data/monoliths
--name -p 8000:8000
Your container will now be available on [localhost:8081](http://localhost:8081)
OC deployment
OC uses the gitlab webhook to deploy the application on project `isric-files`. Pod application is named `rest-wsm`, and external route is ``.
Your container will now be available on [localhost:8000](http://localhost:8000)
Image build is based on the `python3.6` OC image `openshift/python:3.6` and WSGI system is configure with envs:
## K8s deployment
- `ENV` variables are set as k8s secrets on the [ISRIC deployment repository](
Envs are stored on configuration file [here](.s2i/environment)
- Application follows the standard k8s deployment.
Pod mounts the `isric-files/monoliths` on `/opt/app-root/src/monoliths (ro)`
- TODO: Write about semanatic tagging
Image processing and metadata
## Image processing and metadata
Please consult the specific folder **[README](imageProcessing/**
"""Password of database"""
"""This is just an example, change to the correct values and rename the file to"""
DB={"host":"","database":"isric", "user":"isis", "password":"@#%951GHKaef"}
\ No newline at end of file
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment