diff --git a/README.md b/README.md
index a5594c36c2f887d22d71635059020ef9a30e3bf6..cea1eb130965bb6064683bbb78589fc0989f25d1 100644
--- a/README.md
+++ b/README.md
@@ -20,9 +20,9 @@ The ClearML server is setup in the following way:
You can use any s3 interface, however we will show how to setup CloudBerry as a example:
-- Go to https://www.msp360.com/explorer/ for windows or macOS users
-- Download relevant installer
-- Install installer
+- Go to https://www.msp360.com/explorer/ for windows or macOS users
+- Download relevant installer
+- Install installer
- Start CloudBerry explorer
- Click File -> Add New Account
@@ -33,10 +33,10 @@ You can use any s3 interface, however we will show how to setup CloudBerry as a
- Put in **Display name:** name of your choice
-- Put in **Server:** the endpoint with https://
-- Put in **Access Key ID:** the s3 access key
+- Put in **Server:** the endpoint with https://
+- Put in **Access Key ID:** the s3 access key
- Put in **Secret Access Key:** the s3 secret key
-- Click Test connection
+- Click Test connection
- Continue on successful test
@@ -44,23 +44,31 @@ You can use any s3 interface, however we will show how to setup CloudBerry as a
- Select your s3 account via drop down menu at source
- You can now drag files/folders to s3 storage or move files from one source to s3 via interface.
-
+
## Setup ClearML client
-NOTE: It is advised to create a new folder in the s3 storage for file storage from the AI platform. This will keep the root folder of the s3 storage cleaner.
+NOTE: It is advised to create a new folder in the s3 storage for file storage from the AI platform. This will keep the root folder of the s3 storage cleaner.
- pip install -r requirements.txt
- Login to webapp (for example: https://app.clearml.containers.wurnet.nl)
- Create workspace credentials
- Settings -> Workspace -> Create new credentials (optional add label)
- - copy code for later use
-- Go to a CLI on your computer
-- run command: **clearml-init**
- - copy code in clearml-init prompt in your terminal
-- There is no fileserver provided by FB-IT with the clearml workspace (this would limit scalability and control for researchers). We need to connect the s3 storage to the clearml workspace. Lookup clearml.conf in your User folder and change to your provided credentials:
- - files_server: "s3://endpoint_here_without_https:443/bucket_name_here"
- - NOTE: if you created a folder in your s3 storage for file storage from the AI platform, you need to put the folder name behind the files_server URI: "s3://endpoint_here_without_https:443/bucket_name_here/folder_name"
+ - copy api credentials (access_key and secret_key) for later use
+- copy **clearml.conf** in your home directory
+ - Linux: ~/clearml.conf
+ - Mac: $HOME/clearml.conf
+ - Windows: \User\username\clearml.conf
+- Put in your project specific variables in the clearml.conf inside your home directory:
+ - api {
+ api_server: API_SERVER
+ web_server: WEB_SERVER
+ files_server: "FILE_SERVER"
+ # input your api credentials generated in webapp here
+ credentials {"access_key": "ACCESSKEY", "secret_key": "SECRETKEY"}
+ }
+
+- There is no fileserver provided by FB-IT with the clearml workspace (this would limit scalability and control for researchers). We need to connect the s3 storage to the clearml workspace. Lookup clearml.conf again in your home folder and change to your provided credentials:
- aws {
s3 {
@@ -92,6 +100,7 @@ You can deploy agents on your varying compute resources as long as it has access
- api: api.[URL provided to you] (for example: https://api.clearml.containers.wurnet.nl/)
- file: s3://endpoint_here_without_https:443/bucket_name_here
- Put in Git credentials
+ - NOTE: You can also put in [git access tokens](https://clear.ml/docs/latest/docs/clearml_agent/#adding-clearml-agent-to-a-configuration-file) with read access to the repository. In this way you do not have to input your personal git credentials. input your deploy token username as username and the access token itself as password.
4. Lookup clearml.conf in the agents User folder and change to your provided credentials:
- aws {
s3 {
@@ -111,13 +120,13 @@ You can deploy agents on your varying compute resources as long as it has access
## Example task run
-The task run can be used for running a single script. It runs locally by default, but if the **remote_queue** argument is input, the first epoch will run locally, but following epochs will run by the agent listening to the remote queue.
+The task run can be used for running a single script. It runs locally by default, but if the **remote_queue** argument is input, the first epoch will run locally, but following epochs will run by the agent listening to the remote queue.
- Run task_run.py
- You can change arguments inside the script
- - change **remote_queue** argument to run script on agent that is listening to queue.
+ - change **remote_queue** argument to run script on agent that is listening to queue.
-## Example pipeline
+## Example pipeline
- Run stage one script via: python stage_one.py
- You are able to change arguments inside the script
diff --git a/clearml_example.conf b/clearml.conf
similarity index 94%
rename from clearml_example.conf
rename to clearml.conf
index d33981514f3aa91b9b82404469af05d1c1960156..53ac29fc4b22cdf2e98456f0f155ae0919fd8f09 100644
--- a/clearml_example.conf
+++ b/clearml.conf
@@ -1,9 +1,9 @@
# ClearML SDK configuration file
api {
- api_server: https://api.clearml.containers.wurnet.nl
- web_server: https://app.clearml.containers.wurnet.nl
- files_server: "s3://endpoint:443/bucket"
- # Credentials are generated using the webapp, https://app.clearml.containers.wurnet.nl/settings
+ api_server: API_SERVER
+ web_server: WEB_SERVER
+ files_server: "FILE_SERVER"
+ # input your api credentials generated in webapp here
credentials {"access_key": "ACCESSKEY", "secret_key": "SECRETKEY"}
}
sdk {