Project

General

Profile

Feature #1542

OpenAtlas on Kubernetes

Added by Alexander Watzinger 4 months ago. Updated 10 days ago.

Status:
In Progress
Priority:
Low
Category:
Administration
Target version:
Start date:
2021-06-30
Estimated time:

Description

We will try to adapt and implement OpenAtlas with Kubernetes at the ACDH-CH. See working draft

History

#1

Updated by Alexander Watzinger 3 months ago

Today we have a Kubernetes meeting

#2

Updated by Alexander Watzinger 3 months ago

Today I did some cleanup at the ACDH-CH GitLab repository. I deleted the main and develop branch and uploaded them again because, like we discussed, Kubernetes specific changes should only be made in the feature_kubernetes branch. Once everything is working with Kubernetes we may look into merging it to main but have to do this carefully to be sure to not put our productive systems at risk, e.g. now it seems like passwords may be leaked, more about that further down.

Next I wanted to merge our current develop branch to feature_kubernetes. Originally it was a copy of main but I think it would be better to be a branch of develop to have the latest changes already merged. However, when pulling the latest feature_kubernetes form GitLab I noticed a few problematic changes made there:
  • A SQL dump was added to the Git repository by Berni (install/demo-dev-dump.sql) which shouldn't be part of the OpenAtlas repository so please clean this up (e.g. delete and put in .gitignore).
  • Dalibor removed the instance/prodution.py from .gitignore, presumably to change configuration. This is not how it is supposed to work, e.g. passwords are saved in instance/production.py. If persistent changes are needed in the repository they should be made in config/default.py or, if they are Kubernetes specific we should add a config/kubernetes.py.
Please keep in mind Berni and Dalibor that we don't want a super special Kubernetes version that is incompatible with our already proofed, working and productive OpenAtlas main version. So next step would be:
  • fixing .gitignore to ignore instance/ again except specified files (see .gitignore in main branch for how it should be)
  • removing database dump from repository
  • merge develop to feature_kubernetes to be in sync with current development

However, it's great to see that you managed to get already something running at https://demo-acdh-ch.openatlas.eu/.

#3

Updated by Dalibor Pancic 3 months ago

Dalibor removed the instance/prodution.py from .gitignore, presumably to change configuration. This is not how it is supposed to work, e.g. passwords are saved in instance/production.py. If persistent changes are needed in the repository they should be made in config/default.py or, if they are Kubernetes specific we should add a config/kubernetes.py.

The openatlas app must be reconfigured that it can check if variables introduced over instance/prodution.py in feature_kubernetes are present as env variables.
They have to be introduced as Gitlab CI/CD variables for feature_kubernetes branch. By default, Openatlas expects that variables are hard-coded in the instance/prodution.py.
We can return instance/prodution.py to .gitignore after we adjust Openatlas that it can take variables from the "os.environ"

DATABASE_NAME = os.environ.get('POSTGRES_DB')              
DATABASE_USER = os.environ.get('POSTGRES_USER') 
DATABASE_PASS = os.environ.get('POSTGRES_PASSWORD') 
DATABASE_HOST = os.environ.get('POSTGRES_HOST') 
DATABASE_PORT = os.environ.get('POSTGRES_PORT')
DATABASE_URL = os.environ.get('DATABASE_URL')
SECRET_KEY = os.environ.get('SECRET_KEY')
#4

Updated by Alexander Watzinger about 2 months ago

  • Description updated (diff)
#5

Updated by Dalibor Pancic about 2 months ago

New domain https://discovery-demo-acdh-ch.openatlas.eu is created, pointed to the cluster and can be used for deployment.

#6

Updated by Dalibor Pancic about 2 months ago

The problem with the production.py is solved and production.py is returned to the .gitignore.
All Kubernetes related configuration is introduced over requirements.txt, Procfile, start.sh and Gitlab CI/CD evironment variables.
Only these three new files are added to the default Openatlas repo, and they are located in the root of the feature_kubernetes branch.

In order to merge the develop with the feature_kubernetes branch, we just need to remove install/demo-dev*.sql dumps.

#7

Updated by Alexander Watzinger about 2 months ago

Thanks a lot Dalibor for solving this. I really hope we see you at our OpenAtlas Summer Meeting next week. Would be nice to see in person again and your first drinks will be on us for all your help :)

#8

Updated by Alexander Watzinger about 1 month ago

  • Target version changed from Wishlist to 252

Moved this issue to the INDIGO roadmap version because we will need it for that cooperation first.

#9

Updated by Alexander Watzinger 10 days ago

  • Target version changed from 252 to Wishlist

Moving to wishlist again while resolving the INDIGO version.

Also available in: Atom PDF