Kubernetes
Kubernetes Cluster used as infrastructure to deploy Risk Engine components
Introduction
The Risk Engine relies on Docker to package the different services; and on Kubernetes to deploy them.
We use Kubernetes Engine (GKE), a GCP service, to manage our Kubernetes clusters.
For now, we have one cluster deployed in us-central-1` region; but we will deploy a second one later in another region, keeping the first one for staging and using the new one for production.
As you can see in the screenshot below, for now we only have an airflow deployed permanently there; but it spins up other images when running its tasks, that automatically terminate when done. This allows us to run tasks in parallel and only consume resources when they are actually performing work.
Deployment descriptors
Kubernetes Deployment files allow us to specify how an image is to be deployed to staging or production; so that it can be automated and perfectly replicated if needed.
Here is an example of such a configuration file for the repository ds_collateral_features:
Last updated