Deploying on GCP¶
The following instructions create a Kubernetes (k8s) cluster with a Quetzal server running on the staging configuration. Change the sandbox- references to prod- to create a production server.
We need to do perform install several components: a k8s cluster, helm, ingress, certbot and the quetzal application.
Docker images¶
Follow the Local development server instructions and make sure that you are able to run and launch a development environment. You will need to activate your virtual environment.
Read and follow the Google Cloud Platform preparations. You will need to have a gcloud correctly configured, a JSON credentials file, and a reserved external IP address.
Build and upload the Docker container images to Google Container Registry.
$ flask quetzal deploy create-images \ --registry eu.gcr.io/<your-project-id>
Kubernetes cluster¶
Create a kubernetes cluster using gcloud:
gcloud container clusters create quetzal-cluster \ --num-nodes=1 \ --enable-autoscaling --min-nodes=1 --max-nodes=4
Verify that the cluster is up and running:
gcloud container clusters list NAME LOCATION MASTER_VERSION MASTER_IP MACHINE_TYPE NODE_VERSION NUM_NODES STATUS quetzal-cluster europe-west1-c 1.11.7-gke.4 x.x.x.x n1-standard-1 1.11.7-gke.4 2 RUNNING
If you need more resources, you can change the number of nodes with:
gcloud container clusters resize quetzal-cluster --size N
or change the type of VM instance type for another machine type that uses more CPU or memory. This procedure is out of scope of this guide, but you can read more at the node pools documentation.
Verify that
kubectl
is using the correct cluster:kubectl config get-contexts
Part 2: Helm¶
Install helm. In general, follow the installing helm guide. For the particular case of OSX (with homebrew), this can be done with:
brew install kubernetes-helm
Install helm k8s service account. This is explained in the helm installation guide:
kubectl create -f helm/rbac-config.yaml
Install helm k8s resources (also known as tiller) with a service account:
helm init --service-account tiller --wait
Verify that helm was correctly installed:
helm version Client: &version.Version{SemVer:"v2.14.3", GitCommit:"0e7f3b6637f7af8fcfddb3d2941fcc7cbebb0085", GitTreeState:"clean"} Server: &version.Version{SemVer:"v2.14.3", GitCommit:"0e7f3b6637f7af8fcfddb3d2941fcc7cbebb0085", GitTreeState:"clean"}
Part 3: Ingress¶
Install ingress resources. This is a prerequisite described in the ingress installation guide.
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/mandatory.yaml
- 2 Install ingress. If you have a static IP reserved for the Quetzal
application, you must set it here. Otherwise, remove the
--set controller.service.loadBalancerIP
flag:helm install stable/nginx-ingress --set controller.service.loadBalancerIP=X.X.X.X --name nginx-ingress
Certbot¶
This part is optional. You only need it if you want to have a signed certificate.
Install certbot. This part was inspired from the certbot acme nginx installation tutorial.
# Install the cert-manager CRDs. We must do this before installing the Helm # chart in the next step for `release-0.9` of cert-manager: kubectl apply -f https://raw.githubusercontent.com/jetstack/cert-manager/release-0.9/deploy/manifests/00-crds.yaml # Create the namespace for cert-manager kubectl create namespace cert-manager # Label the cert-manager namespace to disable resource validation kubectl label namespace cert-manager certmanager.k8s.io/disable-validation=true ## Add the Jetstack Helm repository helm repo add jetstack https://charts.jetstack.io ## Updating the repo just incase it already existed helm repo update ## Install the cert-manager helm chart helm install \ --name cert-manager \ --namespace cert-manager \ --version v0.9.1 \ jetstack/cert-manager
Customize certbot issuer definition declared on the
helm/acme-issuer.yaml
file and install it:kubectl create -f helm/acme-issuer.yaml
Quetzal¶
Create the TLS secret that will be used for the nginx proxy.
kubectl create secret tls sandbox-tls-secret \ --cert=./conf/ssl/mysite.crt \ --key=./conf/ssl/mysite.key
Create GCP credentials secret that will be used by the app to communicate with the GCP resources (e.g. the data buckets).
kubectl create secret generic sandbox-credentials-secrets \ --from-file=./conf/credentials.json
Generate some passwords. You can do this manually, or use the following helper commands. Keep them secret, keep them safe.
# Generate a random password for the database user. flask quetzal utils generate-secret-key 8 YRADKSrPzlM # Generate a secret key for the Flask application. flask quetzal utils generate-secret-key sB-YgPO8ZVCmZyV5XKH0rg
Install quetzal using helm. Give it a name (like foo) and use the passwords generated in the previous step. Verify that all the configuration values in
helm/quetzal/values.yaml
. Also verify thehelm/quetzal/templates/ingress.yaml
file.helm install \ --set db.username=... \ --set db.password=... \ --set app.flaskSecretKey=... \ --name foo ./helm/quetzal
Note that it is at this point that you will set a database username, password and flask secret.
Verify that everything is running.
You can check that all pods are running with:
kubectl get pods NAME READY STATUS RESTARTS AGE foo-quetzal-app-86669c8bc6-8vt9c 0/1 Pending 0 100s foo-quetzal-app-86669c8bc6-dhwj6 1/1 Running 0 10m foo-quetzal-app-86669c8bc6-s56wl 0/1 Pending 0 115s foo-quetzal-app-86669c8bc6-w2ppm 0/1 Pending 0 115s foo-quetzal-app-86669c8bc6-x5gvk 0/1 Pending 0 115s foo-quetzal-db-cd68d97bc-tdj8l 1/1 Running 0 15m foo-quetzal-rabbitmq-85bf9dddfd-kkvr7 1/1 Running 0 15m foo-quetzal-worker-5dbb8c4dfd-fg8ct 1/1 Running 0 9m41s foo-quetzal-worker-5dbb8c4dfd-fv6bj 1/1 Running 0 10m nginx-ingress-controller-84df6c4c54-2v8n4 1/1 Running 0 22m nginx-ingress-default-backend-7d5dd85c4c-mc89t 1/1 Running 0 22m
Similarly, you can do the same with the services:
kubectl get services NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE app ClusterIP 10.0.11.94 <none> 5000/TCP 16m db ClusterIP 10.0.13.162 <none> 5432/TCP 16m kubernetes ClusterIP 10.0.0.1 <none> 443/TCP 26m nginx-ingress-controller LoadBalancer 10.0.3.187 x.x.x.x. 80:31388/TCP,443:32725/TCP 23m nginx-ingress-default-backend ClusterIP 10.0.11.182 <none> 80/TCP 23m rabbitmq ClusterIP 10.0.10.159 <none> 5672/TCP,15672/TCP 16m
If a pod fails to start correctly, examine it with:
kubectl describe pod foo-quetzal-app-7dcc756c9d-78n5w ... many details that can help determine the problem ...
Initialize the application.
If this is the first time the application is deployed, you need to initialize its database, buckets and users. Connect to a web pod (like
foo-quetzal-app-7dcc756c9d-78n5w
, as listed above, but this will be specific to your deployment) as:kubectl exec -it foo-quetzal-app-7dcc756c9d-78n5w /bin/bash
and then run the initialization script:
./init.sh
which will ask for an administrator password. You can add new users at this point with:
flask quetzal user create alice alice.smith@example.com flask quetzal role add alice public_read public_write
If you installed certbot, you should verify that the certificate was correctly generated with:
kubectl get certificates NAME READY SECRET AGE sandbox-tls-secret True sandbox-tls-secret 1m
And also, the following curl command should work without any errors:
curl -vL https://sandbox.quetz.al/healthz
That’s all, you can now explore the documentation at https://sandbox.quetz.al/redoc, or wherever your configuration points to.