Rails using db:migrate & db:seed on Kubernetes

This post assumes that you have basic understanding of Kubernetes terms like pods and deployments.

Problem

We want to deploy a Rails application on Kubernetes. We assume that the assets:precompile task would be run as part of the Docker image build process.

We want to run rake tasks such as db:migrate and db:seed on the initial deployment, and just db:migrate task on each later deployment.

We cannot run these tasks while building the Docker image as it would not be able to connect to the database at that moment.

So, how to run these tasks?

Solution

We assume that we have a Docker image named myorg/myapp:v0.0.1 which contains the source code for our Rails application.

We also assume that we have included database.yml manifest in this Docker image with the required configuration needed for connecting to the database.

We need to create a Kubernetes deployment template with the following content.

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: myapp
spec:
  template:
    spec:
      containers:
        - image: myorg/myapp:v0.0.1
          name: myapp
          imagePullPolicy: IfNotPresent
          env:
            - name: DB_NAME
              value: myapp
            - name: DB_USERNAME
              value: username
            - name: DB_PASSWORD
              value: password
            - name: DB_HOST
              value: 54.10.10.245
          ports:
            - containerPort: 80
      imagePullSecrets:
        - name: docker_pull_secret

Let's save this template file as myapp-deployment.yml.

We can change the options and environment variables in above template as per our need. The environment variables specified here will be available to our Rails application.

To apply above template for the first time on Kubernetes, we will use the following command.

$ kubectl create -f myapp-deployment.yml

Later on, to apply the same template after modifications such as change in the Docker image name or change in the environment variables, we will use the following command.

$ kubectl apply -f myapp-deployment.yml

After applying the deployment template, it will create a pod for our application on Kuberentes.

To see the pods, we use the following command.

$ kubectl get pods

Let's say that our app is now running in the pod named myapp-4007005961-1st7s.

To execute a rake task, for e.g. db:migrate on this pod, we can run the following command.

$ kubectl exec myapp-4007005961-1st7s                              \
          -- bash -c                                               \
          'cd ~/myapp && RAILS_ENV=production bin/rake db:migrate'

Similarly, we can execute db:seed rake task as well.

If we already have an automated flow for deployments on Kubernetes, we can make use of this approach to programmatically or conditionally run any rake task as per the needs.

Why not to use Kubernetes Jobs to solve this?

We faced some issues while using Kubernetes Jobs to run migration and seed rake tasks.

1. If the rake task returns a non-zero exit code, the Kubernetes job keeps spawning pods until the task command returns a zero exit code.

2. To get around the issue mentioned above we needed to unnecessarily implement additional custom logic of checking job status and the status of all the spawned pods.

3. Capturing the command's STDOUT or STDERR was difficult using Kubernetes job.

4. Some housekeeping was needed such as manually terminating the job if it wasn't successful. If not done, it will fail to create a Kubernetes job with the same name, which is bound to occur when we perform later deployments.

Because of these issues, we choose not to rely on Kubernetes jobs to solve this problem.