Running rails + postgres in kubernetes: Connection refused

Question

My rails deployment is not able to connect to the postgres database. Error in the logs:

PG::ConnectionBad (could not connect to server: Connection refused
  Is the server running on host "db" (10.0.105.11) and accepting
  TCP/IP connections on port 5432?
):

This is my kubectl get services output:

NAME                     TYPE           CLUSTER-IP     EXTERNAL-IP   PORT(S)        AGE
load-balancer            LoadBalancer   10.0.5.147     60.86.4.33    80:31259/TCP   10m
db                       ClusterIP      10.0.105.11    <none>        5432/TCP       10m
kubernetes               ClusterIP      10.0.0.1       <none>        443/TCP        10m
web                      ClusterIP      10.0.204.107   <none>        3000/TCP       10m

db-deployment.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: db
  name: db
spec:
  replicas: 1
  selector:
    matchLabels:
      app: db
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: db
    spec:
      containers:
      - env:
        - name: POSTGRES_DB
          value: postgres
        - name: POSTGRES_HOST_AUTH_METHOD
          value: trust
        - name: POSTGRES_USER
          value: postgres
        - name: PGDATA
          value: /var/lib/postgresql/data/pgdata
        ports:
        - containerPort: 5432
        image: postgres
        imagePullPolicy: ""
        name: db
        resources: {}
        volumeMounts:
        - mountPath: /var/lib/postgresql/data
          name: postgres
      restartPolicy: Always
      serviceAccountName: ""
      volumes:
      - name: postgres
        persistentVolumeClaim:
          claimName: postgres
status: {}

db-service.yaml:

apiVersion: v1
kind: Service
metadata:
  name: db
  labels:
    app: db
spec:
  ports:
    - port: 5432
  selector:
    app: db
    tier: database

config/database.yml:

default: &default
  adapter: postgresql
  encoding: utf8
  host: db
  username: postgres
  pool: 5

development:
  <<: *default
  database: myapp_development

test:
  <<: *default
  database: myapp_test

production:
  <<: *default
  database: myapp_production

How do I ensure the postgres instance is running and accepting connections? Did I misconfigure something in the manifest files, or does the database.yml file need to point to a different host?

VRoxa · Accepted Answer · 2020-05-11 02:50:18Z

4

The first thing that jumps out at me is that your db service is targeting two selectors app: db and tier: database, however the corresponding deployment.yml only has the db label. You will need to add the tier label to your deployment template metadata so that the service will appropriately target the right pod.

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: db
    tier: database 
  name: db
spec:
  replicas: 1
  selector:
    matchLabels:
      app: db
      tier: database
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: db
        tier: database

In general, if a service is not connecting to a backend pod, you can easily diagnose the issue with a simple kubectl command.
This will tell you what current selectors are applied to your service

kubectl describe service db -o yaml

Then you can fetch the pods that the selectors refer to and make sure that this is returning something.

kubectl get pods -l app=db -l tier=database

Finally, I would recommend using the DNS name of the service setup through kube-proxy instead of the cluster IP address. I tend to view this as more resilient then a cluster up, as it will be automatically routed if the services IP ever changes. In your connection string use db:5432 instead of 10.0.105.11.

edited May 11, 2020 at 2:50

VRoxa

1,0519 silver badges27 bronze badges

answered May 10, 2020 at 18:20

Patrick Magee

3622 silver badges5 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

usbToaster Over a year ago

That works, thank you very much for the suggestion!

Collectives™ on Stack Overflow

Running rails + postgres in kubernetes: Connection refused

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related