Can't Connect to Airflow Running in Docker Container When Started in Detached Mode

Question

I'm running Airflow in a docker container using Docker Compose to start things. The system consists of a container running Postgres, and another running the Airflow webserver. When I issue start the containers in detached mode -d I can't connect to the web server. I go to http://127.0.0.1:8080, but it comes up with nothing. Example:

docker-compose --env-file .env up -d

The container is running. I can view the logs docker logs airflow_webserver and I can exec into the container and poke around.

When I start the containers without the -d flag it works fine. I can go to 127.0.0.1:8080 and access the app no problem.

One other note, I only have this issue when running on an EC2 instance with Ubuntu. Running on my local computer (Mac) I have no issues.

Here is the docker-compose and Docker files

Docker Compose

version: '3.7'
services:
    postgres:
        container_name: airflow_pg
        image: postgres:9.6
        env_file:
            - ../.env
        ports:
            - "5432:5432"
        volumes:
            - "${LOCALDATABASE}:/var/lib/postgresql/data/pgdata:Z"
            - ../airflow/logs:/home/airflow/logs:z
        logging:
            options:
                max-size: 10m
                max-file: "3"

    webserver:
        container_name: airflow_webserver
        user: "${USERID}:1000"
        build:
            context: ../
            dockerfile: airflow/DockerAirflow
            args:
                AIRFLOW_UID: "1000"
                AIRFLOW_GID: "1000"
        restart: on-failure
        env_file:
            - ../.env
        depends_on:
            - postgres
        logging:
            options:
                max-size: 10m
                max-file: "3"
        volumes:
            - ../airflow/dags:/opt/airflow/dags:Z
            - ../airflow/logs:/opt/airflow/logs:Z
            - ../app:/opt/airflow/app:Z
            - "${SSHKEYPATH}:/var/ssh/cbc-pwa-us-east-2-bastion.pem:Z"
        ports:
            - "8080:8080"

        entrypoint: /opt/airflow/start_airflow.sh
        healthcheck:
            test: ["CMD-SHELL", "[ -f /usr/local/airflow/airflow-webserver.pid ]"]
            interval: 30s
            timeout: 30s
            retries: 3

Dockerfile

FROM apache/airflow:master-python3.8
USER root
RUN curl -L https://s3.amazonaws.com/redshift-downloads/drivers/odbc/1.4.17.1000/AmazonRedshiftODBC-64-bit-1.4.17.1000-1.x86_64.deb > ~/redshift.deb
RUN dpkg -i ~/redshift.deb
RUN apt update -y && sudo apt install build-essential -y
RUN sudo apt install -y procps
RUN usermod -u 1000 airflow
RUN groupmod -g 1000 airflow
RUN chown airflow:airflow /opt/airflow
COPY ./airflow/start_airflow.sh /opt/airflow/start_airflow.sh
RUN chmod a+x /opt/airflow/start_airflow.sh
USER 1000
ADD ./requirements.txt requirements.txt
RUN pip install convertdate lunarcalendar holidays tqdm pystan
RUN pip install -r requirements.txt

You may notice some oddities with the user groups. That part is working now, but there was some weirdness with the Airflow image using 50000 as the UID. For some reason the ARG wasn't taking so some more direct manipulation was needed. I don't think this is the issue, but as I'm not sure what's wrong...

question from:https://stackoverflow.com/questions/65947436/cant-connect-to-airflow-running-in-docker-container-when-started-in-detached-mo

Answer 1

This is likely related to the permissions of the ../airflow/logs:/opt/airflow/logs:Z volume mount.

When you run via Docker for Mac, the UID is always translated by the VM, but when running a docker container directly on Linux, the UID in the container has to match exactly the UID:GID that can write to the files on the host.

You should be able to see logs showing a permissions error -- try docker logs and see.