azure - Inject env variable into build stage of image

Question

Working on a CI/CD pipeline for a Django based API. I need to inject an environmental variable during the build stage to successfully build the image.

• python manage.py collectstatic is run in the Dockerfile when the image is being built
• collectstatic needs the SECRET_KEY in the settings.py to run
• SECRET_KEY is empty during this stage because it is read in with os.environ['SECRET_KEY]
• Because it is empty, the build fails
• Thus, I need to set the environment to have this variable

I am having trouble putting that much together, so wanted to see if anyone could assist me.

This is what I have so far:

Azure Pipeline

azure-pipelines.yml

trigger:
  branches:
    include:
    - master

resources:
- repo: self

variables:
  # Container registry service connection established during pipeline creation
  secretKey: $(SECRET_KEY)

  # Agent VM image name
  vmImageName: 'ubuntu-latest'

stages:
- stage: Build
  displayName: Build and push stage
  jobs:  
  - job: Build
    displayName: Build
    pool:
      vmImage: $(vmImageName)
    steps:
    - bash:
      env:
        SECRET_KEY: $(secretKey)
    - task: Docker@2
      displayName: Build and push api image to container registry
      inputs:
        command: buildAndPush
        repository: $(imageRepository)-api
        dockerfile: $(dockerfilePath)/api/Dockerfile
        containerRegistry: $(dockerRegistryServiceConnection)
        tags: |
          $(tag)
    - upload: manifests
      artifact: manifests

Dockerfile

FROM python:3.7-slim
ENV PYTHONUNBUFFERED 1
WORKDIR /app
EXPOSE 5000
COPY requirements*.txt ./
RUN pip install -r requirements.txt
COPY . .
RUN python manage.py collectstatic
CMD ["gunicorn", "-b", ":5000", "--log-level", "info", "config.wsgi:application"]

Build fail log (probably not helpful because I know why it failed... SECRET_KEY isn't in env vars)

Step 8/18 : RUN python manage.py collectstatic
 ---> Running in 1f42a5c062aa
Traceback (most recent call last):
  File "manage.py", line 21, in <module>
    main()
  File "manage.py", line 17, in main
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python3.7/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python3.7/site-packages/django/core/management/__init__.py", line 325, in execute
    settings.INSTALLED_APPS
  File "/usr/local/lib/python3.7/site-packages/django/conf/__init__.py", line 79, in __getattr__
    self._setup(name)
  File "/usr/local/lib/python3.7/site-packages/django/conf/__init__.py", line 66, in _setup
    self._wrapped = Settings(settings_module)
  File "/usr/local/lib/python3.7/site-packages/django/conf/__init__.py", line 157, in __init__
    mod = importlib.import_module(self.SETTINGS_MODULE)
  File "/usr/local/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/app/config/settings.py", line 26, in <module>
    SECRET_KEY = os.environ['SECRET_KEY']
  File "/usr/local/lib/python3.7/os.py", line 679, in __getitem__
    raise KeyError(key) from None
KeyError: 'SECRET_KEY'
The command '/bin/sh -c python manage.py collectstatic' returned a non-zero code: 1
##[error]The command '/bin/sh -c python manage.py collectstatic' returned a non-zero code: 1
##[error]The process '/usr/bin/docker' failed with exit code 1

I'm just not sure how to accomplish getting the environmental variable in there. My approach could be totally wrong from the beginning.

So how should I go about:

1. Declaring the environmental variables in the pipeline securely?
2. Passing said environmental variables into the build stage?

Thanks!

Answer 1

Your python manage.py collectstatic is running in the docker container, and its os.environ['SECRET_KEY] will try to get the environment variable of the container where it is running. But the environment variable SECRET_KEY you were setting in your pipeline is for build agent.

You can try following below steps to pass your pipeline env variable to the docker container.

1, I add an ARG and an ENV in your dockerfile ARG SECRET; ENV SECRET_KEY $SECRET

ENV SECRET_KEY refers to the value of ARG SECRET

FROM python:3.7-slim

ARG SECRET
ENV SECRET_KEY $SECRET

ENV PYTHONUNBUFFERED 1
WORKDIR /app
EXPOSE 5000
COPY requirements*.txt ./
RUN pip install -r requirements.txt
COPY . .
RUN python manage.py collectstatic
CMD ["gunicorn", "-b", ":5000", "--log-level", "info", "config.wsgi:application"]

2, I separate docker buildandpush task to dock build and dock push, as buildandpush command cannot accept arguments.

In docker build task. I passed the variable secretKey to the arguments field --build-arg SECRET=$(secretKey). So that when docker run build, ARG SECRET will be replaced by secretKey. And it will be passed to the ENV SECRET_KEY as defined in above dockerfile. So that SECRET_KEY will be set to the docker container's Environment variable.

Then you python code should be able to get the environment variable's value using os.environ['SECRET_KEY]

stages:
- stage: Build
  displayName: Build and push stage
  jobs:  
  - job: Build
    displayName: Build
    pool:
      vmImage: $(vmImageName)
    steps:
    - task: Docker@2
      displayName: Build
      inputs:
        command: build
        repository: $(imageRepository)-api
        dockerfile: $(dockerfilePath)/api/Dockerfile
        containerRegistry: $(dockerRegistryServiceConnection)

        arguments: --build-arg SECRET=$(secretKey)

        tags: |
          $(tag)

    - task: Docker@2
      displayName: Push
      inputs:
        command: push
        repository: $(imageRepository)-api
        containerRegistry: $(dockerRegistryServiceConnection)
        tags: |
          $(tag)


- upload: manifests
  artifact: manifests