django - Can't collectstatic to s3 via Heroku using boto - s3 bucket returns a NoneType

Question

this question pertains to a setup using Heroku, django-storages (w/ boto for s3), and CloudFront to serve static content.

I've been trying for the past several hours to successfully load up my static files to my Heroku app. I have successfully got Cloudfront hooked to my s3 bucket, and it seems as though the bucket is set up properly, but for whatever reason, my value for AWS_STORAGE_BUCKET_NAME does not seem to registering properly.

If anyone has any clues or ideas on how to debug this, I'd be much obliged. I am at my wit's end. Thanks for reading.

settings.py (the important stuff):

try:
  from settings_local import *
except:
  import s3utils
DEBUG = False
#s3 stuff
DEFAULT_FILE_STORAGE = 's3utils.MediaRootS3BotoStorage'
STATICFILES_STORAGE = 's3utils.StaticRootS3BotoStorage'  
STATIC_URL = 'https://[domain].cloudfront.net/'
#use heroku postgres database
import dj_database_url
DATABASES['default'] =  dj_database_url.config()

s3utils.py

from storages.backends.s3boto import S3BotoStorage
from django.utils.functional import SimpleLazyObject
import os

AWS_ACCESS_KEY_ID = os.environ['AWS_ACCESS_KEY_ID']
AWS_SECRET_ACCESS_KEY = os.environ['AWS_SECRET_ACCESS_KEY']
AWS_STORAGE_BUCKET_NAME = 'static.[website].org'

StaticRootS3BotoStorage = lambda: S3BotoStorage(location='static')
MediaRootS3BotoStorage  = lambda: S3BotoStorage(location='media')

This is the traceback I get when attempting to collectstatic, either via 'heroku run' or in the Procfile:

  Traceback (most recent call last):
  File "manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/app/.heroku/python/lib/python2.7/site-packages/django/core/management/__init__.py", line 443, in execute_from_command_line
    utility.execute()
  File "/app/.heroku/python/lib/python2.7/site-packages/django/core/management/__init__.py", line 382, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/app/.heroku/python/lib/python2.7/site-packages/django/core/management/base.py", line 196, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/app/.heroku/python/lib/python2.7/site-packages/django/core/management/base.py", line 232, in execute
    output = self.handle(*args, **options)
  File "/app/.heroku/python/lib/python2.7/site-packages/django/core/management/base.py", line 371, in handle
    return self.handle_noargs(**options)
  File "/app/.heroku/python/lib/python2.7/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 163, in handle_noargs
    collected = self.collect()
  File "/app/.heroku/python/lib/python2.7/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 113, in collect
    handler(path, prefixed_path, storage)
  File "/app/.heroku/python/lib/python2.7/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 287, in copy_file
    if not self.delete_file(path, prefixed_path, source_storage):
  File "/app/.heroku/python/lib/python2.7/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 215, in delete_file
    if self.storage.exists(prefixed_path):
  File "/app/.heroku/python/lib/python2.7/site-packages/storages/backends/s3boto.py", line 284, in exists
    return k.exists()
  File "/app/.heroku/python/lib/python2.7/site-packages/boto/s3/key.py", line 399, in exists
    return bool(self.bucket.lookup(self.name))
  File "/app/.heroku/python/lib/python2.7/site-packages/boto/s3/bucket.py", line 148, in lookup
    return self.get_key(key_name, headers=headers)
  File "/app/.heroku/python/lib/python2.7/site-packages/boto/s3/bucket.py", line 181, in get_key
    query_args=query_args)
  File "/app/.heroku/python/lib/python2.7/site-packages/boto/s3/connection.py", line 458, in make_request
    auth_path = self.calling_format.build_auth_path(bucket, key)
  File "/app/.heroku/python/lib/python2.7/site-packages/boto/s3/connection.py", line 92, in build_auth_path
    path = '/' + bucket
TypeError: cannot concatenate 'str' and 'NoneType' objects

Note that I've omitted domain names and such, I don't actually have [domain] or [website] in the code.

Answer 1

You are not saying from s3utils import *, so AWS_STORAGE_BUCKET_NAME is never imported into the settings module.

S3BotoStorage will pull AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY from environment variables, but AWS_STORAGE_BUCKET_NAME must be set in settings.py. This seems like a bizarre inconsistency, but I think it is because AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY are actually boto parameters and AWS_STORAGE_BUCKET_NAME is not (boto will pull credentials from env vars).*

The other references to s3utils are explicit:

DEFAULT_FILE_STORAGE = 's3utils.MediaRootS3BotoStorage'
STATICFILES_STORAGE = 's3utils.StaticRootS3BotoStorage'

So, the only setting that isn't taken care of is AWS_STORAGE_BUCKET_NAME, and that causes your error.

*I'd like to see django-storages accept other settings from env vars (12-Factor App, anyone?) and am considering opening an issue/submitting a pull request to that effect.