python - Resetting Django Migrations in a Production System

Question

I have been through a lot of posts/articles/trial-and-error involving Django Migrations since I started working with the framework a few years ago, so I decided I would post a self-answered question notating the proper way to accomplish the clean reset of migrations in a Production Database leaving you with the same database structure you left with, but a fresh start with initial migrations.

Overall the issue is this:

When you have a larger project you start to accumulate a large number of migrations for a system built with Django. This isn't normally an issue, but when you start accumulating upwards of 50-100 migration files (where a lot of them are the addition and removal of the same fields) it is nice to have a "cleaning" option as it should be well understood that if you alter migration history incorrectly, you will be left with a system that is more-or-less frozen in a previous database state where the only way to fix the issue is manual sql-based migration changes.

question from:https://stackoverflow.com/questions/65890758/resetting-django-migrations-in-a-production-system

Answer 1

The solution I have come up with for this issue comes in steps:

Step 1 Create migrations to delete any models or fields you want and run them locally, your dev system must be in sync with all other developer systems as well as production...if this is not the case you need to ensure it is!

Step 2

1. Delete the local migration files (a good option is a change of the command I have below, I currently have a directory structure with my applications for my system in a directory called /apps/)

Run calling python manage.py delete_local_migration_files (if you name it that way)

import os

import django.apps
from django.conf import settings
from django.core.management.base import BaseCommand


def delete_migrations(app):
    print(f"Deleting {app}'s migration files")
    migrations_dir = os.path.join(settings.BASE_DIR, f'apps{os.path.sep}{app}{os.path.sep}migrations')

    if os.path.exists(migrations_dir):
        for the_file in os.listdir(migrations_dir):
            file_path = os.path.join(migrations_dir, the_file)
            try:
                if os.path.isfile(file_path):
                    os.unlink(file_path)
            except Exception as e:
                print(e)

            f = open(f"{os.path.join(migrations_dir, '__init__.py')}", "w")
            f.close()

    else:
        print('-' * 20, migrations_dir, 'does not exist')


class Command(BaseCommand):
    """
    Resets migrations and clears directories
    """
    help = 'reset migrations'

    def handle(self, *args, **options):
        set_of_apps = set()
        disregard = []

        # get all apps
        for model in django.apps.apps.get_models():
            if model._meta.app_label not in disregard:
                set_of_apps.add(model._meta.app_label)

        for app in set_of_apps:
            delete_migrations(app)

Step 3

1. Delete the migrations from the database (you can use the command below, it should work universally for any setup that uses Postgres but you will have to update the connection string as needed)

Run calling python manage.py delete_migrations_from_db (if you name it that way)

import os

import psycopg2
from cacheops import invalidate_all
from django.conf import settings
from django.core.management import call_command
from django.core.management.base import BaseCommand
from django.db import connections


class Command(BaseCommand):
    help = 'Migrate on every database in settings.py'

    def handle(self, *args, **options):
        db_list = settings.DATABASES
        # del db_list['default']

        for db, _ in db_list.items():
            if hasattr(settings, 'CACHE') and settings.CACHE is True:
                invalidate_all()

            # we have the db name, now lets remove the migration tables in each

            try:
                host = os.environ['_HOST']
                user = os.environ['_USER']
                port = os.environ['_PORT']
                password = os.environ['_PASSWORD']

                conn_str = f"host={host} port={port} user={user} password={password}"

                conn = psycopg2.connect(conn_str)
                conn.autocommit = True
                with connections[db].cursor() as cursor:
                    delete_statement = 'DELETE from public.django_migrations'
                    cursor.execute(delete_statement)
                    print(f'Migration table cleared: {db}')
            except psycopg2.Error as ex:
                raise SystemExit(f'Error: {ex}')

        print('Done!')

Step 4 Call python manage.py makemigrations to reinitialize the initial migration files

Step 5 Call python manage.py migrate --database=[YourDB] --fake to reinitialize the initial migration files. The --fake arg allows the DB structure to not be changed while the history is restored in the Database (if you want an easy command for running this migration command in all DBs, you can use something like the code below)

Called using python manage.py migrate_all --fake (depending on naming)

from cacheops import invalidate_all
from django.conf import settings
from django.core.management import call_command
from django.core.management.base import BaseCommand


class Command(BaseCommand):
    help = 'Migrate on every database in settings.py'


    def add_arguments(self, parser):
        parser.add_argument(
            '--fake',
            action='store_true',
            help='fake migrations',
        )

    def handle(self, *args, **options):
        db_list = settings.DATABASES


        for db, _ in db_list.items():
            if hasattr(settings, 'CACHE') and settings.CACHE is True:
                invalidate_all()
            self.stdout.write('Migrating database {}'.format(db))

            if options['fake']:
                call_command('migrate', '--fake', database=db)
            else:
                # no fake, call regularly
                call_command('migrate', database=db)


        self.stdout.write('Done!')