Sorting directory by date Python

Question

I have the following directory format:

logs
---- abc_090000_20210124
---- abc_091500_20210124
etc...

The format is abc_(HHMMSS)_(YYYYMMDD)

I want to get the two most recent files by date and time.

If they are all on the same day I use this code:

directory = os.listdir("logs")
base = "abc"

date = sorted([x.split("_")[2] for x in directory])[-1]

t_of_day = sorted([x.split("_")[1] for x in directory if x.split("_")[2] == date])

a = f"logs/{base}_{t_of_day[-1]}_{date}"
b = f"logs/{base}_{t_of_day[-2]}_{date}"
print(a,b)

and it works properly, however, at the change of day the files will change like so:

...
---- abc_234506_20210124
---- abc_000007_20210125
...

and my code will give an error as there will not be two values in the list t_of_day.

I am unsure of how I could change this to suit my situation.

I have tried to use a condition where if date[-1] == date[-2] to use a different value, but as the dataset builds up this may become unreliable.

Is there perhaps an alternative way to sort the files? I must state that changing the file names is not an option.

question from:https://stackoverflow.com/questions/65877082/sorting-directory-by-date-python

Answer 1

You can supply your own function as the key argument to determine how a list is sorted.

import os

def date_sort(filename):
    abc, hms, ymd = filename.split('_')
    return ymd + hms

filenames = os.listdir('logs')
filenames.sort(key=date_sort)

You could also use Python's datetime class, but I don't think it's necessary for simply sorting values like this. All you need are the units of time in descending size: year, month, day, hour, minute, second.

For just getting the two most recent, you don't really need to sort the whole list, but you can still use this same key function for "sorting" a heap queue. (Here's another answer that gives a good example using heapq.)

import os
import heapq

def date_sort(filename):
    abc, hms, ymd = filename.split('_')
    return ymd + hms

filenames = os.listdir('logs')
first, second = heapq.nlargest(2, filenames, key=date_sort)

If you decide you would like to use datetime, you should get the same results by altering your sort function to return a datetime object using datetime.strptime:

from datetime import datetime

def date_sort(filename):
    abc, hms, ymd = filename.split('_')
    return datetime.strptime(hms + ymd, ''%H%M%S%y%m%d')