python - Workaround for datetime timestamp() on Windows for dates preceeding 1970-01-01

Question

I am currently trying to generate numerical features in a dataset by converting the dates to timestamps. If run on Mac, it works flawlessly, on Windows it throws an:

OS Error: [Errno 22] Invalid argument

which is probably due to Windows not supporting unix timestamps from before 1970-01-01. I have dates from 1955 upwards. Here is my code:

import time
import datetime

current_timestamp = time.time()
df.loc[:, "FEATURE_num"] = df["FEATURE"].apply(lambda d: datetime.datetime.strptime(d, '%Y-%m-%d').timestamp() if isinstance(d, str) else current_timestamp)

I somewhere saw suggested to maybe use datetime.timedelta(), but I couldn't figure out how to integrate it.

Answer 1

You could do it by (implicitly) using datetime.timedelta to calculate a "Gregorian" timestamp that would be valid for dates from 1582-Oct-15 to the present (or some other "epoch" you would like to use).

As the function's docstring indicates, date strings will, by default, be parsed using a '%Y-%m-%d' strptime-like format string parameter, but that can be overridden.

from datetime import datetime


GREGORIAN_EPOCH = datetime.strptime('1582-10-15', '%Y-%m-%d')


def gregorian_timestamp(date, format='%Y-%m-%d'):
    """ Calculate timestamp using start of Gregorian calender as epoch.

        The date parameter can be either a string or a datetime.datetime
        object. Strings will be parsed using the '%Y-%m-%d' format by default
        unless a different one is specfied via the optional format parameter.
    """
    try:
        date = datetime.strptime(date, format)
    except TypeError:
        pass
    return (date - GREGORIAN_EPOCH).total_seconds()  # The timedelta in seconds.


if __name__ == '__main__':

    current_date = datetime.now()
    timestamp = gregorian_timestamp(current_date)
    print('gregorian timestamp:', timestamp)  # -> gregorian timestamp: 13768250461.136208

    timestamp = gregorian_timestamp('1970-01-01')
    print('gregorian timestamp:', timestamp)  # -> gregorian timestamp: 12219292800.0

    timestamp = gregorian_timestamp('1955-02-28')
    print('gregorian timestamp:', timestamp)  # -> gregorian timestamp: 11750918400.0

    timestamp = gregorian_timestamp('1582-10-15')
    print('gregorian timestamp:', timestamp)  # -> gregorian timestamp: 0.0