I am writing a script that streams from a finance API. The script streams in data points as instances of 'bytes' class, and later (not shown) outputs them to a pandas dataframe. The stream works just fine for a while, and with shorter runtimes. During longer runtimes, at some point into the stream (exactly when varies, but usually an hr or two), I get this error:
valueError invalid literal for int with base 16 b''
The full error message is included below:
error_full_traceback
A 'normal' data point is shown here:
normal 'bytes' data point
The request code connection I am using is very simple, here:
def get_stream(url=STREAM_API, token=ACCESS_TOKEN, path=STREAM_PATH, instruments=INSTRUMENTS, headers=headers):
'''takes in neccesary parameters and outputs the input stream connection.'''
response = requests.get("https://"+url+path+instruments, headers=headers, stream=True)
return response
It seems like the base problem is that the request itself, during streaming, is encountering a b''{datapoint} item instead of the intended b'{datapoint} item. How can I fix this error? Am I correct in my understanding of the error?
与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…