database - How do I Store JSON data from websockets?

Question

I’m looking at storing raw JSON data received from a bunch of websockets and am feeling a bit overwhelmed with all the choices available. I found this from 2012 but things have moved on a bit since.

My requirements are to be able to query the data through an API, to get the latest message (ideally in realtime) or a subset of messages (like all messages received on a particular date).

I'll be storing around 10,000 messages daily from different sources with different schemas.

From some basic research I think I need some kind of nosql document store? The main ones seem to be:

• Elastic search
• Redis
• MongoDB
• CouchDB

Am I on the right lines here? What DB service should I use to store JSON data?

Thanks.

question from:https://stackoverflow.com/questions/65934877/how-do-i-store-json-data-from-websockets

Answer 1

I ended up streaming data from Kafka to both Postgres (for persistent storage) and Elasticsearch (you know, for search) if that helps anyone.