compression - How to uncompress a message with kdb (32 bit)?

Question

I'm trying to decompress a message optimized by Deflate compression in kdb i.e. the following message:

0xab562a494cca4955b2522a2e4f2cd0cf4d2cca8e2f28ca4c4e55d2514a492c4954b28aae56cacc2b2e292acd4dcd2b89cf4c012a750a71d60d0d76d10d0e770c00aa43d264a56462606a68ac6709142ec9cc4d2d2e49cc2d00ea30323032d4350022cb1043732b230b2b23333d2323a328a5dad85a00

How should one implement this in kdb? Thanks

question from:https://stackoverflow.com/questions/65645652/how-to-uncompress-a-message-with-kdb-32-bit

Answer 1

I think the simplest option here would be to use the zlib Python library via embedPy.

The byte string is passed to the decompress function with the Deflate settings found here. In Python this returns a bytes object, so we can decode this using UTF-8 into a Python string, with the final backtick (`) specifying that we want this to be returned as a q type (also a string).

We can then convert this from a JSON string into a KDB object using .j.k

q)l p.q
q)zlib:.p.import`zlib
q)i:0xab562a494cca4955b2522a2e4f2cd0cf4d2cca8e2f28ca4c4e55d2514a492c4954b28aae56cacc2b2e292acd4dcd2b89cf4c012a750a71d60d0d76d10d0e770c00aa43d264a56462606a68ac6709142ec9cc4d2d2e49cc2d00ea30323032d4350022cb1043732b230b2b23333d2323a328a5dad85a00
q).j.k zlib[`:decompress][i;neg zlib[`:MAX_WBITS]`][`:decode;"utf-8"]`
table| "swap/mark_price"
data | +`instrument_id`mark_price`timestamp!(,"BTC-USD-SWAP";,"40513.9";,"2021-01-09T17:28:26.222Z")