python - How do I use re.sub to remove a repeated block of text within square brackets?

Question

I have a response from an API that is a pseudo dictionary with some 'key':'values' but mostly just a blob of text with 'key:values'. I convert it with .json() to this:

{'status': 'done', 'nextLogId': 'AQAAAXb', 'logs': [{'content': {'service': 't2pipeline', 'tags': ['tag1:value1', 'tag2:value2', 'tag3:value3'], 'timestamp': '2021-01-05T05:25:03.416Z', 'host': 'i-00e17', 'attributes': {'caller': 'psignal/state_machine.go:451', 'ts': 1609824303.416246, 'level': 'warn'}, 'message': 'psignal: Ignoring scte35 segmentation_descriptor (type:Program Start eventID:0 refUTC:Jan 5 05:25:02.387626333): there is an active segment with the same event_id'}, 'id': 'AQAAAXb'}, {'content': {'service': 't2pipeline', 'tags': ['tag1:value1', 'tag2:value2', 'tag3:value3'], 'timestamp': '2021-01-05T05:25:03.416Z', 'host': 'i-00e17', 'attributes': {'caller': 'psignal/state_machine.go:713', 't2': {'scte35': {'event_id': 0, 'event_ptr': '0xc009f32b40', 'seg_type_id': 16}}, 'ts': 1609824303.4161847, 'level': 'info'}, 'message': 'psignal: scte35 segdesc eventID:0 type:Program Start'}, 'id': 'AQAAAXb'}], 'requestId': 'OVZRd3hv'}

There are two entries in here and in reality there will be more.

I convert to a string with json.dumps()

And then use re.sub() to remove the 'tags': [], section from the response and return the string like so

res = re.sub(r'"tags": [.*"],s', "", response_string)

The problem is it only return the last entry.

print(res)

{"status": "done", "nextLogId": "AQAAAXb", "logs": [{"content": {"service": "t2pipeline", "timestamp": "2021-01-05T05:25:03.416Z", "host": "i-00e17b8e872ec7d05", "attributes": {"caller": "psignal/state_machine.go:713", "t2": {"scte35": {"event_id": 0, "event_ptr": "0xc009f32b40", "seg_type_id": 16}}, "ts": 1609824303.4161847, "level": "info"}, "message": "psignal: scte35 segdesc eventID:0 type:Program Start"}, "id": "AQAAAXb"}], "requestId": "OVZRd3hv"}

How do I modify the regex so that every instance of 'tags': [], is removed and returns the whole string with all entries?

Note: Since I can't del by key I think the only way to remove content is treating the response like a string and remove tag with regex.

Answer 1

No need using regex. Use

import json
res = {'status': 'done', 'nextLogId': 'AQAAAXb', 'logs': [{'content': {'service': 't2pipeline', 'tags': ['tag1:value1', 'tag2:value2', 'tag3:value3'], 'timestamp': '2021-01-05T05:25:03.416Z', 'host': 'i-00e17', 'attributes': {'caller': 'psignal/state_machine.go:451', 'ts': 1609824303.416246, 'level': 'warn'}, 'message': 'psignal: Ignoring scte35 segmentation_descriptor (type:Program Start eventID:0 refUTC:Jan  5 05:25:02.387626333): there is an active segment with the same event_id'}, 'id': 'AQAAAXb'}, {'content': {'service': 't2pipeline', 'tags': ['tag1:value1', 'tag2:value2', 'tag3:value3'], 'timestamp': '2021-01-05T05:25:03.416Z', 'host': 'i-00e17', 'attributes': {'caller': 'psignal/state_machine.go:713', 't2': {'scte35': {'event_id': 0, 'event_ptr': '0xc009f32b40', 'seg_type_id': 16}}, 'ts': 1609824303.4161847, 'level': 'info'}, 'message': 'psignal: scte35 segdesc eventID:0 type:Program Start'}, 'id': 'AQAAAXb'}], 'requestId': 'OVZRd3hv'}
for i in range(len(res['logs'])):
    del res['logs'][i]['content']['tags']
print(res)

See Python proof

Results:

{'status': 'done', 'nextLogId': 'AQAAAXb', 'logs': [{'content': {'service': 't2pipeline', 'timestamp': '2021-01-05T05:25:03.416Z', 'host': 'i-00e17', 'attributes': {'caller': 'psignal/state_machine.go:451', 'ts': 1609824303.416246, 'level': 'warn'}, 'message': 'psignal: Ignoring scte35 segmentation_descriptor (type:Program Start eventID:0 refUTC:Jan  5 05:25:02.387626333): there is an active segment with the same event_id'}, 'id': 'AQAAAXb'}, {'content': {'service': 't2pipeline', 'timestamp': '2021-01-05T05:25:03.416Z', 'host': 'i-00e17', 'attributes': {'caller': 'psignal/state_machine.go:713', 't2': {'scte35': {'event_id': 0, 'event_ptr': '0xc009f32b40', 'seg_type_id': 16}}, 'ts': 1609824303.4161847, 'level': 'info'}, 'message': 'psignal: scte35 segdesc eventID:0 type:Program Start'}, 'id': 'AQAAAXb'}], 'requestId': 'OVZRd3hv'}