python 3.x - How to reference a process reliably (using a tag or something similar)?

Question

I have multiple processes (web-scrapers) running in the background (one scraper for each website). The processes are python scripts that were spawned/forked a few weeks ago. I would like to control (they listen on sockets to enable IPC) them from one central place (kinda like a dispatcher/manager python script), while the processes (scrapers) remain individual unrelated processes.

I thought about using the PID to reference each process, but that would require storing the PID whenever I (re)launch one of the scrapers because there is no semantic relation between a number and my use case. I just want to supply some text-tag along with the process when I launch it, so that I can reference it later on.

question from:https://stackoverflow.com/questions/65829822/how-to-reference-a-process-reliably-using-a-tag-or-something-similar

Answer 1

pgrep -f searches all processes by their name and calling pattern (including arguments).

E.g. if you spawned a process as python myscraper --scrapernametag=uniqueid01 then you can run:

TAG=uniqueid01; pgrep -f "scrapernametag=$TAG"

to discover the PID of a process later down the line.