At 17:37 11/5/2015 -0500, you wrote:
. . .Consensus allocation worker. .
The consensus list manger could run as an independent Python process and "message" changes to the scanner processes to avoid complexities of trying to share data (I know very little about Python and whether sharing data is difficult or not).
The consensus manager would keep a snapshot of the list and send delta transactions to the scanners via IPC, by updating complete disk- file lists, or by creating disk-file deltas for the scanners to consume. Whatever is easiest and most appropriate.
Perhaps each scanner would resync somehow when starting a new pass as insurance against list-delta transaction lossage.