CSV Data Randomizer

Overview

Takes a confidential spreadsheet and renders it useless for disclosure while keeping it useful for testing: sample a hundred columns and a hundred rows at random, discard the rest, then overwrite the values that remain.

Handles both flat files and multi-index ones, where the sampling has to be applied per top-level column group rather than across the sheet.

Complexities tackled

Multi-index columns are the actual requirement. A flat CSV is a straightforward sample. A file with grouped columns needs the cap applied within each group independently, and the client explicitly noted there could be more groups than the two in the example, so the number of groups had to be discovered rather than assumed.

Doing nothing is a valid outcome. Where a group already holds fewer than the cap, the correct behaviour is to leave it alone rather than to sample it down or pad it. Small explicit rules like that are what stop a tool corrupting the edge cases it was supposed to handle.

Anonymised is not the same as encrypted. This deliberately destroys information rather than protecting it. The output cannot be recovered, which is the point: it is safe to hand to someone who should never see the original.

Stack

Python with Pandas and NumPy for the sampling and overwriting.