I’ve been experimenting with using Python to generate text-based data for an experimental spin-off app from our team at work, and for my first “real world” use of Python, I’m pretty impressed with how efficient it is for doing this.

I’ve got a simple script iterating over a collection of strings to produce all possible combinations of those strings. The output of that script is being fed into a text file via Bash. So far it’s generating ~52GB of data in roughly 15 minutes, and it’s only part-way through the possible combinations. I’ve had to kill my test run because otherwise I’m going to run out of disk space on my laptop SSD! CPU usage was a moderate 26%, and RAM usage was tiny, at only ~2.8MB. Previous attempts at this using other languages tended to saturate one or both of these resources in fairly short order.

It’s fun to try out a new (to me) tool every now and then!