Testing randomness is a pain, so don’t do it

TL;DR : stub out your random source to avoid sporadic failures

I wrote a spec that ensured that a given input was randomized.  It passed the vast majority of the time, but failed occasionally.  This bothered me greatly after a day of having it randomly pop up, so I spent some time trying to fix it.

I tried increasing the amount of input, thinking that the odds that a combination of 25 items would be the same are very slim; it would still fail sporadically, but not nearly so often.  I would get irritated less.

Of course, increasing the amount of input made no difference (other than maybe making the spec slower.)

Solution: stub out the random source to always return a given output.  Now I can get the test to fail or pass consistently, as I like.

And yes, this sort of a “well, duh.”

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s