Mostly it's just that the Doctor doesn't trust any carnivals. Not since he encountered 'The Greatest Show in the Galaxy'. That was an adventure...

"Well, if you were on the planet, Gabratas Three, you could purchase an entire city. But fine, what kind of currency is valid here? I do a lot of travelling, you see, get them all a bit mucked up."