i would think you would be able to pick one up pretty cheap off ebay the are the kind of thing you would find on there,
Reins and stirrups and stuff are a little trickier to find as even the pony stuff is often quite long so either make sure you get measurements for anything you see online or go and have a look in tack shop,
I got a cub saddle for £20 from ebay, still with the tags on! Likewise bridles you can pick up from £5 upwards, I recently sold one for £10 on there. Its really not worth buying a new one for the amount of use they get. My daughter is almost 3 and already too big for the cub saddle, she's moved on to a GP synthetic.