Right off the top, fingers that do not ply the needle regularly will be sore after hours of stitching without a thimble. I really do urge you to try some of the new ones, which are soft, comfortable and easy to wear. I cannot bear the metal thimbles and prefer a leather one myself. You can also get little dots to stick on your needle finger.
But apart from that, perhaps you are making your stitches too close together? How many stitches per inch did you do? Because binding can be around 4-6 stitches per inch.