I used 5-1/2" because I was using a panel of squares for some of the blocks. It came out great that size. I have another cut out with 5" squares but haven't started assembling that one yet.
I would think the only concern would be using too small of a square because you'd lose a bit of it once it's cut. I don't think I'd go much smaller than 4-1/2".