I've seen them done with batting cut to fit under the 'window'; but, generally.. depending on the weave of your foundation cloth it'll be pretty warm, and heavy by the time you're finished.
for a 9 1/2 window, you would start with a 16 1/2 inch square, I would think....