Did you 'ease' the border around the circle? I haven't 'eased' any curves into anything in 35 years - but know from 'old' experience that curves need to be eased or they don't lay right. Way back when, I was taught to ease a curve by clipping small cuts into the seam allowance of the two fabrics that were joined, making sure not to clip into the actual seam. I don't know if this technique is used anymore.
Found this technique at:
http://caseysclothingclass.blogspot....ng-fabric.html
How to Ease Stitch
To ease in fullness, use one or two rows of ease stitching. Use the same technique as for gathering.
1. Stitch close to the seam line with long machine stitches extending stitching slightly beyond markings, leaving long thread tails.
2. Stitch a second row 1/4" away in the seam allowance for set-in sleeves.
3. Pin fabric with right sides together and eased side up.
4. Pull up thread between markings and distribute fullness evenly.
5. Stitch with standard stitching along seam line, leaving a smooth seam.