I personally do remove the stay stitching because I hand baste & worry about whether the little knots might be visible but otherwise the only time you really need to do it is if your thread color is visible through the binding.
I'm sure it's overkill because I leave the knots I bury from quilting in there & they're not an issue. There was once when I caught my stay stitching & accidentally pulled it up through the binding fabric when machine stitching that part of my binding, but that's been the only issue I've ever had from leaving it in.
Last edited by Bree123; 03-24-2016 at 09:48 AM.