If it only happens on the flannel then that is probably the problem. I am thinking the flannel is just a bit thicker and it is holding up the needle just long enough that the thread misses the stitch. You could try ironing before stitching to compress the sandwich a bit. Caution: if you are using polyester batt, a hot iron will melt it.