Do you sew over your pins? I use IBC silk pins, sew (slowly) right over them, and that helps keep the seams together. Otherwise you're spending all that time pinning and then they escape!
And when you say the seams are going the right way, you mean they're nesting right (one goes one way, the other goes the other way)? I've noticed that depending if the top block has a seam pressed towards me or away from me, even when they nest, it will result in less accurate seams. It has to do with the feed dogs pulling forward and the top foot pushing back, so ... depending how the seams are, it can be pulling them apart or pushing them right together. But I've had some that were nested and should've been fine, but pulled apart a little. But lots of pins helped that.