I love pinwheels, and other HST blocks. Even that small discrepancy would also drive me crazy. Try this.
When you are pinning to combine your subunits, make sure the seam allowance that is on top 'faces' the machine needle. Another way to say it is that the needle comes to the seam allowance before the previous seams.
If you want to have the seam allowance the other way because you will end up with dark under light, clip that previous seam allowance close to the new seam.
Doing it this way is no harder, but it looks much more impressive when the points meet exactly.
If this is not a clear statement, let me know and I'll post pix. I figured this one out for myself then saw it on Simply Quilts, back in the day. The reason is that once the seam allowance is locked in place with a stitch, the pressure foot can't push the upper seam allowance away from the lower one.