My guess is you are making a common beginner mistake of "making sure" you have a 1/4" seam -- which translates to making it a little bigger rather than scant. This, in combination with regular thread and perhaps imprecise pressing, would explain why your blocks are coming out too small. It can be difficult to accept that a scan 1/4" is better than a generous 1/4", but it's true. Many of us who were taught to use 5/8" seams for clothing construction find it difficult to believe that a scant 1/4" is enough to hold a seam together, but for quilting it really is.
It's a good idea to measure and cut 1-1/2" strips to practice on. Sew 3 of these together (about 6 inches in length is enough), press, and then measure to see if they are an exact 4" wide. Adjust your seam allowance until these practice strips measure exactly as they should. I position a strip of thick moleskin on my sewing machine at that point to ensure that all of my seams are sewn to that measurement.