You aren't crazy, I'm hand-piecing a Farmer's Wife! I don't cut the fabric down to finished size + seam allowance- I draw the finished size of the shape I want on the back of the fabric, then cut a rough, somewhat generous seam allowance outside of the dran line. I then sew along that drawn line, and after I'm done then I trim it down to 1/4".
(you can see some of my FW blocks on my flickr page- the outer seam allowances are not trimmed down yet- those will be trimmed when I add sashing and join the square:
http://www.flickr.com/photos/hemlock...7629270049548/)