My first quilt was a twin - my second a double. I agree with starting big - if it is a bed size quilt, and you have it on your bed or where you can see it, you really feel like you've done something. Yes, you may pick out flaws, but nobody besides you will, they will just see a nice quilt!
As a lady in my quilting group said - if the person is looking at your points/blocks and saying bad things about it, it's their problem - not yours!!