No you dont have to use the hoop.
If you try with out it make sure you have basted really really well.
I prefer to hand quilt without the hoop.
Also what may help is to make sure you are not putting the quilt in taunt like with needlework. You want it somewhat lose.
As far as stitches. Well I dont really know I think my mom taught me something like 10-12 per inch but I just do whatever I can get loaded on the needle. Dont know that I have ever really counted.
If you can post a pic of your stitches I am sure someone here will be able to tell you if they are too big.
Just remember your stitches will get better with time