I suck at pot holders... so hang in there. In my experience, small projects can be more time consuming than larger projects. The first one always takes the longest.
Try something else.
Also this is the perfect time to start a sewing or quilting journal. Take a photo of every project and save it by the date. You will enjoy looking back at it someday. Even the 4 hour pot holders will be interesting, looking back.