Lots of good advice has been posted here.
Having spent 25 years on the (very local) craft show circuit I would add a few points:
Do your homework. Go to a few shows as a shopper. Look to see if anybody else is selling items similar to what you would like to do, and if so, note their prices. If their items are VERY competitive, then ask them what other shows they will be at. You probably won't want to do those particular shows.
Actually, you will want to ask a bunch of crafters what shows they do and which ones they think are best. If possible, ask fabric crafters if you're going to be doing fabric crafts. Ask them why they like those particular shows. If they are outdoor shows, ask them if shoppers still turn out in case of rain. Some shows have a very dedicated customer base, others do not.
Never - did I say NEVER? - do a show that's a "first annual" show. No track record, no customer base, and who knows if the organizers know anything at all about getting the word out?
Also never, EVER do a show that is combined with a flea market. Shoppers will be cheap and will want to haggle.
Also find out if you will need a tax id. I've seen tax inspectors come around to the larger shows here, and you can get fined if you do not have your tax number on display.
That said... once you go the tax number route, all your sales will be "on the books" and you cannot remain "invisible".
Personally I'm not a huge fan of the idea of selling on eBay / etsy. If I were starting out, small-time, I would rather just do a few shows. It would be easy for you to spend just as much time doing photos and descriptions for etsy/eBay as you would sitting at the show. You can choose to only accept cash, if you prefer, and you don't have to worry about shipping, or major record-keeping in case of a tax audit. If you sell interstate, you will have to keep records proving where you shipped your product, so that you won't be liable for sales tax on all those sales. At least, that is the way it works in NYS.
Oh - and if you decide to do shows and accept checks - always get a phone number, and wait 2-3 days before cashing it. People tend to overspend at craft shows and they can overdraw their account without even noticing. That gives them a couple of business days to put more money into their account and you won't get hit with a fee for a returned check. (Yes, the banks charge both the issuer of a check and the party receiving it.)