I am a bookkeeper/staff accountant, I used to work for a CPA firm here in AZ.
As a sole proprietor, you can file your business taxes on your 1040 personal return. You need to prepare Schedule C. I highly recommend buying Quickbooks, and do *NOT* go with Quickbooks Online. That always gave us so many problems when trying to get reports for the firm.
I suggest Quickbooks Pro, unless you are hiring employees you do not need the Payroll add-on. If you do ever need it, you can always add it later. I use Quickbooks for my own business as well as for clients (I am a freelance bookkeeper out of my home). Your CPA can help you with any necessary adjustments, and when you get to the point where you have enough business income that you need to take a salary, s/he will tell you how to do that as well (you still won't need the Payroll service, either).
I have never done sales tax for New York, but Quickbooks lets you set up tax rates for in and out of state sales, as well as lets you set up rates for the different regions/cities, as I know NY has that fun going on, just like AZ does. There are reports you can print out that will give you the numbers you need to file your returns. Which reminds me, I need to do mine.
If you have other questions, feel free to send me a PM and I'll help out as best I can.