The main thing is to try out lots of different chairs till you find one that is really comfortable. Sit in the ones that are "maybe's" for longer periods than just a couple of minutes to be sure that they stay comfortable.
I got lucky and picked up mine at a yard sale for $10! It has served me well for 6 or 8 years now and seems like it should last for at least that much longer.
If you can find one you like that is the Boss brand mentioned earlier, they have a good reputation as far as the less expensive office furniture is concerned. Alera is another one with a good reputation.