Originally quilting was done by hand piecing and hand quilting old clothes. So, old clothes, needles, thread, scissors, maybe a sheet for the back or some more old clothes, and some sort of batting... (if any).... $30?
I have purchased some wonderful vintage Singer machines for $20- $25 each, some even came in cabinets. It doesn't have to be expensive.
That being said, I think most people come to quilting with some sort of sewing knowledge/background. So they would already have a machine and supplies to get started.