Obviously retreats can vary widely as to the who, how many, where, and what. Many quilting groups and guilds host retreats as do many LQS's. Quilting magazines even have sections with ads on various retreats and quilting cruises and quilting camps.
If you want to go with folks you know, try to go where you can meet them: quilt show, LQS's, and quilting groups/guilds. Or make it up for just a few of your quilting friends. It can be a destination retreat, where you and friends go to enjoy the surrounding area as well as quilt.
The variety is endless.
If you have a big enough house you could host one if you feel like it. I would caution you to start the first one with a smaller group unless you are very organized. The main item to remember is you would need enough electrical capacity to host all the sewing machines, lights, and irons quilters use. Many places tell retreaters what to bring in the way of irons and lights, so as not to exceed the capacity of the facility.
I hope you post after you go on your first retreat.
ali