The Elgin Sewing Machine & Bicycle Co. and Seuberth-Leach Manufacturing were based along Dickie Avenue, Elgin, IL according to an old article in the Chicago Tribune. Of course, that may be just one factory. They made Safety bikes starting at about $40. Twice the price of many sewing machines then. LIke sewing machines, the name came back to life later after the war and Sears used the name for a while. Elgin watches came from the same folk later on, too.
I would love to have a pre 1900 Elgin bike and sewing machine to display together. I am much more likely to find a VSM though. They didn't have wooden wheels.