Yes, it will vibrate - that vibrating shuttle gives it a real shimmy.
But, I think you'll love sewing with it once you get used to it.
One thing you might try is wrapping some electrical tape around the clip that holds your belt together. That will silence some of the belt noise or maybe all of it.
Better and more permanent, I usually put about a 1½" piece of heat-shrinkable insulation tube over the belt before putting in the clip. When the clip is in and bent properly, I slide the tube of insulation over the clip area and hold a hair dryer on it or a lighter under it, to shrink the insulation. It really quiets that clunk-clunk with every revolution.
The treadle wheel also needs periodic oiling or greasing and the pitman may need to be adjusted if there is a thunking noise when the treadle is depressed or raised.
How wonderful a Christmas present!