I am hoping the measurements you gave were just a "for example" because I have never had one that had a variance of more than an inch and most of the time much less than that.
I agree, it is very hard to ease in 4" and if my variance was that large, I would probably try to figure out why and correct the error as that much excess will cause serious problems trying to keep flat while you are quilting on a LA and quite honestly you would end up having to take some pleats to get it to work. But it is equally difficult to try and stretch an area that much and would more than likely cause popped seams as well as skipped stitches because too much tension was on the quilt.
So, assuming your sizes were simply an exaggeration, the reason you take the smallest measurement and go with that is because quilting takes up excess fabric. So if you have an area that is too big, it is much better to try and work that excess in during quilting, as opposed to trying to stretch the short areas out to match the largest measurement. Also the reasons I state above, even if as small as 1", when you put too much tension on a quilt top loaded on the rack you run the risk of popped seams in the piecing or bad tension or skipped stitches due to too much tension on the top.