I made a similar quilt, with lots of HST & 4 patch units. I trimmed the smaller units first. If you trim a 4 patch, you may need to cut just a sliver, which will not be noticed. If you wait until you join into the 16 patch, any imperfections will be magnified and you may need to trim a larger piece which will be noticed.