Back before rotary cutters and all these fancy rulers, I made a log cabin quilt. All the strips were cut the width of fabric and I cut the strips to length after I sewed them. After I pieced the top it had grown from a full size to queen size. All those tiny little mistakes added up. I was able to use the quilt but if it had gone the other way, it would have been too small.
It is your quilt and you can do what you want. I know it takes time to trim but it may save you time later when trying to piece things together.