Panels with borders are the pits! I usually cut the borders off and then add new borders to kind of straighten the panel. Since you have already quilted it, you can still trim it and attach borders kind of quilt as you go method.
Cut the short border pieces for the top and bottom and longer border pieces for the 2 sides. Cut corresponding back border pieces a little bigger than the fronts. Cut batt sections to match the back border pieces. Lay the front border right side down on the quilt top, pin the back border (laying on the quilt back) with the batt underneath it and sew the top seam. Do the bottom in the same way. Press the fabric outwards on the top and bottom borders. You can then attach the side borders in the same method. Once all the borders are attached, you can quilt the border sections and add binding.
Another option is to set the whole panel at a wonky angle to disguise the crookedness.