I don't have a die cutter, but having used one I would say that yes, I really love it, and I will buy one when I'm closer to retirement (and will have more time for quilting).
My guild had one when I lived in a city, and I was able to cut out an entire apple core quilt in about half an hour. I would never have made that quilt if I had to cut the pieces with a template. That is my primary reason for wanting to buy one; it makes non-square cutting much easier.
I did use the HST die as well, and found that my blocks turned out so accurate that I didn't have to trim down. Maybe I would like HST's more if I had a die cutter... right now they're my least favorite part of quilting, and I haven't made one since I moved away from the Go.
There are certainly downsides; you're limited to the available die sizes, there is some waste (although not much when you get good at folding/preparing your fabric), the dies are expensive relative to rotary cutter blades, and it requires storage space. For me, these are far outweighed by the benefits of speed and accuracy that it provides.
I would definitely recommend the Go over the Go Baby; the Baby doesn't have the variety of dies available that I would want. I've only used the Go Baby at a store demo, but I also found it harder to crank than the Go.