Yes, you did the right thing - the way that you can tell is that you're the one who's suffering, now - not your dear pup. I'm so sorry.
I lost both of my old dogs in the last year and it's gut wrenching, at first. But when they get so sick, we have to do what's best for them, since they can't speak for themselves. It's the hardest part of being a dog's "mom," isn't it?
I hope that this gets easier for you every day and pretty soon, you'll have only good memories and smiles when you think about him.