Re-frame it in this way -
We all have experiences that don't involve the others we love. Even just going to work isn't something we share. But the sum of our experiences - those we share with the ones we love and those that are just ours - make us who were are.
If your husband hadn't gone through that 5 year period away from you and with this boy's mother, perhaps he wouldn't be someone you want to be with again now. That helped shape him to be someone you wanted to love again.
Instead of feeling threatened by that time, seek to feel grateful that it caused him to return to you.