There is hope! The bond remains and is just as special as he grows up. I had my DGS Jake from the time he was 8 mos until he started preschool. I had him 10 hr days 6 days a week. It was like having another child of my own. When mu DIL told me she was taking him to her cousin 3 days aweek so he could go to preschool it broke my heart! I still had him two days a week for another year and then only on weekends. Jake is 12 now and he is just as strongly bonded to me as he was as a little boy.
You have laid the basis for a really strong unbreakable bond. You might think about asking if you could have them for a whole day now and then. I am sure his mom would love to have a free day to do things she needs or wants to do.