I love that idea. I think the parents will know where to find it when they need it.
Another idea would be to make it two sided. Have another scene that can be shown the rest of the year on the back, and hang the bag of numbers behind the wall hanging.