It might work out okay with the baby. Perhaps even better than it would with an older child. Babies often fall asleep when they get sensory overload. It is just their way of slowing down everything their minds have to process.
I hope you are having a lovely time!