Homemade candy - fudge/peanut brittle/divinity etc or homemade cookies you can put them in a tin.
For the 2 yr old you can make a cloth book and put differant sensory things on each page. Glue on or sew on a strip of sandpaper for the rough texture- sew on a square of fake fur for soft etc. This would also work for the baby as he will be growing and will soon be exploring his world.
Or sew a zipper onto one page - a large button and a flap with a button hole to learn to button. Put a snap on one page with the same idea of putting the other half of the snap on a flap. You can get large snaps online or at Joann's. You can sew on shapes - circles squares rectangles etc on one page and sew on velcro. Then cut out a square a circle etc and add the other velcro for a match game.
Another thing I did for day care was to take a poster board and found shoes with differant treads on the bottom. I made an imprint of the shoe tread on the poster board of several differant ones and then the kids had to match the right shoe or boot with the imprint of the tread. You might not want to do shoes but you could find some distinctive toys or items that would work the same way. You could make the poster board more durable by using cloth(make it into a small quilt maybe?) instead and painting the imprints on that.
You can make a cloth book and sew a square of velcro on each page. Then cut out letters or pictures of food or animals and put the other half of the velcro on that. You can glue the pictures onto card stock and laminate them for durability. You can buy laminate plastic at walmart.
For the 4 yr old - you can make a cloth game board for checkers and make checkers out of milk lids. You can sew velcro in the squares and glue the other half onto the milk lids. make sure you have two differant colors of milk lids.
The velcro will give him a way to store the checkers with the board. Or you could make a small bag to store them in.
If you have access to pictures of the family make a matching game by using their pictures. Make 2 identical picture cards of each family member including any pets. If not you could use pictures of toys pictures of food apples oranges etc. You could use pictures of babies as little ones love to look at pictures of babies or puppies etc.=== to make the cards use cardstock to print your pictures onto or glue them onto and then laminate.