We had a similar leak in our house and it got into my fabric. I didn't see it for a couple of days. Some of the cabinet parts looked dry until I took everything out and saw that indeed there were wet spots that mold was actually starting to form on. Ugh. Water damage was also under the cabinet and behind. I washed everything and treated the cabinets to blown heat for several days. (A bathroom heater) (Advice from my insurance carrier)
I know it is a big job, but I would take everything out just to be safe. Sorry this happened to you!