Sounds like you have tried everything but beating it with a stick to soften it up!
Some fabrics do stay stiff. From here, it seems like your options are:
Use it because the color is perfect and the texture is stiff.
Find something else that may not be "perfect" in color but feels better.
If it's for a baby, I think I would choose something that feels better over something that looks nicer.