Is there something about your father such as a hobby, favorite color, favorite season, favorite flower, anything that was special about him that you kids and your mom would recognize if you somehow put it in one or more of the squares? A special memory between the two of them? That way the memory would be there but not "in your face" so to speak against the other men in her life.
Your blocks so far are beautiful.