When my daughter started researching my father's family in England she found a lot of working class ancestors -- many were domestic servants-- and then great excitement-- in one census she found that the family had a servant themselves! (They had temporarily moved up in the world)
On my mother's side we found a few convicts sent out to Australia from England -- but that is a source of pride for anyone in Australia studying their ancestry.