You just created an "original adaptation" of the pattern. Looks good to me. I actually just finished a top where I stitched a row on out of place. Not about to tear it out, looks good enough for me.
Actually, it looks correct. It appears as if you are looking at the closest buildings and as you look away it gets smaller. I would leave it. Looks very natural.