Poland was given a lot of territory in the west and the Soviet Union took a lot of territory from the east, without mass killings. And yes there were people in all of those regions.
In fact a lot of borders changed after WW2, and the only (or one of very few) place this devolved into a decades long murder fest is the middle east.