Each view of the world requires that the entire visible world be loaded twice, so that it can be seen from each players perspective independent of the other.
If we go into a dungeon, I go left and you go right, it has to render both pathways simultaneously. In a single player or single screen two player game, it only has one path to consider.