Unfortunately that’s exactly how it works.
Look at any country’s border and tell me which ones weren’t established by violence.
The actual question is, what alternative to accepting Israel’s existence would you propose. Because forcefully removing them would just be one more crime.