You’re under no obligation to invite anyone aside from your spouse-to-be and legal witnesses. Everyone else is optional, especially people who would create drama and/or hamper the enjoyment of other guests you do want at the wedding.
My partner and I eloped with our witnesses and parents when we got hitched. There was griping on both sides of course, but if I didn’t care enough to have anyone else at the ceremony, why would I give a fuck about their feelings and takes on that decision? They could either get over it or not.
Anyone who gets upset with you for making a choice about YOUR life that in no way materially impacts them or theirs needs a reality check. It’s your life and your wedding. You don’t have to do anything you don’t want to.