They don’t need to be in NATO for us to store weapons there. We have plenty of bases in non NATO countries. We’ve even had bases in countries the Russians invaded. (Georgia, 2008)
But also, NATO doesn’t actually have that as a hard rule. There’s been debate about it and NATO Members are wary of such a member pulling everyone into a war right away, but there’s nothing stopping them from doing it.