Signal can’t see who is texting who. They can’t see which groups you are part of. Those information are end to end encrypted, same as your chats itself, your profile picture, your stories, etc.
Signal doesn’t store message timestamps either.
What Signal itself knows of you is your phone number, the timestamp of your registration, the timestamp of your last connection to the server. That’s it.
Yes metadata is critical but Signal handles metadata very well. Indeed, even though I’m a fan of Matrix, better than Matrix. Matrix is a metadata nightmare due to it’s centralized structure and the way the protocol works.