That’s Synapse being bad and already having a tech debt. Matrix is surely more expensive to run than other protocols, but not much considering federated nature.
slow syncs
Being worked on with syncv3. New sync is crazy fast.
About encryption, it is also being worked on heavly.
The one bad thing I can say about Matrix is just how much is being “work in progress”. But I would choose a protocol that is going to do my checklist than others that would never do.