I don’t think it was a stupid question; frankly, it’s not something I find intuitive. I have to stop and think about it. Also, living with someone with allergies makes you more aware of them, since your brain tends to purge knowledge you don’t use. I’d think it’s a curse - especially since my memory is shit to begin with - except that I know a couple of people with eidetic memories, and that can present its own problems.
I know one guy with an eiditic memory who has a problem with information that he learns wrong the first time. He has to build a sort of linked-list model in his brain for corrections, and then do a sort of very slow O(n) crawl of the list to end up with the right result. So say he’s introduced to you and they say your name is Becky; that gets stored in his memory. Then you correct them and say your name is “Susanne”, so he makes a “correction” link. But because someone coughed when you said it, he heard “Susan”, and that was w what got stored; so he has to make a second correction. From then on, whenever he runs into you he has to go through this “Becky” -> “Susan” -> “Susanne” process, and he says it’s real slow and can take a couple of seconds, and longer if there are more corrections. That seems a poor trade-off to me for being able to glance at a page and then repeat it back verbatim by reading the picture in his memory.