It refers to the annexes you see growing out of the back of the house. People used to (and sometimes still do) start with a modest house, and then as time went on and money came in, build out into the yard, adding a new kitchen, toilet, shed, veranda, etc.
It has since also come to refer to things that get extended and patched up over time without a lot of coherence or forethought, like our tax codes (they just add rules, bonuses, penalties, etc. on top of the baseline instead of doing a thorough tax reform)