Honestly, without going into too much detail - aimlessly wandering and exploring is kind of a Zelda games thing. More so with the older ones. Trying to play the original NES with no guide other than my own notes is a lesson in patience, and I replay it every year. As well as ALttP here. You don't need to find all the heart containers, so don't spend too much time searching for them - a lot of them are going to be hidden behind items you haven't unlocked yet. Save that for part of your final dungeon prep. Do look for the enhanced armor to take less damage though - however I'm not sure when exactly that unlocks to be found.
But the swamp palace is a PiTA. Those water skimmers are a giant pain always running into you, and all the various switches and backtracking. I don't want to say that the dungeons get easier after this one - it's just where the game really starts to Zelda and the mechanics of the game should start clicking as you go through and on to the next one, making them easier.