It works both ways. A hit on the defenders does count as half hits against the wall as well. A gate hit is different, the defenders are not riding the gate, they are riding the wall around the gate, I wouldn't expect men on the wall to take damage from a gate hit. IN thanes example the gate is an extra portion of the wall it is set into, so even if the gate comes off its hinges the wall around it is just fine.
As for breaching a wall....trying to breach it from the bottom is extremely difficult. The stones on the bottom are not only larger they have the pressure of the stones above them, they are much more difficult to move. A breach in the wall was much more likely to occur well above the base where the stone was thinner and the pressure less, and well above the base means hits are likely to shower the defenders with debris from the wall and the stone. The only times that I know of walls being breached at the base involved two things: smaller walls (such as those at Drogheda when Cromwell breached them) and rifled cannon who transferred energy in a much more efficient way. Older cannon, mortar, ballista, catapults, due to their arc are more likely to hit the top of the structure rather than the base anyway, especially those with the greatest arc such as trubechet.