Except that you don’t just use it for whole hours. 12.01 is clearly post meridian, so it has to be PM if you wanna keep the meaning. With your example 12 am and 12.01 pm would either be one minute apart or you’d have to drop the whole meaning of am and pm, which would just make it an entirely arbitrary construct that’s even more difficult to understand.
On the other hand, as a European I’m quite happy with 24h time keeping. This whole am-pm system is quite unnecessary.