In the long term, the EXP penalty matters a small amount. In the short term, especially at low levels, the EXP penalty notably matters! (Yes, I have been one of those "EXP per unit time" calculators!) The advantages of these subraces are also meant to keep them alive longer, meaning fewer reloads in the long-term.
This EXP penalty was probably based off 3.x's level adjustment (LA) system that was a penalty to EXP/levels, at least in the short-term.