It depends what HLAs we're talking here. If you can, for example, win whatever fight(s) you face using your HLAs once per day at most, HLA or not, then it's probably just less convenient to have them as innates. Otherwise, it's likely a difficulty increase.