Humanity invented language though. LLMs did not; they're merely parroting our language back to us.

Wed punish them in training if they came up with their own language

Absolutely a false blanket statement, especially if someone is trying to make a tabula rasa AI that can invent its own language. Far from being punished for it, this would be the AI's reward function!