Isn't that an argument that the function implementation might break, not that callers might?

Assuming implicit conversion for union types, you could either widen argument or narrow return without changing that implementation, but not both. A different implementation may not be able to handle either modification, though.