Nah, it's definitely true.
I have seen plenty of (especially larger-format) TVs where, with "game mode" (or the respective equivalent) disabled, it's unbearable to even do latency-forgiving tasks like office work on them.
It really depends on the TV. And if your gaming console is plugged into a receiver, it could add more.
My previous TV had about 150 ms of video latency. Even if I enabled Game Mode, it was 75 ms, which was still noticeable.
With my current TV, I have no idea what the latency is because I stopped gaming on console and so I'm not playing Rock Band which had a calibration option to compensate for video and audio latency.
I've had a TV that were nearly 200ms, but it was the absolute cheapest panel I could find.
If you don't configure your TV correctly sure, you could get massive amounts of lag. Even with an OLED display if you turn on all the post processing you're going to have problems. That's not really a fault of the TV though. I disable almost all post processing on my TVs and get a better picture without the downsides.
With digital receivers adding lag is minimal at best. Especially with newer models that don't draw on top of the source signal.
It is quite accurate, but only for the highly processed cinematic modes. Latency is essentially irrelevant for video content as long as the audio stays in sync. The game mode drops all the processing and is usually pretty low.
For example, my LG OLED tv is regarded as one of the best for gaming. Game mode latency is about 9ms. Cinema mode is about 90ms.