Would be curious how long ago you tried this and what your setup is? The lowest spec machine I have is a Windows 10 machine with an i7 and 8 GB RAM and it is super responsive on the latest version of VS Code.
A few weeks ago with Ubuntu 22.04 on i7 with 16GB RAM. Perhaps it's related to using the devcontainer feature of VSCode, although I run jupyterlab in the same container
That might be it. I’m not sure about devcontainers (I don’t use them) but as a data point I dual boot into kubuntu on an 7 year old i7 with 8 gigs of RAM and there are no responsive issues.