I doubt that can fully be the case, because there are other functional units on SMs, like Load/Store, ALU / Integer ops, and Special Function Units. But you may be right, we would need to consult the academic "investigatory" papers or blog posts and see whether this has been checked.