An alternative I am seeing mentioned with some frequency is Tailscale, which doesn't need port 22 open to the internet, since it's using its own network's connectivity to facilitate your "tailscale SSH" connectivity. From what I read it's very similar to Amazon's SSM Agent.
The usefulness here is that you're closing off ports and reducing your exposure, the downside is that you need proprietary agents installed on the remote devices, and clients (tailscale itself or the SSM extension to AWS CLI) on proprietary networks doing the routing for you. Which might be perfectly fine for your use cases.
I've done even less reading but is Cloudflare's WARP client the same thing for their own network?
It's a proprietary network but not proprietary agents (except the bits specific to proprietary platforms).
One handy feature this enabled is that you can include their open source go library in your program and avoid needing to install anything besides your own binary.
Isn't there even an open implementation of the network/coordination layer (Headscale)?
But regardless, the best term I've heard for these types of networks is overlays. You can add "zero trust" or whatever to make it seem more fashionable, or VPN if you're old school. I remember using Hamachi for this type of use back in maybe the early aughts.
One of the major issues was that a VPN client requires additional configuration and software. Connectivity is the least of the problems they are trying to solve.
Tailscale is just a commercial service that builds upon wireguard. It automatically generates certificates for each of your devices, ensures they're rotated and up to date, automatically configures routing and DNS between your devices and offers some additional functionality.
Tailscale has open source clients but a proprietary server to do this, but you can use the open source alternative headscale instead: https://github.com/juanfont/headscale
Convenience. you're free to run wireguard yourself, but that's a lot of faffing about with config files that some people don't want to do. and then on top of that, they have clients for mobile and eg AppleTV. that may be outside your use case, but some find it handy.
The usefulness here is that you're closing off ports and reducing your exposure, the downside is that you need proprietary agents installed on the remote devices, and clients (tailscale itself or the SSM extension to AWS CLI) on proprietary networks doing the routing for you. Which might be perfectly fine for your use cases.
I've done even less reading but is Cloudflare's WARP client the same thing for their own network?