> A fast SSD can easily hit a gigabyte per second sequential read speed, far faster than your typical network.
It's important to note that often your disks aren't directly attached to your compute. That's frequently the case in (particularly cheap) cloud instances.
That's the thing, you don't necessarily need a bunch of cloud instances to process data. If you must be doing everything in "the cloud" every service has dedicated instances with fast attached storage available. You can spin up one long enough to rip through your data instead of trying to distribute it over hundreds of workers.
It's also a domain where you can buy an off-the-shelf desktop for a few hundred dollars to do the work. That's the thrust of this whole thread, because scalable "cloud" systems exist and look cheap people obsess about throwing more instances at problems.
Modern commodity systems are ridiculously powerful and far more capable than people tend to assume. Even "the cloud" gets underestimated because people look at the low end cheap instances and assume they need to spin up hundreds of those when one beefy image for a short duration could do the same work.
so, a hundred of the cheapest cloud instances cost you what? 1000 USD per month? You can easily get a deskside workstation for 10000 USD with a nice 2TB NVME-SSD, a 32-Core Threadripper and 128GB of memory... Utilization might be lower of course, but even if it's only utilized 8h/day this seems like a bargain for any solid business. For any startup aiming for exponential growth by burning through cash and having a 15month "half-life" this is not gonna work out though
It's important to note that often your disks aren't directly attached to your compute. That's frequently the case in (particularly cheap) cloud instances.