Really, this is an issue in the library/server: the library/server needs to expose HTTP/2's controls on maximum permitted streams.
> And secondly, because with HTTP/2, the requests were all sent together—instead of staggered like they were with HTTP/1.1—so their start times were closer together, which meant they were all likely to time out.
No, browsers can pipeline requests (send the requests back-to-back, without first waiting for a response) in HTTP/1.1. The server has to send the responses in order, but it doesn't have to process them in that order if it is willing to buffer the later responses in the case of head-of-line blocking.
Honestly, over the long run, this is a feature, not a bug. The server and client can make better use of resources by not having a trivial CSS or JS request waiting on a request that's blocked on a slow DB call. Yes, you shouldn't overload your own server, but that's a matter of not trying to process a large flood all simultaneously. (Or, IDK, maybe do, and just let the OS scheduler deal with it.)
Also, if you don't want a ton of requests… don't have a gajillion CSS/JS/webfont for privacy devouring ad networks? It takes 99 requests and 3.1 MB (before decompression) to load lucidchart.com.
> If you do queue requests, you should be careful not to process requests after the client has timed out waiting for a response
This is a real problem, but I've suffered through that plenty with synchronous HTTP/1.1 servers; a thread blocks, but it's still got other requests buffered, sometimes from that connection, sometimes from others. Good async frameworks can handle these better, but they typically require some form of cancellation, and my understanding is that that's notably absent from JavaScript & Go's async primitives.
> No, browsers can pipeline requests (send the requests back-to-back, without first waiting for a response) in HTTP/1.1. The server has to send the responses in order, but it doesn't have to process them in that order if it is willing to buffer the later responses in the case of head-of-line blocking.
Browsers can pipeline requests on http/1.1, but I don't think any of them actually do in today's world, at least that's what MDN says. [1] And from my recollection, very few browsers did pipelining prior to http/2 either -- the chances of running into something broken were much too high.
When we first tried to enable HTTP/2 on our load balancers a few years ago, we ended up breaking several applications built on (iirc) gunicorn. We eventually determined the root cause to be:
1) The browser was sending a "streaming data follows" header flag followed by a 0-byte DATA packet in the HTTP/2 stream to work around an ancient SPDY/3 bug.
2) The load balancer was responding to the HTTP/2 "streaming data follows" header packet by activating pipelining to the HTTP/1.1 backend.
3) The backend was terminating the HTTP/1.1 connection from the load balancer with a pipelining-unsupported error.
The browser removed the workaround, the load balancer vendor removed the HTTP/2 frontend's ability to activate HTTP/1.1 pipelining, and after a few months we were able to proceed.
Diagnosing this took weeks of wireshark, source code browsing, and experimental testing. We were lucky that it broke so obviously that the proximity to enabling HTTP/2 was obvious.
If you can recollect more details, I would love to know what happeend, but I'm not sure about 3) I'm not aware of a pipelining-unsupported error in http (it is a thing in SMTP). It would take a very special HTTP server to look for another request in the socketbuffer after the current one and respond with failure.
I looked it up and I remembered incorrectly: the bug was due to the load balancer activating chunked transfer encoding to the backend nodes due to receiving the described HTTP2 request. It did not involve pipelining.
cURL recently removed its 1.1 pipelining support, it was rarely used and pretty broken in practice because few clients had been using it: https://news.ycombinator.com/item?id=19595375
Firefox supported pipelining, but as far as I remember, that setting was always disabled by default, and you had to manually enable that through about:config. It was a very common performance tip, and there were even some extensions[1] that enabled that for you, but the usage was still limited to only a small group of power users.
> Good async frameworks can handle these better, but they typically require some form of cancellation, and my understanding is that that's notably absent from JavaScript & Go's async primitives.
Go added cancellation support to the standard library at 1.7. I don't like its coupling with contexts, but the implementation is solid and supported throughout most blocking operations, so this statement is patently untrue for Go.
JavaScript really doesn't have a standard way for doing cancellation, which is a shame.
> And secondly, because with HTTP/2, the requests were all sent together—instead of staggered like they were with HTTP/1.1—so their start times were closer together, which meant they were all likely to time out.
No, browsers can pipeline requests (send the requests back-to-back, without first waiting for a response) in HTTP/1.1. The server has to send the responses in order, but it doesn't have to process them in that order if it is willing to buffer the later responses in the case of head-of-line blocking.
Honestly, over the long run, this is a feature, not a bug. The server and client can make better use of resources by not having a trivial CSS or JS request waiting on a request that's blocked on a slow DB call. Yes, you shouldn't overload your own server, but that's a matter of not trying to process a large flood all simultaneously. (Or, IDK, maybe do, and just let the OS scheduler deal with it.)
Also, if you don't want a ton of requests… don't have a gajillion CSS/JS/webfont for privacy devouring ad networks? It takes 99 requests and 3.1 MB (before decompression) to load lucidchart.com.
> If you do queue requests, you should be careful not to process requests after the client has timed out waiting for a response
This is a real problem, but I've suffered through that plenty with synchronous HTTP/1.1 servers; a thread blocks, but it's still got other requests buffered, sometimes from that connection, sometimes from others. Good async frameworks can handle these better, but they typically require some form of cancellation, and my understanding is that that's notably absent from JavaScript & Go's async primitives.