I have an Angular application, which needs to send N XHTTP requests, where 1 <= N <= 10000.
The application needs to handle it as fast as possible, so preferably there should be multiple active XHTTP requests at the same time, with a slide-window of multiple requests at the same time. Using WebSocket, or other streaming-like solution is not possible due to server-side API limitations.
My first idea was to use something like RxJS forkJoin, but I struggle to limit the concurrent requests number. As far as I know, there are API limitations for max requests number, for instance Chrome will allow only 8 simultaneous requests.
Most of the solutions/tutorials I found either a.) does not limit the maximum number of concurrent connections or b.) does not update dynamically (timeout solutions are not efficient for this task).
For instance:
const test = () =>
request(`https://swapi.co/api/people/1/`)
.pipe(
delay(1000),
switchMap(response => from(response.films)),
concatMap((url: string) => request(url).pipe(delay(1000))),
scan((acc, res) => [...acc, res.title], []),
tap(console.log)
)
.subscribe()
is not good for me, as the limitation is achieved by delay, but I would like to achieve something like a thread based solution: there are maximum of Y number of concurrent connections, and if one finishes, a new request starts immediately.
const test = () =>
request(`https://swapi.co/api/people/1/`)
.pipe{
switchMap(response => from(response.films)),
specialOperatorIAmLookingFor((url: string) => request(url), 8), // where '8' is the maximum number of paralell requests
scan((acc, res) => [...acc, res.title], []),
tap(console.log)
)
.subscribe()
Any ideas how to solve this nicely? RxJS feels like there should be a solution for this already written.
See Question&Answers more detail:
os