Dynamically control speed of background jobs
We currently have a constant rate limit on the number of jobs to run per period. We can change this number, but it’s not depending on the backend capacity/load.
Nice to have:Submitted by Elmer van Chastelet on 9 October 2023 at 16:50
To add automatic dynamic rate limiting for running background jobs, we need to interact more with the backend and monitor its current load and see if more jobs can be spawned per minute. Seems non-trivial, as the expected load per queued job is variable.
Log in to post comments