In a GPU cluster, many jobs compete for limited accelerators. This program studies when running shorter jobs first reduces average waiting time, and when the same policy makes some jobs wait much longer.
The question
How do job size, estimation error, restart cost, and pool capacity change the comparison between size priority and reservation-based scheduling? In particular, can a good average conceal starvation in one class of jobs?
Progress so far
Synthetic simulation isolated a failure mechanism: one job requiring the whole pool can starve when size priority is combined with greedy packing. Two public traces were then checked for that precondition. None of the 11 production pools contained a job that filled its pool.
The seven studies that followed were largely the work of breaking this project’s own numbers. The driver was concurrency, not pool size (EP-0006). The headline — support including the whole cluster causes starvation — disappeared when only the background granularity changed (EP-0008). The flagship phase diagram’s axis was confounded by a factor of 18 (EP-0009). “Decided by two numbers” was retracted (EP-0010).
Then the detector itself turned out to be broken (EP-0011). The flow balance used to locate the boundary is a censoring ratio — mean response time divided by the observation window — and cannot see divergence. While the mean response time grew 3.9×, the detector moved from 0.562 to 0.574. Every operational number was put on hold.
The replacement, alpha (the elasticity of mean response time with respect to the observation window), reads 0 where the system is stable and 1 at linear divergence, and moves the boundary by at most 0.02 when the window is multiplied by eight (EP-0012). The operational numbers are back, now with a frequency column.
Current public state
The three quantities to watch are largest-job share of pool capacity × concurrent job count × large-job frequency. In the 256-server, rho 0.85 synthetic model, the rule-of-thumb ratios are 0.92 (frequency 0.002) or 0.72 (frequency 0.02) at concurrency 10, and 0.73 or 0.66 at concurrency 30. Dropping frequency makes the number conservative for rare large jobs and dangerous for frequent ones.
In a new cluster, inspect whether mean response time grows with the observation window rather than using completion rate. If doubling the window raises the mean by at least 1.4×, that class is diverging. The real-cluster reach is extremely narrow: ten of Philly’s 11 virtual clusters contain no job above a quarter of capacity, and the sole exception has exactly one job out of 19,100 above half of capacity.
Evidence boundary: This is not a scheduler comparison on a real arrival sequence, and there are no independent external replications.
Read the current Research Note (EP-0013) →
Published Research Notes
- EP-0001 — Estimation error and restart cost reverse which GPU scheduler wins
- EP-0002 — A good average, a job class that never finishes, and three broken stability detectors
- EP-0003 — Whole-cluster jobs, not mean gang size, decide when size-based scheduling breaks
- EP-0004 — Real GPU clusters never received a job that filled the pool
- EP-0005 — The safe ratio falls as the pool grows
- EP-0006 — The driver was never pool size. It is how many jobs compete at once
- EP-0007 — Measured at the real cluster’s position, and withdrew an explanation carried for four studies
- EP-0008 — Changing only the granularity of the background made the headline result disappear
- EP-0009 — The axis of the flagship phase diagram was confounded by a factor of 18
- EP-0010 — Retracting “this is decided by two numbers”
- EP-0011 — The detector was not measuring divergence. It divided by the window, so it never moved
- EP-0012 — Replaced it with a statistic whose boundary stays put, and restored the numbers
- EP-0013 — The real-cluster claim rests on one job out of 19,100 (latest)