Writing the minimal contract of a request
A workable request names an owner, a project, an available input, an identified command or procedure, and an expected result. It states the number of simultaneous cards, the memory per card, the estimated duration, and the destination of the outputs. Finally, it specifies the deadline, the reason for that deadline, and the person authorized to accept a delay or a reduction in scope.
The absence of this information must not be hidden by a high priority. A task whose inputs are not ready remains inadmissible, even if the project is important. Ask for what is missing and set the point for re-examination. The queue then becomes a decision tool: it does not turn into a warehouse of requests impossible to launch that everyone hopes to see processed spontaneously.
Distinguishing admission, reservation, and right of way
Admission verifies that the task can be examined: rights, data, compatibility, capacity, and procedure present. A reservation then protects a decided slot or resource. Priority chooses the order among admissible requests competing for an available resource. These three decisions can depend on different people; make this division explicit to avoid an implicit validation of the technical need.
A free card does not make all tasks admissible. The necessary disk space may be missing, a license may limit concurrency, or the previous result may be required. Conversely, a technically ready request may be waiting for a scheduling decision. Display an understandable reason and a next action, rather than a simple numeric rank that no one knows how to interpret.
Defining priorities that do not become privileges
Choose a few readable classes, for example confirmed external deadline, routine work, and exploration without a firm date. Determine who can assign a class and what evidence justifies it. Within a class, a rule of seniority or rotation can make the choice predictable. A request split into a hundred small submissions must not mechanically get a hundred times more chances than a request from another team.
Slurm distinguishes in particular the admissible waiting age and fair share based on allocated and consumed resources. This illustrates two different questions: how long have we been waiting, and what share have we already used? Your rule can remain manual. Do not copy a scoring formula without understanding its parameters; the weights of a particular scheduler are not a universal standard of fairness.
Fictional example: letting an urgent request in without pushing others out
The fictional team "Plans and Benchmarks" uses a single card. At 9 a.m., a task from the Plans project has a confirmed slot until 11 a.m.; it is not interruptible. Two complete requests are waiting: Benchmarks, forty minutes, and Sketches, eighty minutes. At 9:30 a.m., a thirty-minute urgent request arrives, submitted by Benchmarks for a review at noon. All these times are pedagogical assumptions, not measurements.
The local rule allows a validated urgent request to move to the next release point, without interrupting work already in progress. The urgent request therefore starts at 11 a.m. and theoretically ends at 11:30 a.m. The team then grants access to Sketches to prevent Benchmarks, already served by its urgent request, from monopolizing the queue. This exception and Benchmarks' new expected start are announced to the relevant managers.
| Hypothetical slot | Project | Reason for access |
|---|---|---|
| 9 a.m.–11 a.m. | Plans | Slot already committed, not interruptible |
| 11 a.m.–11:30 a.m. | Benchmarks, urgent | Confirmed external deadline before noon |
| 11:30 a.m.–12:50 p.m. | Sketches | Rotation after the urgent request served for Benchmarks |
| 12:50–13:30 | Benchmarks, routine request | Next admissible request |
Preventing long jobs from starving
Always serving the shortest request produces an apparently smooth queue but can postpone a long job indefinitely. Plan reserved windows or a review once a stated waiting age is reached. This threshold is an internal rule to be defined, not a guaranteed service deadline. If the long job genuinely exceeds the available period, admitting it should lead to a new schedule rather than a dead-end wait.
Short gaps can accommodate tasks that do not postpone a reserved departure. This decision assumes a credible estimate, a genuinely compatible resource and a possibility of finishing within the window. Do not fill every margin in the schedule with new commitments. A reserve intended to absorb a variance is not a free slot; its use must have an owner and a known consequence.
Planning a cancellation with a clear scope
Designate who can cancel their own request, a project's request or an execution that has become dangerous for other jobs. For a request still waiting, confirm its removal from the queue order. For a job already started, first check what can be interrupted, what must be preserved and who is waiting for its outputs. A "cancellation requested" message does not yet prove that the resource is free.
The actual tool determines the technical effects. The scancel documentation distinguishes a job from its steps, as well as sending a signal and cancelling the job. Before using an equivalent tool, check the identifier and the scope concerned; avoid a global selection for convenience. Then verify that processes have ended and where partial outputs have gone. The data recovery plan is covered in the migration file, separately from this priority rule.
Limiting attempts and isolating a faulty task
Decide before execution how many automatic attempts are acceptable for the same cause. An invalid input does not become correct because it is restarted; a temporarily unreachable dependency may justify a new attempt after verification. The rule must state the maximum number, any delay and the person responsible for the review. A counter reset on each new submission defeats this protection: link successive requests to the same problem.
At the decided limit, remove the faulty task from normal competition and place it in logical quarantine, with a readable reason. The rest of the admissible jobs can continue. Reinstatement requires a fix or a validated explanation, a small targeted check and a decision by the owner; it does not automatically regain the priority of an old emergency. This procedure handles admission after a failure. Preserving outputs and recovering them remain in the migration file, without inventing a new state registry here.
Testing the policy with four simple situations
Have project owners review the rule with four cases: two identical requests, an emergency during a non-interruptible job, a long task surrounded by short requests and an incomplete request classified as urgent. For each case, a different person must arrive at the same next departure or explicitly identify the arbitration required. A rule that only its author understands is not ready to be applied.
Also check that a cancellation does not grant the same capacity twice and that a blocked job does not needlessly prevent another admissible job. These checks can start on paper; they do not prove that a scheduler has been implemented. If you later automate, reproduce the same scenarios on the configured tool before making a project commitment depend on it.
Measuring the service delivered by the queue
Track the waiting of eligible requests by class and by project, along with the reasons for blocking. Separate waiting, utilization and time to accepted result. A single total of completed jobs can mask a team that is constantly pushed back or an accumulation of urgent tasks. Also examine the gaps between estimated and actual utilization: they directly influence future reservations.
Revise the policy at a deadline chosen with the team, or after an incident that reveals a missing rule. Change one rule at a time and announce its effect on requests already in the queue. The expected outcome is an explainable order of processing, traceable exceptions and a coherent schedule. It replaces neither Capordia account authentication nor machine permissions, and it creates no support commitment.
Questions to decide
Should an urgent request always go first?
An urgent request must be eligible, justified and approved according to the common rule. It does not make in-progress work interruptible. Define its point of entry, the requests it displaces and the person who owns the effect on their deadlines.
How do you keep one team from monopolizing the slots?
Look at the resources already obtained and the waiting of other projects, then use rotation or an explicit sharing rule. The raw number of requests is not a measure of fairness. Also plan for the handling of long jobs.
Does this method require installing Slurm?
No. The Slurm documentation sheds light on certain distinctions, but the policy can first be applied in the team's coordination tool. Any eventual automation requires its own configuration and controls; no scheduler is assumed to be provided.