CapordiaDefine the unit of work that can be separated
Write down what constitutes an accepted result: an image, a processed document, a set of simulations, or a final training state. Then ask whether one unit can be computed without waiting for the others. Two documents can be independent during processing, then depend on a shared indexing step. The breakdown must describe that second step instead of forgetting it in a promise of parallelism.
Distinguish computational independence from business independence. An output can be produced without any technical exchange and still be unusable before the whole batch is validated. So name the unit of computation, the unit of delivery, and the assembly rule. This shared vocabulary prevents purchasing from counting cards while the application team counts instances or fragments of the same model.
Choose between independent jobs and close cooperation
For independent tasks, each process receives its own inputs and produces an identifiable output. Another card can run another task without synchronizing at every step. This mode is useful when the goal is about the quantity completed during a window. It does not automatically speed up an isolated task, and contention on data, storage, or licenses still has to be checked.
For cooperative work, the devices exchange data or wait for an intermediate result. The program must handle that distribution. The CUDA guide explains that using multiple GPUs involves contexts, data distribution, launches, and communications. These are properties of the software path and of the observed system; the name of a card in the catalog does not prove that this path already exists.
| Question | Independent jobs | Cooperative work |
|---|---|---|
| What does a card receive? | A distinct, complete task | A part or a replica of the work |
| What has to be coordinated? | Assignment of inputs and collection of outputs | Exchanges, ordering, and synchronizations |
| What benefit should be verified? | Number of accepted results per window | Identical result with adapted duration or capacity |
| What failure should be anticipated? | An isolated task or publication | The effect of an unavailable participant on the group |
Do not confuse replication with memory distribution
Several copies of a model do not mean that its parameters are split across the cards. PyTorch DistributedDataParallel synchronizes gradients between processes that train their local models. So this approach alone does not allow the conclusion that a model too large for one card will fit on several. A partitioning mechanism requires its own documentation and its own validation.
Each card's memory must cover what the software actually places there: data, model state, temporary spaces, and any exchanges. The peak can vary with settings and the processing phase. Record it per device. A sum of nominal capacities hides an imbalance where one card runs out of memory while the others still have some left. The batch's total capacity is not enough to accept the scenario.
Fictional example: distributing eight independent conversions
The fictional team "Index des ouvrages" is preparing eight document conversions. In its working assumption, each conversion occupies a card for twelve minutes, including loading and output. A task fits on the card alone and no exchange is needed before the final assembly. These durations serve only to build the reasoning: they do not describe any execution or GPU performance offered by Capordia.
With one card, eight successive tasks represent 8 × 12 = 96 minutes. With two cards and a balanced split of four tasks each, the parallelizable portion theoretically represents 48 minutes. If the final assembly requires ten minutes on a separate workstation, the complete path becomes 106 or 58 minutes respectively, excluding wait time before starting. This calculation assumes that the two cards do not slow down each other's loads.
| Organization | Parallelizable portion | Assembly | Calculated duration |
|---|---|---|---|
| One card, eight tasks | 96 min | 10 min | 106 min |
| Two cards, four tasks each | 48 min | 10 min | 58 min |
Revealing what can cancel this theoretical gain
In the same example, the documents may not be of equal size. Assigning the four longest to one card leaves the other waiting at the end of the run. Distributing work as availability allows can improve the balance, but it requires a reliable assignment mechanism and an admission policy. The guide on queuing covers this organization; the present calculation does not attest that it is already in place.
Storage can also become the dominant shared resource. Two processes that read the same data or write large results simultaneously do not necessarily have twice the throughput. Also check the concurrency limits of tools and licenses. If the non-parallelizable portion grows, redo the full schedule. Time saved on one phase must not mask a delay added elsewhere.
For distributed work, receive the real topology
Ask which cards are visible, how they communicate and which host resources their processes share. Check the devices accessible from the environment in use, not only from an administrator console. CUDA states that access to another GPU's memory depends notably on topology and system capabilities. A model supporting a feature does not prove the expected link is present on the machine.
Keep a simple diagram of data flow: input, card or process concerned, exchanges, final gathering and output. For each arrow, identify whether it crosses host memory, a link between GPUs or the network. This view helps choose the reception checks. It does not impose a given technology and does not turn a multi-GPU request into a promise of a particular topology.
Compare two organizations without moving the expected result
Prepare a small representative corpus and run, on your authorized target, a reference organization then the candidate organization. Keep versions, settings, expected quality and assembly rules. Compare time to an accepted result, each card's peak memory, errors and consumption of shared resources. Separate observations from estimates; a proposed command is not the same as a result already obtained.
Check in particular that the number of accepted outputs remains the same and that no input is omitted. For a calculation sensitive to the order of operations, choose a justified tolerance rather than an arbitrary equality. An organization that finishes faster with a different quality answers a different problem. Also keep a case where sharing is not advantageous: this limit explains the choice better than a ranking of cards without a shared workload.
Translate the choice into required capacity
Finally, write down the number of simultaneous tasks, the cards needed for each and the essential conditions. A task spread over two cards and two tasks on one card represent two different organizations, even if the total number of cards is equal. The chosen configuration must allow the required organization; do not start from the nominal total and then invent a use for it.
The catalog lists bundles of cards. A B200 bundle contains two GPUs of 180 GB each; this composition does not guarantee that a piece of software will use them as a single memory. The next step is to match the need to the bundle and then to verify the properties actually required. The schedule, the cost and the decision remain provisional as long as the decisive conditions have not been verified.
Questions to decide
Are two GPUs always faster than one?
No. The gain depends on the parallelizable fraction, the exchanges and the shared resources. For an isolated task that cannot be split, a second card may remain unused. Compare the time to the same accepted result, on an explicit protocol.
Does a bundle of two cards double the memory available to my program?
Not automatically. Each device has its own memory and the program must distribute its state or its data. Replicating a model can, on the contrary, duplicate a large part of that state. Check the software mechanism and the peak per card.
When should you start with independent tasks?
When each unit can be computed separately and the goal is to complete more units. Attribution, load balancing and final assembly still need to be validated, without assuming that storage and licenses follow automatically.