CapordiaDefine the state to recover before choosing files
Start with a verifiable sentence: resume unfinished tasks, review accepted results or continue training from a saved state. These goals do not require the same bundle. Name the application version, the input set, the parameters and the reference point in time. A recent file date alone does not indicate that all components represent the same state of the work.
For PyTorch training, the documentation distinguishes model parameters from a resume checkpoint that notably includes the optimizer state. Determine the elements needed for your own protocol: progress, any scheduler, data and other application states. An export intended only for inference may be valid for that purpose while being insufficient to continue training at the same point.
Make the trial independent of the working directory
Prepare a separate destination, with the necessary space and permissions. Clearly identify the chosen backup and protect its reference copy. The test must not overwrite work in progress or publish its results in place of accepted outputs. Disable or redirect the application's external actions: sending notifications, writing to a shared folder or launching an additional campaign.
Use your tool's restore function within this scope. Restic, for example, lets you choose a destination with --target and preview a restore with --dry-run. Its documentation warns that an in-place restore can leave a partial state if it is interrupted. These options illustrate the method; they do not mean that Restic or a managed backup is provided with a Capordia rental.
Verify the copy, the state and the use in turn
First check that the bundle matches the expected inventory and that the files have not been altered. Then move on to the actual opening: permissions, relative paths, dependencies, data reading and state loading. A bundle can pass its integrity check while omitting a file that was never in the inventory. The functional check answers this second question.
Finally, launch a bounded operation in a fresh process, without implicit access to the original directory. Note what it reads, what it produces and what allows its acceptance. For a batch resume, verify that it does not republish tasks already validated. A test on a few inputs only demonstrates that scope; it guarantees neither the whole campaign nor its full duration.
| Level | Question | Element to record |
|---|---|---|
| Integrity | Does the bundle received match the bundle expected? | Inventory and check result |
| Restored state | Do the files and dependencies allow loading? | Versions, paths and discrepancies noted |
| Use | Does a representative operation give the expected result? | Test inputs, outputs and decision |
Worked example: an intact archive that does not resume
Entirely fictional case: a campaign includes 240 folders. At the saved point, 180 are accepted and 60 remain to be processed. The team restores the archive in a separate space. All listed files pass the integrity check, but the application cannot find a mapping table that had until then been loaded from a colleague's workstation. The restoration is refused: the state cannot be resumed autonomously.
The team adds the exact version of that table to the package and documents its path. It restarts the restoration in a new destination, then checks the registry: 180 + 60 = 240 identifiers, with no duplicates. On ten test folders among the sixty remaining, the processing produces the twenty expected files, i.e. two per folder. Their content is accepted according to the defined criteria. These ten tests remain separate from the production results; the initial registry is not advanced by the exercise.
Measure a full duration and know what it proves
Time from the start of the necessary preparation to the accepted result, distinguishing waiting, restoration, configuration and checking. In our fictional example, these steps take 15, 75, 20 and 25 minutes respectively: the total reaches 135 minutes, i.e. 2 h 15. For an internal window of three hours, the remaining margin is 180 − 135 = 45 minutes. This calculation describes no service performance.
Record the conditions that make this measurement reusable: package size, destination, versions, availability of people and scope of the check. The age of the backup answers another question: what later work might need to be redone? A fast restoration of a state that is too old may remain unacceptable. Define both criteria with the project lead, without confusing them.
| Step | Time |
|---|---|
| Prepare the destination | 15 min |
| Restore the package | 75 min |
| Configure the application | 20 min |
| Accept the test result | 25 min |
| Total / margin over 3 hours | 135 min / 45 min |
Conclude the exercise and keep a clear next action
The report fits on one page: chosen backup, objective, test environment, deviations, accepted result and limitations. If an item is missing, assign its correction and then rerun the relevant check. Do not replace a failure with a vague phrase like "archive available". After the exercise, identify the test outputs and remove them according to your procedure, without touching the copies kept for retention.
Repeat the test when the backup format, the software or an important dependency changes, and before a deadline that would make recovery difficult. The frequency depends on the project; it is not an automatic commitment of the rental. Over a period of 3, 7 or 30 days, plan a slot to verify the recovery and pass its result to the person who will operate what follows.
Questions to decide
Is a correct checksum enough to accept a backup?
It verifies that the relevant files match, not the completeness of the project or its ability to restart. Add a load in a separate space and a representative operation. The report specifies the scope of each of these checks.
Should the entire production be reproduced for each test?
Not necessarily. Choose a scope that covers the dependencies and the essential criteria, then state its limits. A small test is useful for detecting a missing file; it does not measure the ability to restore a much larger volume.
What should be done if the test succeeds only with the old directory accessible?
Look for the implicit dependency: configuration, data, library or external path. Add it to the package or document how to find it, then start again without that shortcut. As long as this need remains unknown, the recovery stays dependent on the original environment.