Jump to content

CryoNAV Data Storage Backup

From CryoNAVwiki

CryoNAV is designed for single-server deployment with a simple, file-based storage model: SQLite for metadata, the host filesystem for raw and processed image data.

Database storage

The database stores only metadata, file paths, processing parameters, and state. It remains small (tens of megabytes even for large projects) and can be backed up with a single file copy.

The database schema mirrors the physical cryo-ET workflow: Projects contain Grids, which contain Search Maps and Tilt Series, from which Processing Runs produce Tomograms and Denoised Tomograms. Cryo-FIB images from instruments such as the Aquilos are stored as a dedicated entity linked to grids.

In addition to file paths, the database records the size of all tracked files (raw data, intermediate outputs, final results), enabling the storage monitoring described below.

Storage monitoring

The Storage page in the CryoNAV interface provides an overview of disk usage across raw data, intermediate processing files, and final outputs for all projects, helping teams monitor their storage footprint and identify candidates for cleanup.

Selective cleanup

Storage Cleanup tab: pick intermediate files to delete while retaining raw data and processing records.


Through the interface, users can selectively delete intermediate processing files (such as motion-corrected stacks, aligned series, or reconstruction volumes from superseded processing branches) while retaining the raw data and database records.

Because every processing step stores its complete parameter set in the database (see Processing provenance), the full pipeline can be re-executed from raw frames at any time, allowing researchers to reclaim storage without losing the ability to reproduce their results.

Backup

The recommended backup model is a single-file copy of the SQLite database plus a filesystem-level backup of the underlying image storage. SQLite's simplicity makes the database portion straightforward; the image data follows the same backup model as any cryo-EM data store.

See also