Jump to content

Data Storage & Backup: Difference between revisions

From CryoNAVwiki
Bot edit via cryonavedit.py
 
Bot edit via cryonavedit.py: redirect to sidebar-target title
Tag: New redirect
 
Line 1: Line 1:
CryoNAV is designed for single-server deployment with a simple, file-based storage model: SQLite for metadata, the host filesystem for raw and processed image data.
#REDIRECT [[CryoNAV Data Storage Backup]]
 
== Database storage ==
 
The database stores only metadata, file paths, processing parameters, and state. It remains small (tens of megabytes even for large projects) and can be backed up with a single file copy.
 
The database schema mirrors the physical cryo-ET workflow: Projects contain Grids, which contain Search Maps and Tilt Series, from which Processing Runs produce Tomograms and Denoised Tomograms. Cryo-FIB images from instruments such as the Aquilos are stored as a dedicated entity linked to grids.
 
In addition to file paths, the database records the size of all tracked files (raw data, intermediate outputs, final results), enabling the storage monitoring described below.
 
== Storage monitoring ==
 
The '''Storage''' page in the CryoNAV interface provides an overview of disk usage across raw data, intermediate processing files, and final outputs for all projects, helping teams monitor their storage footprint and identify candidates for cleanup.
 
== Selective cleanup ==
 
Through the interface, users can selectively delete intermediate processing files (such as motion-corrected stacks, aligned series, or reconstruction volumes from superseded processing branches) while retaining the raw data and database records.
 
Because every processing step stores its complete parameter set in the database (see [[Metadata Management#Processing provenance|Processing provenance]]), the full pipeline can be re-executed from raw frames at any time, allowing researchers to reclaim storage without losing the ability to reproduce their results.
 
== Backup ==
 
The recommended backup model is a single-file copy of the SQLite database plus a filesystem-level backup of the underlying image storage. SQLite's simplicity makes the database portion straightforward; the image data follows the same backup model as any cryo-EM data store.
 
== See also ==
 
* [[Metadata Management]]
* [[Data Sharing & Archiving]]
 
[[Category:Modules]]

Latest revision as of 22:31, 20 May 2026