Admin: Bot edit via cryonavedit.py: redirect to sidebar-target title

2026-05-20T20:30:50Z

Bot edit via cryonavedit.py: redirect to sidebar-target title

← Older revision		Revision as of 22:30, 20 May 2026
Line 1:		Line 1:
	'''SmartScan''' is CryoNAV's automated import engine for cryo-ET acquisition data. It detects acquisition folder structures, parses metadata, maps tilt series to their physical positions on search maps, and recognizes pre-processed outputs from facility pipelines.		#REDIRECT [[CryoNAV Tilt Series Data Collection Import]]

	~~== What SmartScan does ==~~

	~~When pointed at an acquisition root folder, SmartScan performs three operations:~~

	# '''Structure detection.''' SmartScan identifies the acquisition software that generated the data by recognizing characteristic folder layouts and file patterns. It detects atlas images (Atlas_*.mrc), search map directories (SearchMap_XXXXXX_YYYYYYY/), and tilt series data (Position_X.mdoc with associated .eer frame files).
	# '''Metadata extraction.''' The engine parses BatchPositionsList.xml to extract the mapping between search map names and tilt series position names. It reads .mdoc files for acquisition parameters: pixel size, voltage, dose rate, tilt angles, and timestamps. This metadata is stored alongside the data entities for later filtering and processing.
	# '''Coordinate mapping.''' SmartScan establishes the three-tier relationship between grids, search maps, and tilt series. It extracts the X,Y stage coordinates from position metadata and maps tilt series to their physical locations on search map images, enabling the [[~~Grid Visualization#Search map overlay\|search map overlay]] visualization.~~

	~~== Recognizing pre-processed data ==~~

	Many microscope facilities provide motion-corrected stacks, aligned tilt series, and even SIRT reconstructions alongside raw acquisition data. SmartScan detects common facility output structures (e.g., TomoRun directories containing ''_MC.mrc'', ''_ali.mrc'', and ''*_SIRT_rec.mrc'' files) and automatically links processed outputs to their corresponding raw tilt series.

	Similarly, SmartScan can import data from ongoing or completed IMOD processing pipelines, recognizing standard IMOD output files and directory layouts (aligned stacks, reconstruction volumes, .defocus files, transform files). Researchers who have already begun, or finished, processing tilt series through IMOD outside of CryoNAV ~~can bring that work into the platform without starting over, gaining access to CryoNAV's visualization, curation, and management features on top of their existing results.~~

	~~This capability is particularly valuable for rapid quality screening: imported data can be assessed immediately, before any new processing is run.~~

	~~== Manual assembly and integrity checks ==~~

	~~For datasets that do not conform to recognized folder structures, CryoNAV provides a manual assembly mode where users explicitly define the grid hierarchy and link files.~~

	SmartScan also performs integrity checks during import: it validates that expected files are present for each tilt series, flags missing .mdoc files or incomplete frame sets, and routes data with broken metadata relationships to an "unlinked data" queue for manual review.

	~~== Supported acquisition software ==~~

	~~Current support: EPU and SerialEM folder structures. Additional acquisition-software support can be added through the modular detection framework.~~

	~~== See also ==~~

	* [[Metadata Management]]
	* [[Grid Visualization]]
	* [[Tomogram Processing Overview]]

	~~[[Category:Modules~~]]

Admin: Bot edit via cryonavedit.py

2026-05-20T20:19:35Z

Bot edit via cryonavedit.py

New page

'''SmartScan''' is CryoNAV's automated import engine for cryo-ET acquisition data. It detects acquisition folder structures, parses metadata, maps tilt series to their physical positions on search maps, and recognizes pre-processed outputs from facility pipelines.

== What SmartScan does ==

When pointed at an acquisition root folder, SmartScan performs three operations:

# '''Structure detection.''' SmartScan identifies the acquisition software that generated the data by recognizing characteristic folder layouts and file patterns. It detects atlas images (Atlas_*.mrc), search map directories (SearchMap_XXXXXX_YYYYYYY/), and tilt series data (Position_X.mdoc with associated .eer frame files).
# '''Metadata extraction.''' The engine parses BatchPositionsList.xml to extract the mapping between search map names and tilt series position names. It reads .mdoc files for acquisition parameters: pixel size, voltage, dose rate, tilt angles, and timestamps. This metadata is stored alongside the data entities for later filtering and processing.
# '''Coordinate mapping.''' SmartScan establishes the three-tier relationship between grids, search maps, and tilt series. It extracts the X,Y stage coordinates from position metadata and maps tilt series to their physical locations on search map images, enabling the [[Grid Visualization#Search map overlay|search map overlay]] visualization.

== Recognizing pre-processed data ==

Many microscope facilities provide motion-corrected stacks, aligned tilt series, and even SIRT reconstructions alongside raw acquisition data. SmartScan detects common facility output structures (e.g., TomoRun directories containing ''*_MC.mrc'', ''*_ali.mrc'', and ''*_SIRT_rec.mrc'' files) and automatically links processed outputs to their corresponding raw tilt series.

Similarly, SmartScan can import data from ongoing or completed IMOD processing pipelines, recognizing standard IMOD output files and directory layouts (aligned stacks, reconstruction volumes, .defocus files, transform files). Researchers who have already begun, or finished, processing tilt series through IMOD outside of CryoNAV can bring that work into the platform without starting over, gaining access to CryoNAV's visualization, curation, and management features on top of their existing results.

This capability is particularly valuable for rapid quality screening: imported data can be assessed immediately, before any new processing is run.

== Manual assembly and integrity checks ==

For datasets that do not conform to recognized folder structures, CryoNAV provides a manual assembly mode where users explicitly define the grid hierarchy and link files.

SmartScan also performs integrity checks during import: it validates that expected files are present for each tilt series, flags missing .mdoc files or incomplete frame sets, and routes data with broken metadata relationships to an "unlinked data" queue for manual review.

== Supported acquisition software ==

Current support: EPU and SerialEM folder structures. Additional acquisition-software support can be added through the modular detection framework.

== See also ==

* [[Metadata Management]]
* [[Grid Visualization]]
* [[Tomogram Processing Overview]]

[[Category:Modules]]

Tilt Series Data Import - Revision history

Admin: Bot edit via cryonavedit.py: redirect to sidebar-target title

Admin: Bot edit via cryonavedit.py