Jump to content

CryoNAV EPU SerialEM Folder Structures

From CryoNAVwiki

Cryo-EM acquisition software organises raw data in two very different ways. EPU (Thermo Fisher) enforces a rigid hierarchy that mirrors the physical grid (atlas -> grid square -> foil hole -> exposure), while SerialEM produces a mostly flat layout with rich text-based metadata. CryoNAV's SmartScan importer understands both, but you need to know what each acquisition tool writes to disk before pointing it at a session folder.

EPU folder structure

EPU (Thermo Fisher) creates a fixed directory tree rooted at the session folder. Every high-magnification acquisition produces six files: summed image (.mrc/.tif), .jpg thumbnail, dose-fractionated movie (_Fractions.mrc, _Fractions.tiff, or _EER.eer on Falcon 4), two XMLs, and a .dm5 transfer checksum. 

<SessionName>/
    Metadata/
        EpuSession.dm           # binary session config
        ScreeningSession.dm
    Atlas/
        Supervisor_<date>_<id>_Atlas/
            Tile_###_###_*.mrc
            Atlas.mrc           *.xml
    Images-Disc1/
        GridSquare_<8-digit-id>/
            GridSquare_*.mrc  .xml  .jpg
            FoilHoles/
                FoilHole_<id>_<date>_<time>.mrc  .xml
            Data/
                FoilHole_*_Data_*_*_<date>_<time>.mrc
                FoilHole_*_Data_*_*_Fractions.mrc / .tiff / .eer
                FoilHole_*_Data_*_*.xml
                FoilHole_*_Data_*_*.jpg
                FoilHole_*_Data_*_*.dm5
    <GainReference>.gain

Filenames encode internal (non-sequential) EPU IDs: FoilHole_<HoleID>_Data_<AcqID1>_<AcqID2>_<YYYYMMDD>_<HHMMSS>. Per-image XML holds beam-shift (critical for AFIS grouping), defocus, stage position, pixel size, dose, voltage, and detector mode. Multi-grid sessions (EPU Multigrid, up to 12 cartridges) produce one full session directory per grid.

SerialEM folder structure

SerialEM imposes minimal structure; the user controls naming and layout. Metadata lives in .mdoc files (IMOD autodoc: human-readable, bracketed sections, key/value pairs). 

<session_root>/
    GridName_LMM.st        GridName_LMM.st.mdoc    # low-mag atlas
    GridName_MMM.st        GridName_MMM.st.mdoc    # medium-mag montage
    GridName.mrc           GridName.mrc.mdoc       # acquired stack
    GridName.nav                                   # Navigator
    Frames.mdoc                                    # global frame metadata
    log.txt   SerialEMsettings.txt
    frames/
        GridName_0001_Dec23_13.20.21.tif
        GridName_0001_Dec23_13.20.21.tif.mdoc

Key keys inside an .mdoc: TiltAngle, StagePosition, Magnification, Defocus, ImageShift, ExposureDose, PixelSpacing, SubFramePath, NumSubFrames, DateTime; montages also carry PieceCoordinates. The .nav file stores Navigator items: Type 0 (points), Type 1 (polygons), Type 2 (maps with MapFile, MapID, MapSection, and a scale matrix).

What SmartScan looks for

When you point SmartScan at a session root it auto-detects the acquisition software and walks the tree accordingly:

  • EPU detection -- presence of Metadata/EpuSession.dm or Images-Disc1/GridSquare_*/. SmartScan recursively globs Images-Disc1/GridSquare_*/Data/*_Fractions.{mrc,tiff,eer} for movies, pairs each with its XML sibling, and parses beam-shift to reconstruct AFIS exposure groups. The atlas montage is read from Atlas/Supervisor_*/Atlas.mrc; per-square overviews come from GridSquare_*.mrc.
  • SerialEM detection -- presence of a .nav file or any *.mdoc in the root. SmartScan ingests the Navigator to build the grid map, then reads frames/*.mdoc (or the global Frames.mdoc) for per-tilt parameters. For tilt series, the stack .mrc.mdoc is the primary source of TiltAngle, dose, and SubFramePath.
  • Gain references -- *.gain at the EPU session root, or whatever path is recorded in the SerialEM .mdoc/log.
  • Multi-grid -- both formats are handled per-grid; SmartScan creates one CryoNAV Grid record per detected session or per autoloader-labelled subdirectory.

Practical tips: do not rename or flatten EPU directories before import -- SmartScan relies on the GridSquare -> FoilHoles -> Data hierarchy to reconstruct grid topology. For SerialEM, keep the frames/ directory adjacent to the .nav and .mdoc files so SubFramePath resolves correctly.

See also

Retrieved from "https://home.cryonav.org/index.php?title=CryoNAV_EPU_SerialEM_Folder_Structures&oldid=105"