Data Storage

Everything SeqDesk does with sequencing files hangs off one setting: the sequencing data directory. It is the root that file discovery scans, the root that uploads are written into, and the boundary that every file path is validated against. Until it is set, sequencing files cannot be discovered, linked to samples, uploaded, or fed to a pipeline — which is why the sidebar’s Infrastructure entry flags a missing data path as required.

The screen is Data Storage (/admin/data-storage). It has no sidebar entry of its own; reach it from Settings → Infrastructure, or from the Data Storage button on Pipeline Runtime.

Configure it from the server shell

The local SeqDesk command is the shortest post-install path. The installer creates $HOME/seqdesk/data as a writable starting directory; replace it with the existing absolute directory used by your facility when appropriate:

seqdesk storage configure "$HOME/seqdesk/data"
seqdesk storage status

configure rejects relative paths, the filesystem root, files, unreadable directories, and missing directories. It does not copy or move existing sequencing data. Use --create only when a deliberately selected local directory does not exist:

seqdesk storage configure /srv/seqdesk-data --create

The command updates the active settings.json value and the database SiteSettings.dataBasePath together, then reports the effective source. storage status exits non-zero until the configured directory is usable, so it can also gate deployment automation:

seqdesk storage configure /srv/seqdesk-data --yes --json
seqdesk storage status --json

The installer normally records the selected installation, so --dir is not needed. On a host with multiple installations, select one explicitly:

seqdesk storage status --dir /opt/seqdesk/app

A service-level SEQDESK_DATA_PATH remains authoritative. Change or remove that environment variable and restart the service instead of expecting either the CLI or Admin page to override it.

How the path is resolved

The value shown in the UI is not necessarily the value the application uses. SeqDesk resolves the directory in this order and stops at the first hit:

Setting it up in Admin

What Test Path actually checks

It is a cheap sanity check, not a preview of discovery:

• the path exists and is a directory
• the process can read it (R_OK)
• how many files sit directly in the root of that directory, and how many of those match the allowed extensions

Failures are reported verbatim: Directory does not exist or is not accessible, Directory is not readable (permission denied), or Path exists but is not a directory.

What SeqDesk writes underneath the path

The directory is not read-only. Plan permissions and backups accordingly.

Workbench explicitly asserts write access at start-up and fails with Data base path is not configured. Set it in Admin > Infrastructure before starting Workbench imports. when the path is missing. Filenames written by SeqDesk are sanitised — anything outside A–Z a–z 0–9 . _ - becomes _, and each path segment is capped at 120 characters.

Deletion is possible, and it is permanent

Facility admins can bulk-delete sequencing files through POST /api/files/delete. For each path the endpoint validates that it stays inside the base directory, clears or removes the Read records that reference it, then removes the file from disk with fs.rm. There is no trash, no soft delete, and no undo. Sequencing data is only as safe as your backups.

How discovery scans

Discovery is driven by five settings. Only two of them are editable in the admin UI; the rest come from settings.json or the environment.

The Advanced Configuration block is collapsed behind a Show button and contains only the extensions field. Entries are trimmed, de-duplicated, and get a leading dot added if you omit one, so typing fastq.gz, fq.gz is fine.

Scan depth

Depth counts the base directory as level 1:

A value outside 1–10 is rejected by config validation with sequencingFiles.scanDepth must be a number between 1 and 10. Deeper scans touch more of the filesystem, so raise it only as far as your layout needs.

Ignore patterns

Simple globs matched case-insensitively against the path relative to the base: ** matches any run of characters, * matches within one path segment. The defaults skip **/tmp/** and **/undetermined/**, which keeps demultiplexing leftovers out of the sample-linking UI.

Three behaviours that surprise people

• Files younger than 30 seconds are skipped. Anything whose modification time is within the active-write window is left alone so a half-written file from a running sequencer is never linked to a sample. Rescan a moment later and it appears.
• Scan results are cached for 5 minutes. Repeated scans of the same base path, extensions, depth and ignore patterns are served from an in-memory cache. Newly copied files can take up to five minutes to show up unless the caller forces a refresh. The cache is per-process and is cleared by a restart.

Single-end reads are always included: allowSingleEnd is forced to true when the setting is read and when it is written, so it cannot be turned off from the UI, the API, or the config file.

Changing the path after data exists

This is the one genuinely risky operation on this page.

SeqDesk stores file references as paths relative to the base directory. So:

• Moving the whole tree to a new mount point and updating the path keeps every reference valid, because the relative paths are unchanged.
• Pointing at a different tree silently invalidates every stored reference. Samples keep their Read rows, but the files behind them no longer resolve; downloads, previews, checksums and pipeline launches fail for existing data.

Before repointing:

Security model

Path handling is deliberately conservative:

• Every relative path is joined through a helper that rejects absolute paths and any path containing .., then verifies the resolved result is inside the base directory. Escaping raises Path traversal detected: … escapes base path.
• File-serving endpoints only serve paths with an allowed extension.
• Symlinks are not traversed, so a link cannot be used to reach outside the base.
• The delete endpoint applies the same containment check before removing anything.

What that model does not give you: per-user file permissions. Any facility admin can read, download and delete anything under the base path, and with department sharing enabled a researcher’s list widens to their department. Use filesystem permissions and a dedicated service account if you need stronger isolation.

Troubleshooting

See also

• Sequencing Files — how discovered files are linked to samples
• Pipeline Runtime — the run directory, which is a separate path
• Configuration Sources — which layer wins
• Environment Variables — the full SEQDESK_FILES_* reference