Installing & Setting Up Pipelines

SeqDesk installs the core application first. Pipelines are added separately, so a facility can see what is supported and install only the workflows it actually intends to run. You do not need to reinstall the application when you add a pipeline.

There are four separate parts:

Application — SeqDesk itself, PostgreSQL, accounts, and the web interface.
Package — the versioned Nextflow workflow and its SeqDesk contract.
Setup — runtime tools, required configuration, reference databases, and readable local paths.
Activation — whether facility administrators may start the pipeline.

Keeping those states separate is deliberate. A package can install successfully while remaining disabled until its environment is complete. SeqDesk never calls an incomplete pipeline “ready”.

Fast path

Run these commands on the SeqDesk host as the operating-system user that owns the installation:


seqdesk pipelines list
seqdesk pipelines list --catalog order
seqdesk pipelines list --catalog study
seqdesk pipelines install simulate-reads
seqdesk pipelines status simulate-reads

install offers to provision the managed Conda, Java, and Nextflow runtime when it is missing. To request that setup without waiting for the prompt:


seqdesk pipelines install simulate-reads --runtime

If all readiness checks pass, SeqDesk enables the pipeline automatically. If anything is missing, the package stays installed but disabled and the command prints the exact next action.

The installer creates the user command at ~/.local/bin/seqdesk and remembers the selected installation. If the command is not found, use $HOME/.local/bin/seqdesk, add that directory to PATH, or pass --dir "$HOME/seqdesk". The installer prints the appropriate PATH command when it is needed.

Discovering the catalog

The unfiltered catalog includes bundled packages, packages installed from the Pipeline Store, and packages currently available to install:


seqdesk pipelines list

Filter it by the surface on which a run starts:


# Runs from a sequencing order, usually once per sample
seqdesk pipelines list --catalog order
 
# Runs once for a study, across samples from one or more orders
seqdesk pipelines list --catalog study

Or filter by operational state:


seqdesk pipelines list --installed
seqdesk pipelines list --enabled

The human-readable list has four compact columns:

Column	Meaning
`PIPELINE`	Stable package id used in commands, for example `simulate-reads`
`USE WITH`	`Order`, `Study`, or both
`PACKAGE`	`Available`, `Built in`, or `Installed`
`STATE`	Readiness and activation combined, for example `Ready` or `Disabled · needs runtime`

When work remains, the exact next command is printed on an indented line below that pipeline instead of widening the table. On a narrow terminal the same information switches to cards. Colors appear only in an interactive terminal; piped output and --json stay automation-safe.

Package states:

State	Meaning
`Available` (`available`)	Offered by a configured Store source but not present locally
`Built in` (`bundled`)	Already included in the installed SeqDesk release
`Installed` (`installed`)	Added or updated through the Pipeline Store

Setup states:

State	Meaning
`ready`	Every blocking check passed
`not-installed` / `needs-package`	The package or its workflow files are missing or invalid
`needs-runtime`	Nextflow, Java, Conda, the selected SLURM runtime, or the run directory is not ready
`needs-config`	A pipeline-specific required value is missing
`needs-db`	A required reference database or database path is missing
`needs-storage`	The SeqDesk data path is missing or unreadable
`needs-attention`	Another pipeline-specific blocking check failed

ACTIVE is independent of SETUP. This makes it possible to keep a completely configured package intentionally disabled.

What installation does


seqdesk pipelines install <pipeline-id>

The command:

resolves the package from the configured Store sources or the bundled catalog;
downloads it to a staging area when necessary;
verifies the selected version, package contract, and checksum when one was supplied;
activates the package atomically, without exposing a half-written directory;
runs the pipeline-specific readiness checks;
offers guided runtime setup when runtime tools are the remaining blocker;
enables the pipeline only when the final readiness result allows it.

Installing the same version again is safe and does not create a second copy. If an update fails validation or activation, SeqDesk keeps or restores the last working package. The failed candidate is not marked ready.

When several Store sources publish the same id, select one explicitly:


seqdesk pipelines install <pipeline-id> --source <source-id>

Operators can also pin the expected package metadata:


seqdesk pipelines install <pipeline-id> \
  --source <source-id> \
  --version <version> \
  --sha256 <sha256-digest>

Private sources read credentials from the environment rather than command history:

SEQDESK_PIPELINE_ACCESS_KEY for a Store access key;
SEQDESK_PIPELINE_GITHUB_TOKEN, falling back to GITHUB_TOKEN, for a private GitHub release.

Do not put those values into repository files, screenshots, or copied support logs.

What “ready” verifies

Readiness is more than “the package directory exists”. SeqDesk combines common checks with requirements declared by that particular package:

the workflow entry point and package metadata are valid;
Nextflow and Java are available;
Conda is usable for the generated Conda profile, including SLURM runs;
SLURM commands and the selected execution policy are usable when the pipeline is configured for SLURM;
the pipeline run directory is writable;
the configured sequencing-data path exists and is readable;
required pipeline values are present and valid;
required reference databases and local assets are installed or linked;
pipeline-specific compatibility checks pass.

Inspect the result at any time:


seqdesk pipelines status <pipeline-id>

The status output names every blocking item and follows it with concrete commands or Admin locations. seqdesk pipelines enable <pipeline-id> is available for an intentionally disabled, fully ready package; it refuses to turn missing prerequisites into a false-ready state.

Guided setup

Set up the managed runtime

If Nextflow, Java, or Conda is missing, let SeqDesk provision its managed runtime:


seqdesk pipelines setup <pipeline-id> --runtime

The setup preserves the installation’s configured data and run paths. If your facility already manages those tools, configure their paths under Admin → Pipeline Runtime instead. For SLURM, the configured Conda base and environment must also be reachable from the compute nodes.

Supply pipeline-specific configuration

Put non-secret pipeline values in a JSON object and merge them into the pipeline configuration:


seqdesk pipelines setup <pipeline-id> \
  --config-file ./pipeline-config.json

For one small change, JSON can be passed directly:


seqdesk pipelines setup <pipeline-id> \
  --config-json '{"exampleSetting":"value"}'

Use the pipeline’s Settings view in Admin → Pipelines when you want schema-guided fields and descriptions. Do not place credentials in a shell argument on a shared host.

Install or link reference databases

Database assets are never downloaded silently. Their size, license, target filesystem, and update policy require an operator decision. When status reports needs-db, open Admin → Pipelines → pipeline → Databases, review the requirement, and install or link the asset there.

Configure local paths

When status reports needs-storage, configure and verify an existing readable sequencing data directory:


seqdesk storage configure /absolute/path/to/sequencing-data
seqdesk storage status

Use --create only for an intentionally new local directory, or configure the same path under Admin → Data Storage. A missing or unwritable pipeline run directory is reported as needs-runtime and is configured under Admin → Pipeline Runtime.

Recheck and activate

Run:


seqdesk pipelines status <pipeline-id>

setup automatically enables the package when the new result is ready. If a ready package was deliberately left disabled, enable it explicitly:


seqdesk pipelines enable <pipeline-id>

Safe first example: Simulate Reads

Simulate Reads is a small order-scoped Nextflow utility that can generate synthetic FASTQ files or replay configured facility templates. In explicit synthetic mode it requires no existing reads and no reference database, so it is useful for proving that package installation, runtime setup, execution, output discovery, and read writeback work.

Test data only. Simulate Reads does not produce scientific sequencing data. Its default replaceExisting: true setting replaces reads already linked to the selected samples. Use a new, dedicated demo order with no real files, or explicitly set replaceExisting to false.

Install and verify the package


seqdesk pipelines install simulate-reads --runtime
seqdesk pipelines status simulate-reads

The final status should show a green ✓ USABLE NOW heading and green checks for Installed, Setup ready, and Enabled. SeqDesk also confirms that the pipeline applies to sequencing orders. A pipeline that cannot be used instead shows a red ✗ NOT USABLE NOW heading, the failed check, and the exact next command.

Create an isolated demo order

In SeqDesk, create a new sequencing order used only for this test and add at least one sample. Do not attach real FASTQ files to that order.

Run from the order

Open the order, choose Analysis, select Simulate Reads, review the settings, and start it locally. For a safe, deterministic demo, choose Synthetic under Simulation Source and turn off Replace Existing Reads before starting.

An operator who already knows the internal order id can run the same test from the shell:


seqdesk pipeline run simulate-reads \
  --order "<test-order-id>" \
  --execution local \
  --config-json '{"simulationMode":"synthetic","readCount":100,"replaceExisting":false}' \
  --watch

The singular seqdesk pipeline form is used for runs. The plural seqdesk pipelines form is the primary interface for catalog, installation, setup, and activation.

Check the result

The run produces a small paired-end FASTQ set per sample and a simulation summary. SeqDesk discovers those outputs and links the generated files back to the sample’s Read record. Review the run output before using it as input to another demo pipeline such as FASTQ Checksum or FastQC.

Automation and unattended setup

All catalog and management commands can return structured output:


seqdesk pipelines list --json
seqdesk pipelines status simulate-reads --json
seqdesk pipelines install simulate-reads --runtime --yes --json

Use --runtime to make runtime provisioning explicit and --yes only when the host is already approved for that change. A JSON success response can still say that setup is incomplete; automation should check the returned package, setup, and activation states rather than only the process exit code.

For installations that are not the recorded default, pass the root explicitly:


seqdesk pipelines list --dir /srv/seqdesk

Resolution order is: explicit --dir, SEQDESK_DIR, the installer’s default pointer, a recognizable SeqDesk directory at or above the current directory, then $HOME/seqdesk.

Troubleshooting

`seqdesk: command not found`

The installer does not modify shell startup files. Use:


export PATH="$HOME/.local/bin:$PATH"
seqdesk pipelines list

Or call $HOME/.local/bin/seqdesk directly. An installation created before the user wrapper existed can still be targeted with the matching npm launcher and an explicit directory:


npx -y seqdesk@latest pipelines list --dir "$HOME/seqdesk"

Update SeqDesk before expecting a newer launcher to expose commands that the installed application release does not contain.

The package installed, but it is disabled

This is a successful package installation with incomplete setup, not a broken install. Run:


seqdesk pipelines status <pipeline-id>

Follow the first reported action, then recheck. Common causes are a missing runtime, an unset required value, a reference database requiring operator approval, or a local path that does not exist on the SeqDesk host.

Runtime setup was skipped

Resume it without reinstalling the package:


seqdesk pipelines setup <pipeline-id> --runtime

The wrong installation was selected

Pass --dir explicitly or set SEQDESK_DIR for that command. The command reads the selected installation’s local configuration and database directly.

Security boundary

Pipeline management commands do not use a browser session. Local shell access to the SeqDesk installation is treated as operator access because its configuration already contains database credentials and controls executable workflow packages. Restrict ownership and permissions on the install directory, the default-install pointer, pipeline source credentials, data paths, and run directories accordingly.