Submitting to ENA

Both ENA paths start from the same place: open the study, go to the Publishing tab. You get two cards.

• Register at ENA — “Register and publish study metadata to the European Nucleotide Archive”. Carries a status badge: Draft, Ready, Test Registered, Test Expired, Partial, or Registered.
• Submit to ENA — “Submit reads, assemblies, and bins to ENA using SubMG”. Only visible to facility admins, and only when the study has at least one sample.

Everything on this page requires the FACILITY_ADMIN role. Demo accounts see the screens but every action is disabled.

Registering the study and samples

The test-first gate

SeqDesk deliberately makes it hard to publish to Production by accident. The Register at ENA screen has a Test / Production toggle, and the button below it changes with the toggle.

Register at Test Server is disabled when:

• any of the five submission requirements fails;
• the study already has a production accession but is not yet marked submitted — re-registering on Test would overwrite the stored production accession;
• the study already has an unexpired test registration and every sample already has an accession, so there is nothing left to send.

Register at Production is disabled unless all three of these hold:

Hovering a disabled button explains which condition is blocking it. The practical effect is that you cannot reach Production without having proved the same study, with the same metadata, against ENA’s real validator within the last day.

What happens when you click

Where the accessions land

A test registration writes a real-looking PRJEB… into studyAccessionId but leaves submitted false, which is why the study badge reads Test Registered rather than Registered.

The XML SeqDesk generates

Study (PROJECT):

<?xml version="1.0" encoding="UTF-8"?>
<PROJECT_SET>
  <PROJECT alias="human-gut-shotgun-prjeb54724">
    <TITLE>Human Gut Shotgun Metagenomes (PRJEB54724)</TITLE>
    <DESCRIPTION>Twelve paired-end Illumina shotgun metagenomes...</DESCRIPTION>
    <SUBMISSION_PROJECT>
      <SEQUENCING_PROJECT/>
    </SUBMISSION_PROJECT>
  </PROJECT>
</PROJECT_SET>

Samples, with the study’s checklist type emitted first as ENA-CHECKLIST and the merged attributes after it:

<?xml version="1.0" encoding="UTF-8"?>
<SAMPLE_SET>
  <SAMPLE alias="HGM-01">
    <TITLE>Human faecal shotgun metagenome Subject-01</TITLE>
    <SAMPLE_NAME>
      <TAXON_ID>408170</TAXON_ID>
      <SCIENTIFIC_NAME>human gut metagenome</SCIENTIFIC_NAME>
    </SAMPLE_NAME>
    <SAMPLE_ATTRIBUTES>
      <SAMPLE_ATTRIBUTE>
        <TAG>ENA-CHECKLIST</TAG>
        <VALUE>host-associated</VALUE>
      </SAMPLE_ATTRIBUTE>
      <SAMPLE_ATTRIBUTE>
        <TAG>collection date</TAG>
        <VALUE>2016-05-12</VALUE>
      </SAMPLE_ATTRIBUTE>
      <SAMPLE_ATTRIBUTE>
        <TAG>geographic location (country and/or sea)</TAG>
        <VALUE>Netherlands</VALUE>
      </SAMPLE_ATTRIBUTE>
    </SAMPLE_ATTRIBUTES>
  </SAMPLE>
</SAMPLE_SET>

The SAMPLE alias is the sample’s sampleId, and the title falls back to the sampleId when sampleTitle is empty. The SCIENTIFIC_NAME element is omitted entirely when the sample has no scientific name.

Wrapper:

<?xml version="1.0" encoding="UTF-8"?>
<SUBMISSION>
  <ACTIONS>
    <ACTION>
      <ADD/>
    </ACTION>
  </ACTIONS>
</SUBMISSION>

Endpoints and authentication

HTTP Basic auth with the Webin credentials, multipart/form-data with the XML documents as file parts. Your SeqDesk host needs outbound HTTPS to ebi.ac.uk.

The connection test

Test connection in ENA settings registers nothing. It POSTs a throwaway project (alias test-connection-<timestamp>) wrapped in a VALIDATE action rather than ADD. Any receipt containing success="true" or success="false" counts as a pass, because either proves the credentials authenticated — a VALIDATE that fails on the dummy data is still a successful login. A 401 fails the test. The username is checked against ^Webin-\d+$ before any network call is made.

The Register at ENA screen runs this same check automatically when a facility admin opens it, which is why the button can briefly read Checking ENA… and may turn into Set ENA credentials.

Broker-account mode

If the installation is configured as an ENA broker (brokerAccount enabled with a center name in ENA settings), the registration path adds a center_name attribute to both the PROJECT and every SAMPLE element, so records are attributed to the brokering center. Broker mode applies to XML registration only — the submg path does not inject a center name.

Concurrency

A study can have only one registration in flight. The API takes a PostgreSQL advisory lock keyed on the study and rejects a second attempt with A submission for this study is already in progress while a PENDING or SUBMITTED submission exists. If a row is stuck in PENDING because the server died mid-request, delete it from the Submissions Dashboard before retrying.

Re-running a registration

There is no per-sample retry. Fix the underlying records and run the whole registration again — from the study page, or with Retry Submission on the dashboard, which re-POSTs the same study against the same target.

SeqDesk regenerates fresh XML every time, so database fixes take effect immediately. When the study already has an accession for that target, the project call is skipped and only the samples still missing an accession are submitted.

Submitting data files with submg

The registration path publishes metadata. To publish the actual sequencing data, SeqDesk runs submg — a wrapper around the submg  CLI, which in turn drives ENA’s webin-cli. It appears in the pipeline catalog as Submit to ENA, category submission.

It is a normal pipeline run: it appears in the run list, writes logs and artifacts, and can be scoped to a whole study or a subset of samples. Researchers can see it but cannot start it — the package declares userCanStart: false.

Which server it submits to

Unlike registration, submg does not offer a per-run toggle. It reads the global enaTestMode setting and passes --development-service 1 (test) or 0 (production) to submg. Check the ENA setting in Administration → ENA Credentials before you start a run — the pipeline configuration screen will not tell you which archive you are about to write to.

What the runner builds

For each selected sample the runner writes a YAML manifest into the run folder and a shell script that calls submg once per sample. The manifest carries:

• STUDY — the study’s PRJ… accession, and PROJECT_NAME — the study title.
• METAGENOME_TAXID and METAGENOME_SCIENTIFIC_NAME from the sample (scientificName defaults to metagenome when unset).
• NEW_SAMPLES with the sample title, collection date, geographic location (country and/or sea), and every other scalar checklist field under ADDITIONAL_SAMPLESHEET_FIELDS.
• PAIRED_END_READS — one entry per paired read, with the FASTQ paths, LIBRARY_SOURCE/LIBRARY_SELECTION/LIBRARY_STRATEGY from the sequencing order (defaulting to METAGENOMIC / RANDOM / WGS), INSERT_SIZE (default 300) and SEQUENCING_INSTRUMENT.
• ASSEMBLY — the resolved FASTA path, ASSEMBLY_SOFTWARE (default MEGAHIT) and the resolved COVERAGE_VALUE.
• BINS — only when bins exist on disk, with generated CheckM-style quality and taxonomy TSVs alongside.

The generated script then runs, per sample:

submg-cli submit --config <manifest>.yaml \
  --staging-dir … --logging-dir … \
  --development-service 1 \
  --submit-samples --submit-reads --submit-assembly \
  [--submit-bins] [--skip-checks]

--submit-bins is added only when that sample has usable bin files. --skip-checks is added when the skipChecks option is on — and it is on by default in the package configuration, on the assumption that SeqDesk’s own validation has already covered the same ground. Turn it off if you want submg’s independent pre-flight checks as a second opinion.

Gzipped FASTQs are decompressed into a per-run scratch directory before submg sees them, and that directory is deleted afterwards.

Configurable options

What comes back

When the run finishes, SeqDesk walks the submg logging directories and parses:

• sample_preliminary_accessions.txt under biological_samples/ — a tab-separated alias / accession / external-accession table. Both sampleAccessionNumber (ERS…) and biosampleNumber (SAMEA…) are written to the sample. This is the only path that populates biosampleNumber.
• webin-cli.report under each reads_* directory — run and experiment accessions, matched back to the right Read row by comparing the checksums in the emitted run.xml against the stored MD5s.
• webin-cli.report under assembly_fasta/ — the ERZ… analysis accession.
• bin_to_preliminary_accession.tsv under bins/ — per-bin accessions.

The run’s results field records samplesUpdated, readsUpdated, assembliesUpdated, binsUpdated, artifactsCreated and any warnings. A run that reports zero updates but no errors means submg ran and SeqDesk could not find or map its receipts — read the warnings and the run artifacts before assuming nothing was submitted.

Test versus Production, side by side

Where to go next

• Submissions Dashboard — inspect, retry, and clean up submissions.
• Troubleshooting ENA — the errors both paths produce.
• Running a Pipeline — how pipeline runs work in general, including execution settings for submg.