| Source | DNAnexus path |
|---|---|
| Existing apps/applets assembled visually or by API | Native workflow |
| WDL draft-2, 1.0, or 1.1 | dxCompiler |
| CWL 1.2 | dxCompiler |
| Nextflow source folder/repository | dx build --nextflow |
| One Python/Bash program | Applet/app, not a workflow |
Keep the source workflow, compiler/importer version, container digests, input JSON, and generated executable IDs together as provenance.
Native workflows connect app/applet stages. Build from a dxworkflow.json:
dx build --workflow "path/to/workflow-source" \
--destination "project-xxxx:/workflows/my-workflow"
Inspect inputs:
dx run "workflow-xxxx" -h
Run:
dx run "workflow-xxxx" \
--input-json-file "inputs.json" \
--destination "project-xxxx:/runs/run-001" \
--cost-limit 50
This returns an analysis-... ID. Each stage runs as a job or nested analysis.
Use native workflows when:
Promote stable inputs/outputs to workflow-level fields. Keep project-qualified default links only when the referenced data is deliberately shared and accessible in every intended context.
Current baseline: dxCompiler 2.17.0.
Supported language versions at that baseline:
WDL 2.0/development support is not production-ready. CWL 1.0 and 1.1 must be upgraded to 1.2 before compilation.
Prerequisites:
dx toolkit and authenticated project contextPin the JAR release and verify release checksums or provenance before use. Do not use an unversioned download URL in production automation.
dxCompiler uses strict WDL parsing. Validate and lint with the matching wdlTools release before compilation.
Then compile:
java -jar "dxCompiler-2.17.0.jar" compile \
"workflow.wdl" \
-project "project-xxxx" \
-folder "/workflows" \
-inputs "inputs.json"
The compile command can create a DNAnexus-formatted input JSON. Run the
compiled workflow with dx run and the generated input file:
dx run "workflow-xxxx" \
--input-json-file "inputs.dx.json" \
--destination "project-xxxx:/runs/run-002" \
--cost-limit 50
Run java -jar dxCompiler-2.17.0.jar help for the exact options supported by
that pinned release.
dxCompiler 2.17.0 supports per-domain bearer tokens for protected HTTP(S) WDL
imports through DXCOMPILER_WDL_IMPORT_BEARER_TOKENS.
This variable is highly sensitive:
Prefer immutable source references and checksum imported WDL dependencies.
dxCompiler expects one packed CWL 1.2 document.
cwltool --validate.cwltool --validate "workflow.cwl.json"
java -jar "dxCompiler-2.17.0.jar" compile \
"workflow.cwl.json" \
-project "project-xxxx" \
-folder "/workflows"
Pin cwltool, the upgrader/packer, and dxCompiler in reproducible
environments.
Review the current dxCompiler README and release notes before migration. At the documented baseline, relevant limitations include:
dxni is not supported.Do not promise byte-identical behavior to Cromwell or another CWL runner without conformance tests.
Creating a DNAnexus app/applet from Nextflow source requires a DNAnexus license.
Current dx-toolkit supports Nextflow engine versions 25.10 and 24.10;
25.10 is the current default and 24.10 is retained only as an unmaintained
compatibility option.
dx build --nextflow \
--repository "https://github.com/nextflow-io/hello" \
--repository-tag "<pinned-tag-or-commit>" \
--destination "project-xxxx:/applets/hello"
Pin a repository tag or commit. Do not import a floating default branch for a production pipeline.
For a private repository, --git-credentials accepts a DNAnexus file. Use a
short-lived, read-only credential and store it in a project whose membership
has been reviewed. Anyone with sufficient project access may be able to read
that file.
dx build --nextflow "path/to/pipeline" \
--nextflow-version "25.10" \
--destination "project-xxxx:/applets/my-pipeline"
An nf-core-style layout is encouraged but not required.
dx run "applet-xxxx" -h
The generated executable exposes fields for Nextflow run options, configuration files, and a YAML/JSON parameter file. Use the generated help rather than assuming field names.
dx run "applet-xxxx" \
--input-json-file "nextflow-inputs.json" \
--destination "project-xxxx:/runs/run-003" \
--cost-limit 100
The execution is a head job supervising process subjobs. Monitor both:
dx watch "job-head"
dx watch "job-child"
dx find jobs --root-execution "job-head"
Check current CLI help if a filter name differs; dx-toolkit evolves.
DNAnexus Nextflow execution supports Docker as the container runtime. Pin containers by immutable digest.
Options include:
--cache-docker to cache image tarballs in the selected project--docker-secrets for a DNAnexus file containing private registry
credentialsRegistry credentials are readable data objects to users with sufficient project access. Use pull-only, short-lived credentials and review membership.
External image pulls require network access. Caching images improves reproducibility and reduces registry dependence.
Nextflow pipeline applets support resume/cache-preservation inputs. Current platform guidance limits preserved sessions in a project to 20. Clean old work directories and cache sessions deliberately to control storage costs.
Only one job can resume and preserve the same session at a time. Do not launch concurrent writes to one preserved session.
The head job orchestrates the pipeline; process subjobs perform scientific work. Size them separately:
cpus, memory, and disk directives for processes.DNAnexus documents using an S3 bucket as a Nextflow work directory through OIDC. This requires a reviewed cloud trust policy and narrow bucket permissions. Do not embed AWS credentials; use the DNAnexus job OIDC provider and least-privilege IAM role.
For any imported workflow:
Scientific equivalence matters more than successful compilation. Validate reference builds, coordinate systems, sort orders, floating-point tolerances, and tool versions.