Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
116 changes: 115 additions & 1 deletion docs/process-typed.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,8 @@ All {ref}`standard types <stdlib-types>` except for the dataflow types (`Channel

Nextflow automatically stages `Path` inputs and `Path` collections (such as `Set<Path>`) into the task directory.

### Nullable inputs

By default, tasks fail if any input receives a `null` value. To allow `null` values, add `?` to the type annotation:

```nextflow
Expand All @@ -85,10 +87,84 @@ process cat_opt {
}
```

### Stage directives
### Record inputs

Inputs with type `Record` can declare the name and type of each record field:

```nextflow
process fastqc {
input:
(id: String, fastq: Path): Record

script:
"""
echo 'id: ${id}`
echo 'fastq: ${fastq}'
"""
}
```

This pattern is called *record destructuring*. Each record field is staged into the task the same way as an individual input.

When the process is invoked, the incoming record should contain the specified fields, or else the run will fail. If the record has additional fields not declared by the process input, they are ignored.

:::{tip}
Record inputs are a useful way to select a subset of fields from a larger record. This way, the process only stages what it needs, allowing you to keep related data together in your workflow logic.
:::

### Record type inputs

Record inputs can also be declared using a custom record type:

```nextflow
process fastqc {
input:
sample: Sample

script:
"""
echo 'id: ${sample.id}`
echo 'fastq: ${sample.fastq}'
"""
}

record Sample {
id: String
fastq: Path
}
```

In this example, the record is staged into the task as `sample`, and `sample.fastq` is staged as an input file since the `fastq` field is declared with type `Path`.

When the process is invoked, the incoming record should contain the fields specified by the record type, or else the run will fail. If the record has additional fields not declared by the record type, they are ignored.

### Tuple inputs

Inputs with type `Tuple` can declare the name of each tuple component:

```nextflow
process fastqc {
input:
(id, fastq): Tuple<String,Path>

script:
"""
echo 'id: ${id}`
echo 'fastq: ${fastq}'
"""
}
```

This pattern is called *tuple destructuring*. Each tuple component is staged into the task the same way as an individual input.

The generic types inside the `Tuple<...>` annotation specify the type of each tuple compomnent and should match the component names. In the above example, `id` has type `String` and `fastq` has type `Path`.

## Stage directives

The `stage:` section defines custom staging behavior using *stage directives*. It should be specified after the `input:` section. These directives serve the same purpose as input qualifiers such as `env` and `stdin` in the legacy syntax.

### Environment variables

The `env` directive declares an environment variable in terms of task inputs:

```nextflow
Expand All @@ -106,6 +182,8 @@ process echo_env {
}
```

### Standard input (stdin)

The `stdin` directive defines the standard input of the task script:

```nextflow
Expand All @@ -123,6 +201,8 @@ process cat {
}
```

### Custom file staging

The `stageAs` directive stages an input file (or files) under a custom file pattern:

```nextflow
Expand Down Expand Up @@ -222,6 +302,40 @@ process foo {
}
```

### Structured outputs

Whereas legacy process outputs could only be structured using specific qualifiers like `val` and `tuple`, typed process outputs are regular values.

The `record()` standard library function can be used to create a record:

```nextflow
process fastqc {
input:
(id: String, fastq: Path): Record

output:
record(id: id, fastqc: file('fastqc_logs'))

script:
// ...
}
```

The `tuple()` standard library function can be used to create a tuple:

```nextflow
process fastqc {
input:
(id, fastq): Tuple<String,Path>

output:
tuple(id, file('fastqc_logs'))

script:
// ...
}
```

## Topics

The `topic:` section emits values to {ref}`topic channels <channel-topic>`. A topic emission consists of an output value and a topic name:
Expand Down
7 changes: 5 additions & 2 deletions docs/reference/stdlib-namespaces.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ The global namespace contains globally available constants and functions.
: Create a branch criteria to use with the {ref}`operator-branch` operator.

`env( name: String ) -> String`
: :::{versionadded} 24.11.0-edge
: :::{versionadded} 25.04.0
:::
: Get the value of the environment variable with the specified name in the Nextflow launch environment.

Expand Down Expand Up @@ -108,8 +108,11 @@ The global namespace contains globally available constants and functions.
`sleep( milliseconds: long )`
: Sleep for the given number of milliseconds.

`record( [options] ) -> Record`
: Create a record from the given named arguments.

`tuple( args... ) -> Tuple`
: Create a tuple object from the given arguments.
: Create a tuple from the given arguments.

(stdlib-namespaces-channel)=

Expand Down
32 changes: 32 additions & 0 deletions docs/reference/stdlib-types.md
Original file line number Diff line number Diff line change
Expand Up @@ -767,6 +767,38 @@ The following methods are available for splitting and counting the records in fi
`splitText() -> List<String>`
: Splits a text file into a list of lines. See the {ref}`operator-splittext` operator for available options.

(stdlib-types-record)=

## Record

A record is an immutable map of fields to values (i.e., `Map<String,?>`). Each value can have its own type.

A record can be created using the `record` function:

```nextflow
sample = record(id: '1', fastq_1: file('1_1.fastq'), fastq_2: file('1_2.fastq'))
```

Record fields can be accessed as properties:

```nextflow
sample.id
// -> '1'
```

The following operations are supported for records:

`+ : (Record, Record) -> Record`
: Given two records, returns a new record containing the fields and values of both records. When a field is present in both records, the value of the right-hand record takes precedence.

`- : (Record, Iterable<String>) -> Record`
: Given a record and a collection of strings, returns a copy of the record with the given fields removed.

The following methods are available for a record:

`subMap( keys: Iterable<String> ) -> Record`
: Returns a new record containing only the given fields.

(stdlib-types-set)=

## Set\<E\>
Expand Down
17 changes: 14 additions & 3 deletions docs/reference/syntax.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ A Nextflow script may contain the following top-level declarations:
- Process definitions
- Function definitions
- Enum types
- Record types
- Output block

Script declarations are in turn composed of statements and expressions.
Expand Down Expand Up @@ -107,6 +108,8 @@ The following definitions can be included:
- Functions
- Processes
- Named workflows
- *New in 26.04:* Enum types
- *New in 26.04:* Record types

### Params block

Expand Down Expand Up @@ -360,9 +363,17 @@ enum Day {

Enum values in the above example can be accessed as `Day.MONDAY`, `Day.TUESDAY`, and so on.

:::{note}
Enum types cannot be included across modules at this time.
:::
### Record type

A record type declaration consists of a name and a body. The body consists of one or more fields, where each field has a name and a type:

```nextflow
record FastqPair {
id: String
fastq_1: Path
fastq_2: Path
}
```

### Output block

Expand Down
30 changes: 30 additions & 0 deletions docs/script.md
Original file line number Diff line number Diff line change
Expand Up @@ -111,6 +111,36 @@ Copying a map with the `+` operator is a safer way to modify maps in Nextflow, s

See {ref}`stdlib-types-map` for the set of available map operations.

(script-records)=

## Records

Records are used to store a set of related fields, where each field can have its own type. They are created using the `record` function:

```nextflow
person = record(name: 'Alice', age: 42, is_male: false)
```

Record fields are accessed by name:

```nextflow
name = person.name
age = person.age
is_male = person.is_male
```

Records are immutable -- once a record is created, it cannot be modified. Use record operations to create new records instead.

For example:

```nextflow
person + record(age: 43) - ['is_male']

// record(name: 'Alice', age: 43)
```

See {ref}`stdlib-types-record` for the set of available record operations.

(script-tuples)=

## Tuples
Expand Down
32 changes: 24 additions & 8 deletions docs/strict-syntax.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,14 +50,29 @@ def json = new groovy.json.JsonSlurper().parseText(json_file.text)

Some users use classes in Nextflow to define helper functions or custom types. Helper functions should be defined as standalone functions in Nextflow. Custom types should be moved to the `lib` directory.

:::{note}
Enums, a special type of class, are supported, but they cannot be included across modules at this time.
:::
You can use an enum type to model a choice between a fixed set of categories:

:::{note}
Record types will be addressed in a future version of the Nextflow language specification.
```nextflow
enum Color {
RED,
GREEN,
BLUE
}
```

:::{versionadded} 26.04.0
:::

You can use a record type to model a composition of multiple values:

```nextflow
record FastqPair {
id: String
fastq_1: Path
fastq_2: Path
}
```

### Mixing script declarations and statements

In the strict syntax, a script may contain any of the following top-level declarations:
Expand Down Expand Up @@ -230,13 +245,14 @@ In the strict syntax, use `System.getenv()` instead:
println "PWD = ${System.getenv('PWD')}"
```

:::{versionadded} 24.04.0
The `env()` function should be used instead of `System.getenv()`:
:::{versionadded} 25.04.0
:::

Use the `env()` function instead of `System.getenv()`:

```nextflow
println "PWD = ${env('PWD')}"
```
:::

## Restricted syntax

Expand Down
9 changes: 9 additions & 0 deletions modules/nextflow/src/main/groovy/nextflow/Nextflow.groovy
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ import nextflow.splitter.FastaSplitter
import nextflow.splitter.FastqSplitter
import nextflow.util.ArrayTuple
import nextflow.util.CacheHelper
import nextflow.util.RecordMap
import org.slf4j.Logger
import org.slf4j.LoggerFactory
/**
Expand Down Expand Up @@ -150,6 +151,14 @@ class Nextflow {
return result instanceof Collection ? result : [result]
}

/**
* Creates a {@link RecordMap} from the given named arguments.
*
* @param props
*/
static RecordMap record(Map<String,?> props) {
return new RecordMap(props)
}

/**
* Creates a {@link ArrayTuple} object with the given open array items
Expand Down
Loading
Loading