README.adoc

FTP Source

Messages emitted by the source are provided as a byte array by default. However, this can be customized using the --mode option:

• ref Provides a java.io.File reference

• lines Will split files line-by-line and emit a new message for each line

• contents The default. Provides the contents of a file as a byte array

When using --mode=lines, you can also provide the additional option --withMarkers=true. If set to true, the underlying FileSplitter will emit additional start-of-file and end-of-file marker messages before and after the actual data. The payload of these 2 additional marker messages is of type FileSplitter.FileMarker. The option withMarkers defaults to false if not explicitly set.

See also MetadataStore options for possible shared persistent store configuration used to prevent duplicate messages on restart.

Input

N/A (Fetches files from an FTP server).

Output

mode = contents

Headers:

• Content-Type: application/octet-stream

• file_originalFile: <java.io.File>

• file_name: <file name>

Payload:

A byte[] filled with the file contents.

mode = lines

Headers:

• Content-Type: text/plain

• file_orginalFile: <java.io.File>

• file_name: <file name>

• correlationId: <UUID> (same for each line)

• sequenceNumber: <n>

• sequenceSize: 0 (number of lines is not know until the file is read)

Payload:

A String for each line.

The first line is optionally preceded by a message with a START marker payload. The last line is optionally followed by a message with an END marker payload.

Marker presence and format are determined by the with-markers and markers-json properties.

mode = ref

Headers:

None.

Payload:

A java.io.File object.

Options

The ftp source has the following options:

See Spring Functions Catalog for configuration options.

Examples

java -jar ftp_source.jar --ftp.supplier.remote-dir=foo --file.consumer.mode=lines --ftp.factory.host=ftpserver \
         --ftp.factory.username=user --ftp.factory.password=pw --ftp.local-dir=/foo