SFTP writes with 'append' mode may insert instead

[smitsjelle]

Version

2.14.0

Bug description

I'm using Spring Integration to write files to an SFTP server. I noticed that on one specific server I get odd behavior. Instead of adding the contents to the existing file, it overrides the content. It seems to be inserting the data from position 0 until it is done. If there was already more data in the file, that data remains.

For example, if I have a test.txt on my server with content abcdefghi and I send 123456 with setting APPEND, the resulting file content is 123456ghi instead of abcdefghi123456.

In order to do the append, Spring Integrations' SFTPSession#append(..) method uses AbstractSftpClient#write(String path, int bufferSize, Collection mode) to do appends. When I change Spring's method to use the SftpClient#write(Handle handle, long fileOffset, byte[] src) instead, where I pass the current length of the file as its offset (retrieved by calling sftpClient.stat(destination).getSize()), the append works as expected.

It feels like the write method of AbstractSftpClient doesn't handle the append correctly if the supplied OpenModes are Write, Append, Create. Is this correct, or is there something wrong with the parameters that are passed?

Actual behavior

The data gets inserted from position 0 instead of getting added after the last position in the document. I.e.: when the inserted data is less than the length of the existing document, it gets overriden until the new data ends, and remains the same after that.

Expected behavior

When the OpenMode 'Append' is provided, data is always added at the end of the file.

Relevant log output

Other information

The 'getVersion' command for the server I'm encountering the issue for returns version 3. In testcases where I cannot reproduce the issue, the version is 6. Unfortunately, I don't have any further details on the configuration of the SFTP server I'm encountering the issue on.

[tomaswolf]

So you have an SFTP v3 server that ignores the SSH_FXF_APPEND flag given in the SSH_FXP_OPEN message and that instead always writes at the offset given in the SSH_FXP_WRITE messages. What kind of server is that? Maybe the identification string sent when the connection is established shows something interesting. (Visible with debug logging in the client.)

Not sure we can do anything about it in the client.

[tomaswolf]

For instance OpenSSH can append only since version 6.3 (released 2013-09-13). Older OpenSSH versions would show the behavior you mentioned. Compare also hierynomus/sshj#390 (comment) .

[smitsjelle]

The 'doReadIdentification' reports 'Server version string: SSH-2.0-7.9.0.0_openssh'. Looks to me like a standard version, although it indeed is an older one. Yet I'd expect this version to the support for append protocol, right?

[tomaswolf]

I don't know. I have no idea what version "7.9.0.0_openssh" is. Not sure that would be OpenSSH 7.9. I find four digits a bit strange.

[tomaswolf]

As far as I can determine it looks like this is some GlobalScape SFTP server, so most likely running on Windows. No idea what version of OpenSSH they might be using, or how old that software reporting this version is.

SFTP can be run over OpenSSH as an external process, so even if this server used OpenSSH 7.9 it might still use a proprietary SFTP implementation. It also appears that on these servers, there is a separate permission setting to allow appending to files, but I have not found any information about what happens if a user doesn't have that permission and tries appending all the same. (Does one get a failure response? Or does it silently ignore the append flag?)

To me this very much looks like a server-side problem. Does appending to a remote file on a plain vanilla OpenSSH server work for you? (OpenSSH is always be SFTP v3.)

[smitsjelle]

I did a quick test with some OpenSSH SFTP server (from this Docker image) that reports doReadIdentification SSH-2.0-OpenSSH_8.4p1 Debian-5+deb11u3. For that one it does indeed work fine, which suggests that the issue might indeed be either server-specific or maybe somewhere in OpenSSH 7.9.

The reason why I'm not sure whether it is a server specific issue yet, is that this server was handling my append operations just fine when I used an older Spring Integration version that still used Jsch as SFTP library. Also, with a small change in the append code I seem to be able to do the append (see the code snippet below). Do you know if that is because it inherently does a write from rather than a more dynamic append, or would this still be considered a 'normal' append?

This snippet does produce working append behavior for this server:

SftpClient.CloseableHandle handle = this.sftpClient.open(
				destination,
				SftpClient.OpenMode.Create,
				SftpClient.OpenMode.Write,
				SftpClient.OpenMode.Append);
		this.sftpClient.write(handle, offset, inputStream.readAllBytes());

[tomaswolf]

What is offset in this snippet?

[smitsjelle]

Excuse me, missed that part.

Offset is the size, as fetched from the Attributes

SftpClient.Attributes attrs = this.sftpClient.stat(destination);
long offset = (attrs != null) ? attrs.getSize() : 0;

[tomaswolf]

Well, then it's clear why that worked: you're not making use of the append flag at all. I suspect this would also work if you didn't pass SftpClient.OpenMode.Append. You're writing at the current end of the file, and that's it.

So far this is consistent with having a "buggy server that ignores the append flag".

You mentioned that with JSch append with that server worked. As far as I see JSch gets the remote file size, and then explicitly sends SSH_FXF_WRITE requests with that as base offset.

Apache MINA sshd doesn't do this. SFTP v3 says that the append flag "forces all writes to append data at the end of the file", so the offset should just be ignored. Later drafts for SFTP v4/5/6 are more explicit and state "Data is always written at the end of the file. The offset field of SSH_FXP_WRITE requests is ignored."

So: still a buggy server IMO, but there is something we can do to try to work around this server issue. (At the cost of an additional stat remote call.)