added explanation of performance issues on osxfs from @dsheets post

[londoncalling]

Proposed changes

Added topic on "Performance issues, solutions, and roadmap".

Related issues

issue: https://github.com/docker/pinata/issues/5564

forum topic: https://forums.docker.com/t/file-access-in-mounted-volumes-extremely-slow-cpu-bound/

Please take a look

@samoht @dsheets @justincormack @thaJeztah

Signed-off-by: Victoria Bialas [email protected]

[dsheets]

LGTM. I just have a few small comments. :-) Thanks!

[dsheets]

We wanted to surface it in the documentation for wider reach.

reads awkwardly but is probably fine. The text is in the present with the reader.

[dsheets]

s/I am personally/We are/

[dsheets]

s/kernel module/kernel patch/

[londoncalling]

Great @dsheets , thanks.

On Tuesday, November 1, 2016, David Sheets [email protected] wrote:

@dsheets commented on this pull request.

LGTM. I just have a few small comments. :-) Thanks!

In docker-for-mac/osxfs.md
#379 (review)
:

+### Performance issues, solutions, and roadmap
+
+With regard to reported performance issues (GitHub issue 77: File access in mounted
+volumes extremely slow), and a
+similar thread on Docker for Mac forums on topic: File access in mounted volumes
+extremely
+slow,
+this topic provides an explanation of the issues, what we are doing about it,
+how the community can help us, and what you can expect in the future.
+This explanation is a slightly re-worked version of an understanding
+performance
+post
+from David Sheets (@dsheets) on the Docker development
+team to the forum topic just
+mentioned. We wanted to surface it in the documentation for wider reach.

We wanted to surface it in the documentation for wider reach.

reads awkwardly but is probably fine. The text is in the present with the

reader.

In docker-for-mac/osxfs.md
#379 (review)
:

+so must be invalidated if OS X ever reports an event delivery failure.
+
+2. Running ember build in a shared file system results in ember creating many
+different temporary directories and performing lots of intermediate activity
+within them. An empty ember project is over 300MB. This usage pattern does not
+require coherence between Linux and OS X but, because we cannot distinguish this
+fact at run-time, we maintain coherence during its hundreds of thousands of file
+system accesses to manipulate temporary state. There is no "correct" solution in
+this case. Either ember needs to change, the volume mount needs to have
+coherence properties specified on it somehow, some heuristic needs to be
+introduced to detect this access pattern and compensate, or the behavior needs
+to be indicated via, e.g., extended attributes in the OS X file system.
+
+These two examples come from performance use cases contributed by users and they
+are incredibly helpful in prioritizing aspects of file system performance to
+improve. I am personally developing statistical file system trace analysis tools

s/I am personally/We are/

In docker-for-mac/osxfs.md
#379 (review)
:

+fact at run-time, we maintain coherence during its hundreds of thousands of file
+system accesses to manipulate temporary state. There is no "correct" solution in
+this case. Either ember needs to change, the volume mount needs to have
+coherence properties specified on it somehow, some heuristic needs to be
+introduced to detect this access pattern and compensate, or the behavior needs
+to be indicated via, e.g., extended attributes in the OS X file system.
+
+These two examples come from performance use cases contributed by users and they
+are incredibly helpful in prioritizing aspects of file system performance to
+improve. I am personally developing statistical file system trace analysis tools
+to characterize slow-performing workloads more easily in order to decide what to
+work on next.
+
+Under development, we have:
+
+1. A Linux kernel module to reduce data path latency by 2/7 copies and 2/5

s/kernel module/kernel patch/

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#379 (review),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ALHuA3nhUAY3SK5PQlv8Ss_vaPZrgJeKks5q5y2CgaJpZM4Kj-Am
.

[londoncalling]

@dsheets @thaJeztah @MagnusS @dave-tucker @mstanleyjones @johndmulhausen Can someone give this a final review and approve my changes so I can merge? Thanks!

[londoncalling]

@dsheets :-) how about now? merge if you think it's good, thanks!

[mdlinville]

LGTM