Add support for Linux

[lewissbaker]

While most of cppcoro is platform agnostic, some of the thread-pooling and I/O code is OS-specific.

This issue is for providing an implemention of the cppcoro I/O abstractions for Linux.

Will need to get it building under Clang as a first step. See #3.
Then:

• Look into writing an epoll I/O event loop, based io_service implementation.
• Port file and socket (once written) interfaces to Linux.
• Add build-system support for Linux

[lewissbaker]

cppcoro is now at least buildable under Linux/Clang on the clang branch.

Still outstanding is the need to work on an implementation of io_service using epoll and then subsequent file/socket implementations.

[lewissbaker]

For async file I/O for io_service I'm thinking of using kernel aio (ie. io_submit, io_getevents and friends).
Although from my research this seems to be somewhat limited in that it's only truly async for files opened with O_DIRECT.

For async socket I/O it seems like epoll is the go.

Need to investigate how to mix the two approaches (aio/epoll) in the same event loop.

[neo5167]

So, did you guys start working on this?

[lewissbaker]

I haven't started on Linux async I/O support yet.

[neo5167]

I will work on it. But before I start can you help me compile the current code on linux? The build instructions aren't working for me.

[lewissbaker]

Sure. Can you create a new issue to work through the build issue?

[neo5167]

scheduler_operator_tests.cpp and io_service_tests.cpp are now working in linux. I will get file tests working as well and send a PR. I am using linux message queues as a sub for iocp, timerfd/eventfd for WaitableTimer/AutoresetEvent, and epoll for WaitForMultipleObjects. For files, I plan to send messages to the message queue from AIO's signal handling functions.

[lewissbaker]

Great! I look forward to the PR :)

For AIO, can you use an eventfd as the aio_resfd parameter of the iocb block for an I/O request as a way to have it wake up the epoll() loop when the operation completes. When epoll() indicates that the AIO eventfd is signalled then you know you need to call io_getevents() to retrieve pending I/O completion events. This should avoid the need for signal-handling functions.

Or are you talking about something different?

[neo5167]

That's right! I will try the resfd way. I remember a while ago that it wasn't stable. Its been long. They should've worked it out by now...

[lewissbaker]

@neo5167 How are you going with the epoll/aio io_service for Linux? Do you need any help?

[neo5167]

Was on vacation. How about I send a PR for everything without files and then send the files one once its done. WIN32 APIs are more prevalent in the files code than in timer/scheduler code, so taking longer than I expected...

[tavi-cacina]

Have you seen AFIO already? https://ned14.github.io/afio/
It states it supports coroutine TS (async_file_handle) and @ned14 seems very convinced he is on the right path (I watched few presentations from him).

[ned14]

I intend to propose AFIO as the File I/O TS in the June WG21 meeting. AFIO is the complete redesign after a Boost peer review in 2013, and therefore ought to carry some weight with regards to design with WG21 (but we shall see!). As you'll notice in AFIO, we don't prioritise async file i/o in the design, mainly because it isn't faster for most use cases, is definitely not faster on SSDs and is positively retrograde on SCM. I suspect you'll thus want to stick with your implementation. But keep at your library, looks valuable and I'd encourage you to submit it to Boost soon. We sorely need abstractions built on top of Coroutines both for Boost, and thence the C++ library.

[lewissbaker]

@neo5167 A PR with the basic io_service event loop and support for scheduling coroutines sounds like a great first step! I'd be happy to review it at that stage.
I've also been working on adding some socket APIs for Win32 in the sockets/sockets2 branch - this does some refactoring of the file_read/write_operation classes to eliminate some of the boiler-plate and also to add separate cancellable and non-cancellable variants of the operations (non-cancellable version generates much tighter assembly). Maybe check that out and see if a similar approach can be taken for the Linux implementations.

@tavi-cacina Thanks for pointing AFIO out. I had heard of it but haven't had a chance to look into it in any depth yet. I'll check it out.

@ned14 Thanks for the encouragement :) There's still a lot I want to add to it and a few things I want to revisit before I feel it will be baked enough for submitting to Boost.