Linux support

[danpalmer]

Hey! It would be great to get Linux support in OllamaKit.

Currently there's a dependency on Combine, which is an Apple OS proprietary framework, rather than a Swift framework. This means that it can only compile for Apple platforms, and not Linux or Windows.

With the increasing use of Swift on the server, Linux support in particular would be great to have. I'm working on a Swift web application that uses Ollama, and have so far used OllamaKit without realising it had this dependency (my own fault!).

How amenable is the project to dropping the Combine dependency? If that's a hard-no, no worries, I can look for alternatively or fork the library.

[pd95]

Hi @danpalmer

I think this should be possible to do. As far as I have seen, the code provides both APIs natively.
I mean, it has implemented the logic twice: once using Combine and once using Swift async-await.
I do not have a Linux/Docker setup for testing, but I will quickly draft a version which uses #if canImport(Combine) everywhere where Combine is necessary. Then you can check if that's what you want and you can work with...

I will let you know when I have something to test...

Kind regards
Philipp

[pd95]

OK... it is not that easy... Probably URLSession and URLRequest must be replaced too...

See my "work in progress PR" #48, where I use Github Actions for testing Swift on Linux 😉

[pd95]

After looking more into details, I know that I had to use where those URLSession and URLRequest elements were missing.

#if canImport(FoundationNetworking)
import FoundationNetworking
#endif

Now I'm at a point where the async "convenience method" func bytes(for request: URLRequest) async throws -> (URLSession.AsyncBytes, URLResponse) is missing on Linux.
This seems to be a known limitation of the swift-corelibs-foundation library: swiftlang/swift-corelibs-foundation#3205

Tomorrow I will look into whether I can somehow "simulate" the same API using URLSession.dataTask using a delegate handler. The "shimming" approach shown in kishikawakatsumi/D1Kit@eef9fdb won't work for us, as we need a "streaming API": we want the partial responses from Ollama with continuous progress and not only the final complete response.

---

I've just found this: swiftlang/swift-corelibs-foundation#4972
But I'm not sure whether this addresses our problem.

[pd95]

I think, I have a working solution... probably I need to clean it up some more.
But can you please checkout the version available in #48 ?

I would like to do some more testing, but I don't know how... I've only tested in on Mac and used it in an Ollamac for testing. The streaming did work properly. Using GitHub actions, I can confirm, that the package successfully builds on Linux... that's all I can do for the moment.

@danpalmer : Can you help out?

[danpalmer]

Will give this a go, thanks!

To set expectations, I'm travelling for a few days and without stable internet, and my own Linux build is a day or so of work away from working, so between that and work it might be a little while until I can give this a go.

That said, if it builds and the test suite passes, it's probably all good!

[pd95]

Test suite? You probably haven’t looked at https://github.com/kevinhermawan/OllamaKit/tree/main/Tests/OllamaKitTests yet 😜

[pd95]

Great news: I made progress in debugging the Linux issues.
As documented in #48: Streaming responses works now on Linux too!

[danpalmer]

Amazing. So far this seems to be working but I've not had a chance to test everything yet. I think we can probably close this and I'll raise any separate bugs as I find them.

[pd95]

Not sure I this too early... as the PR #48 has not yet been merged to main. "main" has only been updated so far with a "remove Combine" refactoring. But if the branch in PR #48 is also working OK for you, let's ask @kevinhermawan to review and merge the changes in.