Issue running the HotPotQA example with DSPy & Ollama

[haoxianghz]

I'm trying to run this example: https://github.com/GenseeAI/cognify/blob/main/examples/HotPotQA/ref.ipynb
with ollama_chat/llama3.1,

but getting into this issue below when running cognify optimize workflow.py. Any idea how to fix it? Thanks

Process Process-4:
Traceback (most recent call last):
File ".../python3.12/site-packages/litellm/main.py", line 2748, in completion
generator = ollama_chat.get_ollama_response(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ".../python3.12/site-packages/litellm/llms/ollama_chat.py", line 324, in get_ollama_response
raise OllamaError(status_code=response.status_code, message=response.text)
litellm.llms.ollama_chat.OllamaError: {"error":"json: cannot unmarshal array into Go struct field ChatRequest.messages of type string"}

[reyna-abhyankar]

Seems to be an underlying issue with the way LiteLLM is calling ollama, referenced here and here in the context of the image API. We'll hotfix this on our end for now so at least the non-vision API works as expected.

[haoxianghz]

Thanks for looking into this.

It is not about image API. Ollama model (text only model) is going through .../litellm/llms/ollama_chat.py by DSPy, where the actual calling is at "response = requests.post(**_request)", this is handled ok in DSPy, so its input to model has a format of "context": "str". But in Cognify, the format is "context": [{..., "type": "text"}, ...].

For the langchain example (quickstart), I was able to get it working by changing "chat_model._get_invocation_params()" into "chat_model._get_ls_params()" in "/cognify/frontends/langchain/connector.py". But I find it still difficult to twist Cognify working with DSPy / Ollama (that is, /examples/HotPotQA/ref.ipynb but not using OpenAI API instead using Ollama). Maybe if you can test this setup from your end, that would be great!