NULL Pointer access when embed_documents

[raymondshe]

When I was working on the samples here on my Mac M1 laptop.

https://ai.plainenglish.io/%EF%B8%8F-langchain-streamlit-llama-bringing-conversational-ai-to-your-local-machine-a1736252b172

Everything looks fine until run "Part of the journey is the Embeddings !!!" section.

I was using:

embeddings = LlamaCppEmbeddings(model_path="/Users/raymond/moduls/mellogpt.Q4_K_M.gguf",n_gpu_layers= 1)

and the model load fine and I can use it for query. But when

embedded_texts = embeddings.embed_documents(_texts)

The full code is at, but I changed some parameters shown above.
https://miro.medium.com/v2/resize:fit:1400/format:webp/1*_Mw2-tecTljL8gF_IS6XGw.png

I got the problem:

ValueError Traceback (most recent call last)
ValueError: NULL pointer access

The above exception was the direct cause of the following exception:

SystemError Traceback (most recent call last)
Cell In[13], line 1
----> 1 embedded_texts = embeddings.embed_documents(_texts)
2 len(embedded_texts), len(embedded_texts[0])

File ~/miniforge3/envs/llama/lib/python3.9/site-packages/langchain_community/embeddings/llamacpp.py:113, in LlamaCppEmbeddings.embed_documents(self, texts)
104 def embed_documents(self, texts: List[str]) -> List[List[float]]:
105 """Embed a list of documents using the Llama model.
106
107 Args:
(...)
111 List of embeddings, one for each text.
112 """
--> 113 embeddings = [self.client.embed(text) for text in texts]
114 return [list(map(float, e)) for e in embeddings]

File ~/miniforge3/envs/llama/lib/python3.9/site-packages/langchain_community/embeddings/llamacpp.py:113, in (.0)
104 def embed_documents(self, texts: List[str]) -> List[List[float]]:
105 """Embed a list of documents using the Llama model.
106
107 Args:
(...)
111 List of embeddings, one for each text.
112 """
--> 113 embeddings = [self.client.embed(text) for text in texts]
114 return [list(map(float, e)) for e in embeddings]

File ~/miniforge3/envs/llama/lib/python3.9/site-packages/llama_cpp/llama.py:1338, in Llama.embed(self, input)
1329 def embed(self, input: str) -> List[float]:
1330 """Embed a string.
1331
1332 Args:
(...)
1336 A list of embeddings
1337 """
-> 1338 return list(map(float, self.create_embedding(input)["data"][0]["embedding"]))

File ~/miniforge3/envs/llama/lib/python3.9/site-packages/llama_cpp/llama.py:1309, in Llama.create_embedding(self, input, model)
1304 total_tokens += n_tokens
1305 embedding = llama_cpp.llama_get_embeddings(self._ctx.ctx)[
1306 : llama_cpp.llama_n_embd(self._model.model)
1307 ]
-> 1309 data.append(
1310 {
1311 "object": "embedding",
1312 "embedding": embedding,
1313 "index": index,
1314 }
1315 )
1316 if self.verbose:
1317 llama_cpp.llama_print_timings(self._ctx.ctx)

SystemError: <method 'append' of 'list' objects> returned a result with an error set

[raymondshe]

python --version
Python 3.9.18
pip show llama_cpp_python
Name: llama_cpp_python
Version: 0.2.23
Summary: Python bindings for the llama.cpp library
Home-page:
Author:
Author-email: Andrei Betlen [email protected]
License: MIT
Location: /Users/raymond/miniforge3/envs/llama/lib/python3.9/site-packages
Requires: diskcache, numpy, typing-extensions
Required-by:

[brandonrobertz]

This should be fixed in latest with this PR: #1019 (0.2.24 released a few hours ago)

[abetlen]

Closing this as I believe @brandonrobertz PR fixed this issue, will reopen if issue still persists.

[raymondshe]

Thanks, I am going to try it.

[alexandrebcaruso]

still seems to be broken

pip show llama_cpp_python
Name: llama_cpp_python
Version: 0.2.56

hope it helps

[brandonrobertz]

still seems to be broken

pip show llama_cpp_python
Name: llama_cpp_python
Version: 0.2.56

hope it helps

That's a different error. Open a new issue