HuggingFaceEndpoint will access "mistralai/Mistral-Nemo-Instruct-2407" even we don't use this model

[XinyaoWa]

Checked other resources

• I added a very descriptive title to this issue.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.
• I am sure that this is a bug in LangChain rather than my code.
• The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code

from langchain_community.llms import HuggingFaceEndpoint
from langchain.chains.summarize import load_summarize_chain
from langchain.docstore.document import Document
from langchain.text_splitter import CharacterTextSplitter
import asyncio

async def invoke():
client = HuggingFaceEndpoint(
endpoint_url="http://0.0.0.0:8000", # my tgi endpoint
task="text-generation",
streaming=False,
)

message = "What's deep learning?"
llm_chain = load_summarize_chain(llm=client)
text_splitter = CharacterTextSplitter()
texts = text_splitter.split_text(message)

docs = [Document(page_content=t) for t in texts]
response = await llm_chain.ainvoke(docs)
response = response["output_text"]
print(response)

asyncio.run(invoke())

Error Message and Stack Trace (if applicable)

Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/aiohttp/client.py", line 703, in _request
conn = await self._connector.connect(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/aiohttp/connector.py", line 548, in connect
proto = await self._create_connection(req, traces, timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/aiohttp/connector.py", line 1056, in _create_connection
_, proto = await self._create_direct_connection(req, traces, timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/aiohttp/connector.py", line 1369, in _create_direct_connection
transp, proto = await self._wrap_create_connection(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/aiohttp/connector.py", line 1115, in _wrap_create_connection
sock = await aiohappyeyeballs.start_connection(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/aiohappyeyeballs/impl.py", line 78, in start_connection
sock, _, _ = await _staggered.staggered_race(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/aiohappyeyeballs/_staggered.py", line 160, in staggered_race
done = await _wait_one(
^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/aiohappyeyeballs/_staggered.py", line 41, in _wait_one
return await wait_next
^^^^^^^^^^^^^^^
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/huggingface_hub/inference/_generated/_async_client.py", line 314, in _inner_post
response = await session.post(
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/huggingface_hub/inference/_generated/_async_client.py", line 3364, in _request
response = await session._wrapped_request(method, url, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/aiohttp/client.py", line 607, in _request
with timer:
File "/usr/local/lib/python3.11/site-packages/aiohttp/helpers.py", line 671, in exit
raise asyncio.TimeoutError from exc_val
TimeoutError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/home/user/comps/llms/src/faq-generation/test.py", line 24, in
asyncio.run(invoke())
File "/usr/local/lib/python3.11/asyncio/runners.py", line 190, in run
return runner.run(main)
^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/asyncio/runners.py", line 118, in run
return self._loop.run_until_complete(task)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/asyncio/base_events.py", line 654, in run_until_complete
return future.result()
^^^^^^^^^^^^^^^
File "/home/user/comps/llms/src/faq-generation/test.py", line 20, in invoke
response = await llm_chain.ainvoke(docs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/langchain/chains/base.py", line 221, in ainvoke
raise e
File "/usr/local/lib/python3.11/site-packages/langchain/chains/base.py", line 212, in ainvoke
await self._acall(inputs, run_manager=run_manager)
File "/usr/local/lib/python3.11/site-packages/langchain/chains/combine_documents/base.py", line 154, in _acall
output, extra_return_dict = await self.acombine_docs(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/langchain/chains/combine_documents/stuff.py", line 277, in acombine_docs
return await self.llm_chain.apredict(callbacks=callbacks, **inputs), {}
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/langchain/chains/llm.py", line 335, in apredict
return (await self.acall(kwargs, callbacks=callbacks))[self.output_key]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/langchain_core/_api/deprecation.py", line 189, in awarning_emitting_wrapper
return await wrapped(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/langchain/chains/base.py", line 439, in acall
return await self.ainvoke(
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/langchain/chains/base.py", line 221, in ainvoke
raise e
File "/usr/local/lib/python3.11/site-packages/langchain/chains/base.py", line 212, in ainvoke
await self._acall(inputs, run_manager=run_manager)
File "/usr/local/lib/python3.11/site-packages/langchain/chains/llm.py", line 300, in _acall
response = await self.agenerate([inputs], run_manager=run_manager)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/langchain/chains/llm.py", line 165, in agenerate
return await self.llm.agenerate_prompt(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/langchain_core/language_models/llms.py", line 770, in agenerate_prompt
return await self.agenerate(
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/langchain_core/language_models/llms.py", line 1211, in agenerate
output = await self._agenerate_helper(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/langchain_core/language_models/llms.py", line 1027, in _agenerate_helper
await self._agenerate(
File "/usr/local/lib/python3.11/site-packages/langchain_core/language_models/llms.py", line 1542, in _agenerate
await self._acall(prompt, stop=stop, run_manager=run_manager, **kwargs)
File "/usr/local/lib/python3.11/site-packages/langchain_community/llms/huggingface_endpoint.py", line 301, in _acall
response = await self.async_client.post(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/huggingface_hub/inference/_generated/_async_client.py", line 267, in post
return await self._inner_post(
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/huggingface_hub/inference/_generated/_async_client.py", line 333, in _inner_post
raise InferenceTimeoutError(f"Inference call timed out: {request_parameters.url}") from error # type: ignore
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
huggingface_hub.errors.InferenceTimeoutError: Inference call timed out: https://api-inference.huggingface.co/models/mistralai/Mistral-Nemo-Instruct-2407

Description

After update to latest langchain version, I meet an issue same as #29685
After add task="text-generation" according to the solution proposed, I meet another error:
HuggingFaceEndpoint will try to access mistralai/Mistral-Nemo-Instruct-2407 and get time out error
However, I don't need to use this model.

After downgrade the version as following, the bug fixed
huggingface_hub==0.27.1
langchain==0.3.14
langchain_community==0.3.14

System Info

System Information

OS: Linux
OS Version: #141-Ubuntu SMP Fri Jan 10 21:18:28 UTC 2025
Python Version: 3.11.11 (main, Feb 4 2025, 04:55:09) [GCC 12.2.0]

Package Information

langchain_core: 0.3.34
langchain: 0.3.18
langchain_community: 0.3.17
langsmith: 0.2.11
langchain_huggingface: 0.1.2
langchain_openai: 0.3.5
langchain_text_splitters: 0.3.6
langchainhub: 0.1.21

Optional packages not installed

langserve

Other Dependencies

aiohttp<4.0.0,>=3.8.3: Installed. No version info available.
async-timeout<5.0.0,>=4.0.0;: Installed. No version info available.
dataclasses-json<0.7,>=0.5.7: Installed. No version info available.
httpx: 0.28.1
httpx-sse<1.0.0,>=0.4.0: Installed. No version info available.
huggingface-hub: 0.28.1
jsonpatch<2.0,>=1.33: Installed. No version info available.
langchain-anthropic;: Installed. No version info available.
langchain-aws;: Installed. No version info available.
langchain-cohere;: Installed. No version info available.
langchain-community;: Installed. No version info available.
langchain-core<1.0.0,>=0.3.34: Installed. No version info available.
langchain-deepseek;: Installed. No version info available.
langchain-fireworks;: Installed. No version info available.
langchain-google-genai;: Installed. No version info available.
langchain-google-vertexai;: Installed. No version info available.
langchain-groq;: Installed. No version info available.
langchain-huggingface;: Installed. No version info available.
langchain-mistralai;: Installed. No version info available.
langchain-ollama;: Installed. No version info available.
langchain-openai;: Installed. No version info available.
langchain-text-splitters<1.0.0,>=0.3.6: Installed. No version info available.
langchain-together;: Installed. No version info available.
langchain<1.0.0,>=0.3.18: Installed. No version info available.
langsmith-pyo3: Installed. No version info available.
langsmith<0.4,>=0.1.125: Installed. No version info available.
langsmith<0.4,>=0.1.17: Installed. No version info available.
numpy<2,>=1.26.4;: Installed. No version info available.
numpy<3,>=1.26.2;: Installed. No version info available.
openai<2.0.0,>=1.58.1: Installed. No version info available.
orjson: 3.10.15
packaging: 24.2
packaging<25,>=23.2: Installed. No version info available.
pydantic: 2.10.6
pydantic-settings<3.0.0,>=2.4.0: Installed. No version info available.
pydantic<3.0.0,>=2.5.2;: Installed. No version info available.
pydantic<3.0.0,>=2.7.4: Installed. No version info available.
pydantic<3.0.0,>=2.7.4;: Installed. No version info available.
PyYAML>=5.3: Installed. No version info available.
requests: 2.32.3
requests-toolbelt: 1.0.0
requests<3,>=2: Installed. No version info available.
sentence-transformers: 3.4.1
SQLAlchemy<3,>=1.4: Installed. No version info available.
tenacity!=8.4.0,<10,>=8.1.0: Installed. No version info available.
tenacity!=8.4.0,<10.0.0,>=8.1.0: Installed. No version info available.
tiktoken<1,>=0.7: Installed. No version info available.
tokenizers: 0.21.0
transformers: 4.48.3
types-requests: 2.32.0.20241016
typing-extensions>=4.7: Installed. No version info available.
zstandard: 0.23.0

[dosubot]

Hi, @XinyaoWa. I'm Dosu, and I'm helping the LangChain team manage their backlog. I'm marking this issue as stale.

Issue Summary:

• You reported a bug with HuggingFaceEndpoint defaulting to "mistralai/Mistral-Nemo-Instruct-2407".
• Example code was provided to demonstrate the issue.
• The problem persists even after updating to the latest LangChain version.
• No further comments or activity have been added to the issue.

Next Steps:

• Please confirm if this issue is still relevant with the latest version of LangChain by commenting here.
• If no updates are provided, the issue will be automatically closed in 7 days.

Thank you for your understanding and contribution!