Skip to content

Commit 9dfe0ef

Browse files
committed
Adds more parameters and explanations.
1 parent b4c99e6 commit 9dfe0ef

File tree

2 files changed

+49
-0
lines changed

2 files changed

+49
-0
lines changed

specification/inference/_types/CommonTypes.ts

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -770,6 +770,17 @@ export class CustomServiceSettings {
770770
* ```
771771
*/
772772
headers?: UserDefinedValue
773+
/**
774+
* Specifies the query parameters as a list of tuples. The arrays inside the `query_parameters` must have two items, a key and a value.
775+
* For example:
776+
* ```
777+
* "query_parameters":[
778+
* ["param_key", "some_value"],
779+
* ["param_key", "another_value"]
780+
* ]
781+
* ```
782+
*/
783+
query_parameters?: UserDefinedValue
773784
/**
774785
* The request configuration object.
775786
*/

specification/inference/put_custom/PutCustomRequest.ts

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,44 @@ import { InferenceChunkingSettings } from '@inference/_types/Services'
3232
*
3333
* The custom service gives more control over how to interact with external inference services that aren't explicitly supported through dedicated integrations.
3434
* The custom service gives you the ability to define the headers, url, query parameters, request body, and secrets.
35+
* The custom service supports the template replacement functionality, which enables you to define a template that can be replaced with the contents of a value that defines that key.
36+
* Templates are portions of a string that start with `${` and end with `}`.
37+
* The parameters `secret_parameters` and `task_settings` are checked for keys for template replacement. Template replacement is supported in the `request`, `headers`, `url`, and `query_parameters`.
38+
* If the definition (key) is not found for a template, an error message is returned.
39+
* In case of an endpoint definition like the following:
40+
* ```
41+
* PUT _inference/text_embedding/test-text-embedding
42+
* {
43+
* "service": "custom",
44+
* "service_settings": {
45+
* "secret_parameters": {
46+
* "api_key": "<some api key>"
47+
* },
48+
* "url": "...endpoints.huggingface.cloud/v1/embeddings",
49+
* "headers": {
50+
* "Authorization": "Bearer ${api_key}",
51+
* "Content-Type": "application/json"
52+
* },
53+
* "request": "{\"input\": ${input}}",
54+
* "response": {
55+
* "json_parser": {
56+
* "text_embeddings":"$.data[*].embedding[*]"
57+
* }
58+
* }
59+
* }
60+
* }
61+
* ```
62+
* To replace `${api_key}` the `secret_parameters` and `task_settings` are checked for a key named `api_key`.
63+
*
64+
* > info
65+
* > Templates should not be surrounded by quotes.
66+
*
67+
* Pre-defined templates:
68+
* * `${input}` refers to the array of input strings that comes from the `input` field of the subsequent inference requests.
69+
* * `${input_type}` refers to the input type translation values.
70+
* * `${query}` refers to the query field used specifically for reranking tasks.
71+
* * `${top_n}` refers to the `top_n` field available when performing rerank requests.
72+
* * `${return_documents}` refers to the `return_documents` field available when performing rerank requests.
3573
* @rest_spec_name inference.put_custom
3674
* @availability stack since=8.13.0 stability=stable visibility=public
3775
* @availability serverless stability=stable visibility=public

0 commit comments

Comments
 (0)