本文為英文版的機器翻譯版本，如內容有任何歧義或不一致之處，概以英文版為準。

# Cohere Embed v4
<a name="model-parameters-embed-v4"></a>

Cohere Embed v4 是一種多模態嵌入模型，支援文字和影像輸入。其可處理交錯的文字和影像內容，因此非常適合文件理解、視覺化搜尋和多模態擷取應用程式。此模型支援各種嵌入類型，包括 float、int8、uint8、binary 和 ubinary 格式，具有從 256 到 1536 的可設定輸出維度。

Cohere Embed v4 的模型 ID 是 `cohere.embed-v4`。

**其他使用說明**  

+ **內容長度：**每份文件最多 \~128k 個字符；對於 RAG，較小的區塊通常會改善擷取和成本。
+ **影像大小：**> 2,458,624 像素的影像會縮減取樣至該大小；< 3,136 像素的影像會擴大取樣。
+ **交錯輸入：**類似頁面的多模態內容偏好使用 inputs.content[]，因此文字內容 (例如檔案名稱、實體) 會隨影像一起移動。

**Topics**
+ [請求與回應](#model-parameters-embed-v4-request-response)
+ [不同 input\_types 的請求和回應](#api-inference-examples-cohere-embed-v4)
+ [程式碼範例](#code-examples-cohere-embed-v4)

## 請求與回應
<a name="model-parameters-embed-v4-request-response"></a>

------
#### [ Request ]

Content type: application/json

```
{
  "input_type": "search_document | search_query | classification | clustering",
  "texts": ["..."],                      // optional; text-only
  "images": ["data:<mime>;base64,..."],  // optional; image-only
  "inputs": [
    { "content": [
        { "type": "text",      "text": "..." },
        { "type": "image_url", "image_url": {"url": "data:<mime>;base64,..."} }
      ]
    }
  ],                                     // optional; mixed (interleaved) text+image
  "embedding_types": ["float" | "int8" | "uint8" | "binary" | "ubinary"],
  "output_dimension": 256 | 512 | 1024 | 1536,
  "max_tokens": 128000,
  "truncate": "NONE | LEFT | RIGHT"
}
```

**Parameters**  

+ **input\_type** (必要) – 新增特殊字符以區分使用案例。允許：`search_document`、`search_query`、`classification`、`clustering`。對於搜尋/RAG，使用 `search_document` 嵌入主體，使用 `search_query` 嵌入查詢。
+ **texts** (選用) – 要嵌入的字串陣列。每次呼叫最多 96 個。如果您使用 `texts`，請勿在同一個通話中傳送 `images`。
+ **image** (選用) – 要嵌入的資料 URL base64 影像陣列。每次呼叫最多 96 個。請勿將 `texts` 和 `images` 一起傳送。(`inputs` 用於交錯。)
+ **input** (選用；混合/融合模式) – 此清單中的每個項目都有組件內容清單。每個組件都是 `{ "type": "text", "text": ... }` 或 `{ "type": "image_url", "image_url": {"url": "data:<mime>;base64,..."} }`。在此處傳送交錯的類似頁面內容 (例如，PDF 頁面影像 \+ 字幕/中繼資料)。最多 96 個項目。
+ **embeddedding\_types** (選用) – 一或多個 `float`、`int8`、`uint8`、`binary`、`ubinary`。如果省略，則傳回浮點數嵌入。
+ **output\_dimension** (選用) – 選取向量長度。允許：`256`、`512`、`1024`、 `1536` (如果未指定，則預設為 `1536`)。
+ **max\_tokens** （選用） – 每個輸入文件的截斷預算。模型每個文件最多支援 \~128，000 個字符；適用時 RAG 的區塊較小。
+ **截斷** (選用) – 如何處理超長輸入：`LEFT` 從起點捨棄字符；`RIGHT` 從結尾捨棄字符；如果輸入超過限制，則 `NONE` 傳回錯誤。

**限制和調整大小**  

+ 每個請求的項目數：最多 96 個影像。原始影像檔案類型必須是 png、jpeg、webp 或 gif 格式，大小最多可達 5 MB。
+ 請求大小上限：總承載約 20 MB。
+ 輸入字符上限：最多 128k 字符。影像檔案會轉換為字符，且字符總數應小於 128k。
+ 影像：大於 2,458,624 像素則縮減取樣；小於 3,136 像素的影像則擴大取樣。以 `data:<mime>;base64,....` 形式傳送影像
+ 字符會計 (每個 `inputs` 項目)：來自影像輸入的字符數 ≈ (影像像素 ÷ 784) x 4；來自交錯文字和影像輸入的字符數 = (影像像素 ÷ 784) x 4 \+ (文字字符數)

**提示：**對於 PDF，將每個頁面轉換為影像，並透過 `inputs` 與相鄰文字部分中的頁面中繼資料 (例如 file\_name、實體) 一起傳送。

------
#### [ Response ]

Content type: application/json

如果您請求單一嵌入類型 (例如，僅 `float`)：

```
{
"id": "string",
"embeddings": [[ /* length = output_dimension */ ]],
"response_type": "embeddings_floats",
"texts": ["..."], // present if text was provided
"inputs": [ { "content": [ ... ] } ] // present if 'inputs' was used
}
```

如果您請求多個嵌入類型 (例如 `["float","int8"]`)：

```
{
  "id": "string",
  "embeddings": {
    "float": [[ ... ]],
    "int8":  [[ ... ]]
  },
  "response_type": "embeddings_by_type",
  "texts": ["..."],     // when text used
  "inputs": [ { "content": [ ... ] } ] // when 'inputs' used
}
```
+ 傳回的向量數量符合 `texts` 陣列的長度或 `inputs` 項目數量。
+ 每個向量的長度等於 `output_dimension` (預設為 `1536`)。

------

## 不同 input\_types 的請求和回應
<a name="api-inference-examples-cohere-embed-v4"></a>

**A) 具有精簡 int8 向量的交錯頁面 (影像 \+ 字幕)**

**請求**  


```
{
  "input_type": "search_document",
  "inputs": [
    {
      "content": [
        { "type": "text", "text": "Quarterly ARR growth chart; outlier in Q3." },
        { "type": "image_url", "image_url": {"url": "data:image/png;base64,{{BASE64_PAGE_IMG}}"} }
      ]
    }
  ],
  "embedding_types": ["int8"],
  "output_dimension": 512,
  "truncate": "RIGHT",
  "max_tokens": 128000
}
```

**回應 (截斷)**  


```
{
  "id": "836a33cc-61ec-4e65-afaf-c4628171a315",
  "embeddings": { "int8": [[ 7, -3, ... ]] },
  "response_type": "embeddings_by_type",
  "inputs": [
    { "content": [
      { "type": "text", "text": "Quarterly ARR growth chart; outlier in Q3." },
      { "type": "image_url", "image_url": {"url": "data:image/png;base64,{{...}}"} }
    ] }
  ]
}
```

**B) 純文字主體索引 (預設浮點數，1536-dim)**

**請求**  


```
{
  "input_type": "search_document",
  "texts": [
    "RAG system design patterns for insurance claims",
    "Actuarial loss triangles and reserving primer"
  ]
}
```

**回應 (範例)**  


```
{
  "response_type": "embeddings_floats",
  "embeddings": [
    [0.0135, -0.0272, ...],   // length 1536
    [0.0047,  0.0189, ...]
  ]
}
```

## 程式碼範例
<a name="code-examples-cohere-embed-v4"></a>

------
#### [ Text input ]

```
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
# SPDX-License-Identifier: Apache-2.0
"""
Shows how to generate embeddings using the Cohere Embed v4 model.
"""
import json
import logging
import boto3


from botocore.exceptions import ClientError

logger = logging.getLogger(__name__)
logging.basicConfig(level=logging.INFO)


def generate_text_embeddings(model_id, body, region_name):
    """
    Generate text embedding by using the Cohere Embed model.
    Args:
        model_id (str): The model ID to use.
        body (str) : The reqest body to use.
        region_name (str): The AWS region to invoke the model on
    Returns:
        dict: The response from the model.
    """

    logger.info("Generating text embeddings with the Cohere Embed model %s", model_id)

    accept = '*/*'
    content_type = 'application/json'

    bedrock = boto3.client(service_name='bedrock-runtime', region_name=region_name)

    response = bedrock.invoke_model(
        body=body,
        modelId=model_id,
        accept=accept,
        contentType=content_type
    )

    logger.info("Successfully generated embeddings with Cohere model %s", model_id)

    return response


def main():
    """
    Entrypoint for Cohere Embed example.
    """

    logging.basicConfig(level=logging.INFO, format="%(levelname)s: %(message)s")
    
    region_name = 'us-east-1'

    model_id = 'cohere.embed-v4:0'
    text1 = "hello world"
    text2 = "this is a test"
    input_type = "search_document"
    embedding_types = ["float"]

    try:
        body = json.dumps({
            "texts": [
                text1,
                text2],
            "input_type": input_type,
            "embedding_types": embedding_types
        })
        
        response = generate_text_embeddings(model_id=model_id, body=body, region_name=region_name)

        response_body = json.loads(response.get('body').read())

        print(f"ID: {response_body.get('id')}")
        print(f"Response type: {response_body.get('response_type')}")

        print("Embeddings")
        embeddings = response_body.get('embeddings')
        for i, embedding_type in enumerate(embeddings):
            print(f"\t{embedding_type} Embeddings:")
            print(f"\t{embeddings[embedding_type]}")

        print("Texts")
        for i, text in enumerate(response_body.get('texts')):
            print(f"\tText {i}: {text}")

    except ClientError as err:
        message = err.response["Error"]["Message"]
        logger.error("A client error occurred: %s", message)
        print("A client error occured: " +
              format(message))
    else:
        print(
            f"Finished generating text embeddings with Cohere model {model_id}.")


if __name__ == "__main__":
    main()
```

------
#### [ Mixed modalities ]

```
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
# SPDX-License-Identifier: Apache-2.0
"""
Shows how to generate image embeddings using the Cohere Embed v4 model.
"""
import json
import logging
import boto3
import base64


from botocore.exceptions import ClientError

logger = logging.getLogger(__name__)
logging.basicConfig(level=logging.INFO)

def get_base64_image_uri(image_file_path: str, image_mime_type: str):
    with open(image_file_path, "rb") as image_file:
        image_bytes = image_file.read()
        base64_image = base64.b64encode(image_bytes).decode("utf-8")
    return f"data:{image_mime_type};base64,{base64_image}"


def generate_embeddings(model_id, body, region_name):
    """
    Generate image embedding by using the Cohere Embed model.
    Args:
        model_id (str): The model ID to use.
        body (str) : The reqest body to use.
        region_name (str): The AWS region to invoke the model on
    Returns:
        dict: The response from the model.
    """

    logger.info("Generating image embeddings with the Cohere Embed model %s", model_id)

    accept = '*/*'
    content_type = 'application/json'

    bedrock = boto3.client(service_name='bedrock-runtime', region_name=region_name)

    response = bedrock.invoke_model(
        body=body,
        modelId=model_id,
        accept=accept,
        contentType=content_type
    )

    logger.info("Successfully generated embeddings with Cohere model %s", model_id)

    return response


def main():
    """
    Entrypoint for Cohere Embed example.
    """

    logging.basicConfig(level=logging.INFO, format="%(levelname)s: %(message)s")
    
    region_name = 'us-east-1'

    image_file_path = "image.jpg"
    image_mime_type = "image/jpg"
    text = "hello world"

    model_id = 'cohere.embed-v4:0'
    input_type = "search_document"
    image_base64_uri = get_base64_image_uri(image_file_path, image_mime_type)
    embedding_types = ["int8","float"]

    try:
        body = json.dumps({
            "inputs": [
                {
                  "content": [
                    { "type": "text", "text": text },
                    { "type": "image_url", "image_url": {"url": "data:image/png;base64,{{image_base64_uri}}"} }
                  ]
                }
              ],
            "input_type": input_type,
            "embedding_types": embedding_types
        })
        
        response = generate_embeddings(model_id=model_id, body=body, region_name=region_name)

        response_body = json.loads(response.get('body').read())

        print(f"ID: {response_body.get('id')}")
        print(f"Response type: {response_body.get('response_type')}")

        print("Embeddings")
        embeddings = response_body.get('embeddings')
        for i, embedding_type in enumerate(embeddings):
            print(f"\t{embedding_type} Embeddings:")
            print(f"\t{embeddings[embedding_type]}")

        print("inputs")
        for i, input in enumerate(response_body.get('inputs')):
            print(f"\tinput {i}: {input}")

    except ClientError as err:
        message = err.response["Error"]["Message"]
        logger.error("A client error occurred: %s", message)
        print("A client error occured: " +
              format(message))
    else:
        print(
            f"Finished generating embeddings with Cohere model {model_id}.")


if __name__ == "__main__":
    main()
```

------