Cohere 嵌入 v4

Cohere Embed v4是一种支持文本和图像输入的多模态嵌入模型。它可以处理交错的文本和图像内容，因此非常适合文档理解、视觉搜索和多模式检索应用程序。该模型支持各种嵌入类型，包括 float、int8、uint8、二进制和 ubinary 格式，输出维度可配置为 256 到 1536。

的型号 ID Cohere Embed v4 为cohere.embed-v4。

其他用法说明

上下文长度：最多支持大约 12.8 万个令牌；对于 RAG 来说，较小的区块通常可以提高检索率和成本。
图像大小：大于 2,458,624 像素的图像将缩减为该大小；对小于 3,136 像素的图像进行上采样。
交错输入：对于类似页面的多模态内容，首选 inputs.content []，这样文本上下文（例如文件名、实体）就会随图像一起移动。

请求和响应

Request

内容类型：应用程序/json


{
  "input_type": "search_document | search_query | classification | clustering",
  "texts": ["..."],                      // optional; text-only
  "images": ["data:<mime>;base64,..."],  // optional; image-only
  "inputs": [
    { "content": [
        { "type": "text",      "text": "..." },
        { "type": "image_url", "image_url": "data:<mime>;base64,..." }
      ]
    }
  ],                                     // optional; mixed (interleaved) text+image
  "embedding_types": ["float" | "int8" | "uint8" | "binary" | "ubinary"],
  "output_dimension": 256 | 512 | 1024 | 1536,
  "max_tokens": 128000,
  "truncate": "NONE | LEFT | RIGHT"
}

参数

input_type（必填）-添加特殊标记以区分用例。允许：search_document、search_query、classification、clustering。对于 Search/rag，请将语料库嵌入并使用进行search_document查询。search_query
文本（可选）-要嵌入的字符串数组。每次通话最多 96 个。如果您使用texts，请不要发送images同一个呼叫。
图像（可选）— 要嵌入的 data-URI base64 图像数组。每次通话最多 96 个。不要images一起texts发送。（inputs用于交错存取。）
inp uts（可选 mixed/fused ；modality）-一个列表，其中每个项目都有部分内容列表。每个部分都是{ "type": "text", "text": ... }或{ "type": "image_url", "image_url": "data:<mime>;base64,..." }。在此处发送类似页面的交错内容（例如，PDF 页面图片 + 标题/元数据）。最多 96 件物品。
embedding_types（可选）— 一个或多个：float、、、int8、uint8。binary ubinary如果省略，则返回浮点嵌入。
输出尺寸（可选）-选择向量长度。允许：256、、5121024、1536（1536如果未指定，则为默认值）。
max_token s（可选）-每个输入对象的截断预算。该模型最多支持大约 128,000 个代币；对于 RAG，请酌情缩小区块数量。
truncate（可选）— 如何处理超长输入：从一开始就LEFT丢弃标记；从结尾RIGHT丢弃；如果输入超过限制，则NONE返回错误。

限制和规模

每个请求的项目：最多 96 张图片。原始图像文件类型必须采用 png、jpeg、webp 或 gif 格式，大小不超过 5 MB。
请求大小上限：总有效载荷约为 20 MB。
最大输入令牌：最多 12.8 万个代币。图像文件将转换为标记，总令牌数应小于 128k。
图像：缩减像素采样前的最大像素为 2,458,624 像素；对小于 3,136 像素的图像进行上采样。将图片提供为 data:<mime>;base64,....
代币记账（每inputs件商品）：来自图像输入的代币 ↑（图像像素 ε 784）x 来自交错文本和图像输入的 4 个代币 =（图像像素 ε 784）x 4 +（文本标记）

提示：对于 PDFs，将每个页面转换为图像，然后将页面元数据（例如 file_name、实体）与相邻文本部分中的页面元数据（例如 file_name、实体）inputs一起发送。

Response

内容类型：应用程序/json

如果您请求单一嵌入类型（例如，仅限float）：


{
"id": "string",
"embeddings": [[ /* length = output_dimension */ ]],
"response_type": "embeddings_floats",
"texts": ["..."], // present if text was provided
"inputs": [ { "content": [ ... ] } ] // present if 'inputs' was used
}

如果您请求了多种嵌入类型（例如["float","int8"]）：


{
  "id": "string",
  "embeddings": {
    "float": [[ ... ]],
    "int8":  [[ ... ]]
  },
  "response_type": "embeddings_by_type",
  "texts": ["..."],     // when text used
  "inputs": [ { "content": [ ... ] } ] // when 'inputs' used
}

返回的向量数量与数组的长度或inputs项texts数相匹配。
每个向量的长度等于output_dimension（默认1536）。

不同输入类型的请求和响应

A) 带有紧凑的 int8 向量的交错页面（图片+标题）

请求


{
  "input_type": "search_document",
  "inputs": [
    {
      "content": [
        { "type": "text", "text": "Quarterly ARR growth chart; outlier in Q3." },
        { "type": "image_url", "image_url": "data:image/png;base64,{{BASE64_PAGE_IMG}}" }
      ]
    }
  ],
  "embedding_types": ["int8"],
  "output_dimension": 512,
  "truncate": "RIGHT",
  "max_tokens": 128000
}

响应（截断）


{
  "id": "836a33cc-61ec-4e65-afaf-c4628171a315",
  "embeddings": { "int8": [[ 7, -3, ... ]] },
  "response_type": "embeddings_by_type",
  "inputs": [
    { "content": [
      { "type": "text", "text": "Quarterly ARR growth chart; outlier in Q3." },
      { "type": "image_url", "image_url": "data:image/png;base64,{{...}}" }
    ] }
  ]
}

B) 纯文本语料库索引（默认浮点数，1536-dim）

请求


{
  "input_type": "search_document",
  "texts": [
    "RAG system design patterns for insurance claims",
    "Actuarial loss triangles and reserving primer"
  ]
}

响应（示例）


{
  "response_type": "embeddings_floats",
  "embeddings": [
    [0.0135, -0.0272, ...],   // length 1536
    [0.0047,  0.0189, ...]
  ]
}

代码示例

Text input


# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
# SPDX-License-Identifier: Apache-2.0
"""
Shows how to generate embeddings using the Cohere Embed v4 model.
"""
import json
import logging
import boto3


from botocore.exceptions import ClientError

logger = logging.getLogger(__name__)
logging.basicConfig(level=logging.INFO)


def generate_text_embeddings(model_id, body, region_name):
    """
    Generate text embedding by using the Cohere Embed model.
    Args:
        model_id (str): The model ID to use.
        body (str) : The reqest body to use.
        region_name (str): The AWS region to invoke the model on
    Returns:
        dict: The response from the model.
    """

    logger.info("Generating text embeddings with the Cohere Embed model %s", model_id)

    accept = '*/*'
    content_type = 'application/json'

    bedrock = boto3.client(service_name='bedrock-runtime', region_name=region_name)

    response = bedrock.invoke_model(
        body=body,
        modelId=model_id,
        accept=accept,
        contentType=content_type
    )

    logger.info("Successfully generated embeddings with Cohere model %s", model_id)

    return response


def main():
    """
    Entrypoint for Cohere Embed example.
    """

    logging.basicConfig(level=logging.INFO, format="%(levelname)s: %(message)s")
    
    region_name = 'us-east-1'

    model_id = 'cohere.embed-v4:0'
    text1 = "hello world"
    text2 = "this is a test"
    input_type = "search_document"
    embedding_types = ["float"]

    try:
        body = json.dumps({
            "texts": [
                text1,
                text2],
            "input_type": input_type,
            "embedding_types": embedding_types
        })
        
        response = generate_text_embeddings(model_id=model_id, body=body, region_name=region_name)

        response_body = json.loads(response.get('body').read())

        print(f"ID: {response_body.get('id')}")
        print(f"Response type: {response_body.get('response_type')}")

        print("Embeddings")
        embeddings = response_body.get('embeddings')
        for i, embedding_type in enumerate(embeddings):
            print(f"\t{embedding_type} Embeddings:")
            print(f"\t{embeddings[embedding_type]}")

        print("Texts")
        for i, text in enumerate(response_body.get('texts')):
            print(f"\tText {i}: {text}")

    except ClientError as err:
        message = err.response["Error"]["Message"]
        logger.error("A client error occurred: %s", message)
        print("A client error occured: " +
              format(message))
    else:
        print(
            f"Finished generating text embeddings with Cohere model {model_id}.")


if __name__ == "__main__":
    main()

Mixed modalities


# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
# SPDX-License-Identifier: Apache-2.0
"""
Shows how to generate image embeddings using the Cohere Embed v4 model.
"""
import json
import logging
import boto3
import base64


from botocore.exceptions import ClientError

logger = logging.getLogger(__name__)
logging.basicConfig(level=logging.INFO)

def get_base64_image_uri(image_file_path: str, image_mime_type: str):
    with open(image_file_path, "rb") as image_file:
        image_bytes = image_file.read()
        base64_image = base64.b64encode(image_bytes).decode("utf-8")
    return f"data:{image_mime_type};base64,{base64_image}"


def generate_embeddings(model_id, body, region_name):
    """
    Generate image embedding by using the Cohere Embed model.
    Args:
        model_id (str): The model ID to use.
        body (str) : The reqest body to use.
        region_name (str): The AWS region to invoke the model on
    Returns:
        dict: The response from the model.
    """

    logger.info("Generating image embeddings with the Cohere Embed model %s", model_id)

    accept = '*/*'
    content_type = 'application/json'

    bedrock = boto3.client(service_name='bedrock-runtime', region_name=region_name)

    response = bedrock.invoke_model(
        body=body,
        modelId=model_id,
        accept=accept,
        contentType=content_type
    )

    logger.info("Successfully generated embeddings with Cohere model %s", model_id)

    return response


def main():
    """
    Entrypoint for Cohere Embed example.
    """

    logging.basicConfig(level=logging.INFO, format="%(levelname)s: %(message)s")
    
    region_name = 'us-east-1'

    image_file_path = "image.jpg"
    image_mime_type = "image/jpg"
    text = "hello world"

    model_id = 'cohere.embed-v4:0'
    input_type = "search_document"
    image_base64_uri = get_base64_image_uri(image_file_path, image_mime_type)
    embedding_types = ["int8","float"]

    try:
        body = json.dumps({
            "inputs": [
                {
                  "content": [
                    { "type": "text", "text": text },
                    { "type": "image_url", "image_url": "data:image/png;base64,{{image_base64_uri}}" }
                  ]
                }
              ],
            "input_type": input_type,
            "embedding_types": embedding_types
        })
        
        response = generate_embeddings(model_id=model_id, body=body, region_name=region_name)

        response_body = json.loads(response.get('body').read())

        print(f"ID: {response_body.get('id')}")
        print(f"Response type: {response_body.get('response_type')}")

        print("Embeddings")
        embeddings = response_body.get('embeddings')
        for i, embedding_type in enumerate(embeddings):
            print(f"\t{embedding_type} Embeddings:")
            print(f"\t{embeddings[embedding_type]}")

        print("inputs")
        for i, input in enumerate(response_body.get('inputs')):
            print(f"\tinput {i}: {input}")

    except ClientError as err:
        message = err.response["Error"]["Message"]
        logger.error("A client error occurred: %s", message)
        print("A client error occured: " +
              format(message))
    else:
        print(
            f"Finished generating embeddings with Cohere model {model_id}.")


if __name__ == "__main__":
    main()

Javascript 在您的浏览器中被禁用或不可用。

要使用 Amazon Web Services 文档，必须启用 Javascript。请参阅浏览器的帮助页面以了解相关说明。

文档惯例

Cohere Embed 和 Cohere Embed v4 模型

Cohere 嵌入 v3