本文為英文版的機器翻譯版本，如內容有任何歧義或不一致之處，概以英文版為準。
# Amazon Comprehend AWS SDKs案例
下列程式碼範例示範如何在 Amazon Comprehend AWS SDKs 中實作常見案例。這些案例示範如何呼叫 Amazon Comprehend 中的多個函數，或與其他 AWS 服務結合，藉以完成特定任務。每個案例均包含完整原始碼的連結，您可在連結中找到如何設定和執行程式碼的相關指示。
案例的目標是獲得中等水平的經驗，協助您了解內容中的服務動作。
**Topics**
+ [建置 Amazon Transcribe 串流應用程式](example_cross_TranscriptionStreamingApp_section.md)
+ [建置 Amazon Lex 聊天機器人](example_cross_LexChatbotLanguages_section.md)
+ [建立即時通訊軟體](example_cross_SQSMessageApp_section.md)
+ [建立應用程式以分析客戶意見回饋](example_cross_FSA_section.md)
+ [偵測文件元素](example_comprehend_Usage_DetectApis_section.md)
+ [偵測從影像擷取的文字中的實體](example_cross_TextractComprehendDetectEntities_section.md)
+ [針對範例資料執行主題建模任務](example_comprehend_Usage_TopicModeler_section.md)
+ [訓練自訂分類器並分類文件](example_comprehend_Usage_ComprehendClassifier_section.md)
# 建置 Amazon Transcribe 串流應用程式
下面的程式碼範例說明如何建置可即時記錄、轉錄和翻譯直播音訊並透過電子郵件傳送結果的應用程式。
------
#### [ JavaScript ]
**適用於 JavaScript (v3) 的 SDK**
說明如何使用 Amazon Transcribe 建置應用程式，該應用程式可即時記錄、轉錄和翻譯直播音訊，並可使用 Amazon Simple Email Service (Amazon SES) 透過電子郵件傳送結果。
如需完整的原始碼和如何設定及執行的指示，請參閱 [GitHub](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/javascriptv3/example_code/cross-services/transcribe-streaming-app) 上的完整範例。
**此範例中使用的服務**
+ Amazon Comprehend
+ Amazon SES
+ Amazon Transcribe
+ Amazon Translate
------
如需 AWS SDK 開發人員指南和程式碼範例的完整清單，請參閱 [搭配 SDK 使用 Amazon Comprehend AWS](sdk-general-information-section.md)。此主題也包含有關入門的資訊和舊版 SDK 的詳細資訊。
# 建立 Amazon Lex 聊天機器人與網站訪客互動
下列程式碼範例示範如何建立聊天機器人，與網站訪客互動。
------
#### [ Java ]
**適用於 Java 2.x 的 SDK **
示範如何使用 Amazon Lex API 在 Web 應用程式中建立 Chatbot，與網站訪客的互動。
如需完整的原始碼和如何設定及執行的指示，請參閱 [GitHub](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/javav2/usecases/creating_lex_chatbot) 上的完整範例。
**此範例中使用的服務**
+ Amazon Comprehend
+ Amazon Lex
+ Amazon Translate
------
#### [ JavaScript ]
**適用於 JavaScript (v3) 的 SDK**
示範如何使用 Amazon Lex API 在 Web 應用程式中建立 Chatbot，與網站訪客的互動。
如需完整的原始程式碼和如何設定和執行的指示，請參閱 適用於 JavaScript 的 AWS SDK 開發人員指南中的[建置 Amazon Lex 聊天機器人](https://docs.aws.amazon.com/sdk-for-javascript/v3/developer-guide/lex-bot-example.html)完整範例。
**此範例中使用的服務**
+ Amazon Comprehend
+ Amazon Lex
+ Amazon Translate
------
如需 AWS SDK 開發人員指南和程式碼範例的完整清單，請參閱 [搭配 SDK 使用 Amazon Comprehend AWS](sdk-general-information-section.md)。此主題也包含有關入門的資訊和舊版 SDK 的詳細資訊。
# 建立 Web 應用程式，以使用 Amazon SQS 傳送和擷取訊息
下列程式碼範例示範如何搭配使用 Amazon SQS 建立即時通訊軟體。
------
#### [ Java ]
**適用於 Java 2.x 的 SDK **
示範如何使用 Amazon SQS API 開發用於傳送和擷取訊息的 Spring REST API。
如需完整的原始碼和如何設定及執行的指示，請參閱 [GitHub](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/javav2/usecases/creating_message_application) 上的完整範例。
**此範例中使用的服務**
+ Amazon Comprehend
+ Amazon SQS
------
#### [ Kotlin ]
**SDK for Kotlin**
示範如何使用 Amazon SQS API 開發用於傳送和擷取訊息的 Spring REST API。
如需完整的原始碼和如何設定及執行的指示，請參閱 [GitHub](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/kotlin/usecases/creating_message_application) 上的完整範例。
**此範例中使用的服務**
+ Amazon Comprehend
+ Amazon SQS
------
如需 AWS SDK 開發人員指南和程式碼範例的完整清單，請參閱 [搭配 SDK 使用 Amazon Comprehend AWS](sdk-general-information-section.md)。此主題也包含有關入門的資訊和舊版 SDK 的詳細資訊。
# 建立可分析客戶意見回饋並合成音訊的應用程式
下列程式碼範例示範如何建立應用程式，以分析客戶評論卡、從其原始語言進行翻譯、判斷對方情緒，以及透過翻譯後的文字產生音訊檔案。
------
#### [ .NET ]
**適用於 .NET 的 SDK**
此範例應用程式會分析和存儲客戶的意見回饋卡。具體來說，它滿足了紐約市一家虛構飯店的需求。飯店以實體評論卡的形式收到賓客以各種語言撰寫的意見回饋。這些意見回饋透過 Web 用戶端上傳至應用程式。評論卡的影像上傳後，系統會執行下列步驟：
+ 文字內容是使用 Amazon Textract 從影像中擷取。
+ Amazon Comprehend 會決定擷取文字及其用語的情感。
+ 擷取的文字內容會使用 Amazon Translate 翻譯成英文。
+ Amazon Polly 會使用擷取的文字內容合成音訊檔案。
完整的應用程式可透過 AWS CDK 部署。如需原始程式碼和部署的說明，請參閱 [GitHub](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/dotnetv3/cross-service/FeedbackSentimentAnalyzer) 中的專案。
**此範例中使用的服務**
+ Amazon Comprehend
+ Lambda
+ Amazon Polly
+ Amazon Textract
+ Amazon Translate
------
#### [ Java ]
**適用於 Java 2.x 的 SDK**
此範例應用程式會分析和存儲客戶的意見回饋卡。具體來說，它滿足了紐約市一家虛構飯店的需求。飯店以實體評論卡的形式收到賓客以各種語言撰寫的意見回饋。這些意見回饋透過 Web 用戶端上傳至應用程式。評論卡的影像上傳後，系統會執行下列步驟：
+ 文字內容是使用 Amazon Textract 從影像中擷取。
+ Amazon Comprehend 會決定擷取文字及其用語的情感。
+ 擷取的文字內容會使用 Amazon Translate 翻譯成英文。
+ Amazon Polly 會使用擷取的文字內容合成音訊檔案。
完整的應用程式可透過 AWS CDK 部署。如需原始程式碼和部署的說明，請參閱 [GitHub](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/javav2/usecases/creating_fsa_app) 中的專案。
**此範例中使用的服務**
+ Amazon Comprehend
+ Lambda
+ Amazon Polly
+ Amazon Textract
+ Amazon Translate
------
#### [ JavaScript ]
**適用於 JavaScript (v3) 的 SDK**
此範例應用程式會分析和存儲客戶的意見回饋卡。具體來說，它滿足了紐約市一家虛構飯店的需求。飯店以實體評論卡的形式收到賓客以各種語言撰寫的意見回饋。這些意見回饋透過 Web 用戶端上傳至應用程式。評論卡的影像上傳後，系統會執行下列步驟：
+ 文字內容是使用 Amazon Textract 從影像中擷取。
+ Amazon Comprehend 會決定擷取文字及其用語的情感。
+ 擷取的文字內容會使用 Amazon Translate 翻譯成英文。
+ Amazon Polly 會使用擷取的文字內容合成音訊檔案。
完整的應用程式可透過 AWS CDK 部署。如需原始程式碼和部署的說明，請參閱 [GitHub](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/javascriptv3/example_code/cross-services/feedback-sentiment-analyzer) 中的專案。以下摘錄顯示如何在 Lambda 函數內 適用於 JavaScript 的 AWS SDK 使用 。
```
import {
ComprehendClient,
DetectDominantLanguageCommand,
DetectSentimentCommand,
} from "@aws-sdk/client-comprehend";
/**
* Determine the language and sentiment of the extracted text.
*
* @param {{ source_text: string}} extractTextOutput
*/
export const handler = async (extractTextOutput) => {
const comprehendClient = new ComprehendClient({});
const detectDominantLanguageCommand = new DetectDominantLanguageCommand({
Text: extractTextOutput.source_text,
});
// The source language is required for sentiment analysis and
// translation in the next step.
const { Languages } = await comprehendClient.send(
detectDominantLanguageCommand,
);
const languageCode = Languages[0].LanguageCode;
const detectSentimentCommand = new DetectSentimentCommand({
Text: extractTextOutput.source_text,
LanguageCode: languageCode,
});
const { Sentiment } = await comprehendClient.send(detectSentimentCommand);
return {
sentiment: Sentiment,
language_code: languageCode,
};
};
```
```
import {
DetectDocumentTextCommand,
TextractClient,
} from "@aws-sdk/client-textract";
/**
* Fetch the S3 object from the event and analyze it using Amazon Textract.
*
* @param {import("@types/aws-lambda").EventBridgeEvent<"Object Created">} eventBridgeS3Event
*/
export const handler = async (eventBridgeS3Event) => {
const textractClient = new TextractClient();
const detectDocumentTextCommand = new DetectDocumentTextCommand({
Document: {
S3Object: {
Bucket: eventBridgeS3Event.bucket,
Name: eventBridgeS3Event.object,
},
},
});
// Textract returns a list of blocks. A block can be a line, a page, word, etc.
// Each block also contains geometry of the detected text.
// For more information on the Block type, see https://docs.aws.amazon.com/textract/latest/dg/API_Block.html.
const { Blocks } = await textractClient.send(detectDocumentTextCommand);
// For the purpose of this example, we are only interested in words.
const extractedWords = Blocks.filter((b) => b.BlockType === "WORD").map(
(b) => b.Text,
);
return extractedWords.join(" ");
};
```
```
import { PollyClient, SynthesizeSpeechCommand } from "@aws-sdk/client-polly";
import { S3Client } from "@aws-sdk/client-s3";
import { Upload } from "@aws-sdk/lib-storage";
/**
* Synthesize an audio file from text.
*
* @param {{ bucket: string, translated_text: string, object: string}} sourceDestinationConfig
*/
export const handler = async (sourceDestinationConfig) => {
const pollyClient = new PollyClient({});
const synthesizeSpeechCommand = new SynthesizeSpeechCommand({
Engine: "neural",
Text: sourceDestinationConfig.translated_text,
VoiceId: "Ruth",
OutputFormat: "mp3",
});
const { AudioStream } = await pollyClient.send(synthesizeSpeechCommand);
const audioKey = `${sourceDestinationConfig.object}.mp3`;
// Store the audio file in S3.
const s3Client = new S3Client();
const upload = new Upload({
client: s3Client,
params: {
Bucket: sourceDestinationConfig.bucket,
Key: audioKey,
Body: AudioStream,
ContentType: "audio/mp3",
},
});
await upload.done();
return audioKey;
};
```
```
import {
TranslateClient,
TranslateTextCommand,
} from "@aws-sdk/client-translate";
/**
* Translate the extracted text to English.
*
* @param {{ extracted_text: string, source_language_code: string}} textAndSourceLanguage
*/
export const handler = async (textAndSourceLanguage) => {
const translateClient = new TranslateClient({});
const translateCommand = new TranslateTextCommand({
SourceLanguageCode: textAndSourceLanguage.source_language_code,
TargetLanguageCode: "en",
Text: textAndSourceLanguage.extracted_text,
});
const { TranslatedText } = await translateClient.send(translateCommand);
return { translated_text: TranslatedText };
};
```
**此範例中使用的服務**
+ Amazon Comprehend
+ Lambda
+ Amazon Polly
+ Amazon Textract
+ Amazon Translate
------
#### [ Ruby ]
**SDK for Ruby**
此範例應用程式會分析和存儲客戶的意見回饋卡。具體來說，它滿足了紐約市一家虛構飯店的需求。飯店以實體評論卡的形式收到賓客以各種語言撰寫的意見回饋。這些意見回饋透過 Web 用戶端上傳至應用程式。評論卡的影像上傳後，系統會執行下列步驟：
+ 文字內容是使用 Amazon Textract 從影像中擷取。
+ Amazon Comprehend 會決定擷取文字及其用語的情感。
+ 擷取的文字內容會使用 Amazon Translate 翻譯成英文。
+ Amazon Polly 會使用擷取的文字內容合成音訊檔案。
完整的應用程式可透過 AWS CDK 部署。如需原始程式碼和部署的說明，請參閱 [GitHub](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/ruby/cross_service_examples/feedback_sentiment_analyzer) 中的專案。
**此範例中使用的服務**
+ Amazon Comprehend
+ Lambda
+ Amazon Polly
+ Amazon Textract
+ Amazon Translate
------
如需 AWS SDK 開發人員指南和程式碼範例的完整清單，請參閱 [搭配 SDK 使用 Amazon Comprehend AWS](sdk-general-information-section.md)。此主題也包含有關入門的資訊和舊版 SDK 的詳細資訊。
# 使用 Amazon Comprehend 和 AWS SDK 偵測文件元素
以下程式碼範例顯示做法：
+ 偵測文件中的語言、實體和關鍵片語。
+ 偵測文件中的個人身分識別資訊 (PII)。
+ 偵測文件的情緒。
+ 偵測文件中的語法元素。
------
#### [ Python ]
**適用於 Python 的 SDK (Boto3)**
GitHub 上提供更多範例。尋找完整範例，並了解如何在 [AWS 程式碼範例儲存庫](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/python/example_code/comprehend#code-examples)中設定和執行。
建立包裝 Amazon Comprehend 動作的類別。
```
import logging
from pprint import pprint
import boto3
from botocore.exceptions import ClientError
logger = logging.getLogger(__name__)
class ComprehendDetect:
"""Encapsulates Comprehend detection functions."""
def __init__(self, comprehend_client):
"""
:param comprehend_client: A Boto3 Comprehend client.
"""
self.comprehend_client = comprehend_client
def detect_languages(self, text):
"""
Detects languages used in a document.
:param text: The document to inspect.
:return: The list of languages along with their confidence scores.
"""
try:
response = self.comprehend_client.detect_dominant_language(Text=text)
languages = response["Languages"]
logger.info("Detected %s languages.", len(languages))
except ClientError:
logger.exception("Couldn't detect languages.")
raise
else:
return languages
def detect_entities(self, text, language_code):
"""
Detects entities in a document. Entities can be things like people and places
or other common terms.
:param text: The document to inspect.
:param language_code: The language of the document.
:return: The list of entities along with their confidence scores.
"""
try:
response = self.comprehend_client.detect_entities(
Text=text, LanguageCode=language_code
)
entities = response["Entities"]
logger.info("Detected %s entities.", len(entities))
except ClientError:
logger.exception("Couldn't detect entities.")
raise
else:
return entities
def detect_key_phrases(self, text, language_code):
"""
Detects key phrases in a document. A key phrase is typically a noun and its
modifiers.
:param text: The document to inspect.
:param language_code: The language of the document.
:return: The list of key phrases along with their confidence scores.
"""
try:
response = self.comprehend_client.detect_key_phrases(
Text=text, LanguageCode=language_code
)
phrases = response["KeyPhrases"]
logger.info("Detected %s phrases.", len(phrases))
except ClientError:
logger.exception("Couldn't detect phrases.")
raise
else:
return phrases
def detect_pii(self, text, language_code):
"""
Detects personally identifiable information (PII) in a document. PII can be
things like names, account numbers, or addresses.
:param text: The document to inspect.
:param language_code: The language of the document.
:return: The list of PII entities along with their confidence scores.
"""
try:
response = self.comprehend_client.detect_pii_entities(
Text=text, LanguageCode=language_code
)
entities = response["Entities"]
logger.info("Detected %s PII entities.", len(entities))
except ClientError:
logger.exception("Couldn't detect PII entities.")
raise
else:
return entities
def detect_sentiment(self, text, language_code):
"""
Detects the overall sentiment expressed in a document. Sentiment can
be positive, negative, neutral, or a mixture.
:param text: The document to inspect.
:param language_code: The language of the document.
:return: The sentiments along with their confidence scores.
"""
try:
response = self.comprehend_client.detect_sentiment(
Text=text, LanguageCode=language_code
)
logger.info("Detected primary sentiment %s.", response["Sentiment"])
except ClientError:
logger.exception("Couldn't detect sentiment.")
raise
else:
return response
def detect_syntax(self, text, language_code):
"""
Detects syntactical elements of a document. Syntax tokens are portions of
text along with their use as parts of speech, such as nouns, verbs, and
interjections.
:param text: The document to inspect.
:param language_code: The language of the document.
:return: The list of syntax tokens along with their confidence scores.
"""
try:
response = self.comprehend_client.detect_syntax(
Text=text, LanguageCode=language_code
)
tokens = response["SyntaxTokens"]
logger.info("Detected %s syntax tokens.", len(tokens))
except ClientError:
logger.exception("Couldn't detect syntax.")
raise
else:
return tokens
```
呼叫在包裝函式類別上的函數，以偵測文件中的實體、片語等。
```
def usage_demo():
print("-" * 88)
print("Welcome to the Amazon Comprehend detection demo!")
print("-" * 88)
logging.basicConfig(level=logging.INFO, format="%(levelname)s: %(message)s")
comp_detect = ComprehendDetect(boto3.client("comprehend"))
with open("detect_sample.txt") as sample_file:
sample_text = sample_file.read()
demo_size = 3
print("Sample text used for this demo:")
print("-" * 88)
print(sample_text)
print("-" * 88)
print("Detecting languages.")
languages = comp_detect.detect_languages(sample_text)
pprint(languages)
lang_code = languages[0]["LanguageCode"]
print("Detecting entities.")
entities = comp_detect.detect_entities(sample_text, lang_code)
print(f"The first {demo_size} are:")
pprint(entities[:demo_size])
print("Detecting key phrases.")
phrases = comp_detect.detect_key_phrases(sample_text, lang_code)
print(f"The first {demo_size} are:")
pprint(phrases[:demo_size])
print("Detecting personally identifiable information (PII).")
pii_entities = comp_detect.detect_pii(sample_text, lang_code)
print(f"The first {demo_size} are:")
pprint(pii_entities[:demo_size])
print("Detecting sentiment.")
sentiment = comp_detect.detect_sentiment(sample_text, lang_code)
print(f"Sentiment: {sentiment['Sentiment']}")
print("SentimentScore:")
pprint(sentiment["SentimentScore"])
print("Detecting syntax elements.")
syntax_tokens = comp_detect.detect_syntax(sample_text, lang_code)
print(f"The first {demo_size} are:")
pprint(syntax_tokens[:demo_size])
print("Thanks for watching!")
print("-" * 88)
```
+ 如需 API 詳細資訊，請參閱《適用於 Python (Boto3) 的AWS SDK API 參考》**中的下列主題。
+ [DetectDominantLanguage](https://docs.aws.amazon.com/goto/boto3/comprehend-2017-11-27/DetectDominantLanguage)
+ [DetectEntities](https://docs.aws.amazon.com/goto/boto3/comprehend-2017-11-27/DetectEntities)
+ [DetectKeyPhrases](https://docs.aws.amazon.com/goto/boto3/comprehend-2017-11-27/DetectKeyPhrases)
+ [DetectPiiEntities](https://docs.aws.amazon.com/goto/boto3/comprehend-2017-11-27/DetectPiiEntities)
+ [DetectSentiment](https://docs.aws.amazon.com/goto/boto3/comprehend-2017-11-27/DetectSentiment)
+ [DetectSyntax](https://docs.aws.amazon.com/goto/boto3/comprehend-2017-11-27/DetectSyntax)
------
如需 AWS SDK 開發人員指南和程式碼範例的完整清單，請參閱 [搭配 SDK 使用 Amazon Comprehend AWS](sdk-general-information-section.md)。此主題也包含有關入門的資訊和舊版 SDK 的詳細資訊。
# 使用 AWS SDK 偵測從映像擷取的文字中的實體
下列程式碼範例示範如何使用 Amazon Comprehend 偵測 Amazon Textract 從存放在 Amazon S3 中的影像中提取的文字中的實體。
------
#### [ Python ]
**適用於 Python 的 SDK (Boto3)**
顯示如何在 Jupyter 筆記本 適用於 Python (Boto3) 的 AWS SDK 中使用 來偵測從影像擷取的文字中的實體。本範例使用 Amazon Textract 從儲存於 Amazon Simple Storage Service (Amazon S3) 和 Amazon Comprehend 中的影像提取文字，以偵測擷取文字中的實體。
此範例是 Jupyter 的筆記型電腦，必須在可以託管的筆記型電腦的環境中運行。如需使用 Amazon SageMaker AI 執行範例的指示，請參閱 [TextractAndComprehendNotebook.ipynb](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/python/cross_service/textract_comprehend_notebook/TextractAndComprehendNotebook.ipynb) 中的說明。
如需完整的原始碼和如何設定及執行的指示，請參閱 [GitHub](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/python/cross_service/textract_comprehend_notebook#readme) 上的完整範例。
**此範例中使用的服務**
+ Amazon Comprehend
+ Amazon S3
+ Amazon Textract
------
如需 AWS SDK 開發人員指南和程式碼範例的完整清單，請參閱 [搭配 SDK 使用 Amazon Comprehend AWS](sdk-general-information-section.md)。此主題也包含有關入門的資訊和舊版 SDK 的詳細資訊。
# 使用 SDK 在範例資料上執行 Amazon Comprehend 主題建模任務 AWS
以下程式碼範例顯示做法：
+ 對範例資料執行 Amazon Comprehend 主題建模任務。
+ 取得任務相關資訊。
+ 從 Amazon S3 擷取任務輸出資料。
------
#### [ Python ]
**適用於 Python 的 SDK (Boto3)**
GitHub 上提供更多範例。尋找完整範例，並了解如何在 [AWS 程式碼範例儲存庫](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/python/example_code/comprehend#code-examples)中設定和執行。
建立包裝函式類別以呼叫 Amazon Comprehend 主題建模動作。
```
class ComprehendTopicModeler:
"""Encapsulates a Comprehend topic modeler."""
def __init__(self, comprehend_client):
"""
:param comprehend_client: A Boto3 Comprehend client.
"""
self.comprehend_client = comprehend_client
def start_job(
self,
job_name,
input_bucket,
input_key,
input_format,
output_bucket,
output_key,
data_access_role_arn,
):
"""
Starts a topic modeling job. Input is read from the specified Amazon S3
input bucket and written to the specified output bucket. Output data is stored
in a tar archive compressed in gzip format. The job runs asynchronously, so you
can call `describe_topics_detection_job` to get job status until it
returns a status of SUCCEEDED.
:param job_name: The name of the job.
:param input_bucket: An Amazon S3 bucket that contains job input.
:param input_key: The prefix used to find input data in the input
bucket. If multiple objects have the same prefix, all
of them are used.
:param input_format: The format of the input data, either one document per
file or one document per line.
:param output_bucket: The Amazon S3 bucket where output data is written.
:param output_key: The prefix prepended to the output data.
:param data_access_role_arn: The Amazon Resource Name (ARN) of a role that
grants Comprehend permission to read from the
input bucket and write to the output bucket.
:return: Information about the job, including the job ID.
"""
try:
response = self.comprehend_client.start_topics_detection_job(
JobName=job_name,
DataAccessRoleArn=data_access_role_arn,
InputDataConfig={
"S3Uri": f"s3://{input_bucket}/{input_key}",
"InputFormat": input_format.value,
},
OutputDataConfig={"S3Uri": f"s3://{output_bucket}/{output_key}"},
)
logger.info("Started topic modeling job %s.", response["JobId"])
except ClientError:
logger.exception("Couldn't start topic modeling job.")
raise
else:
return response
def describe_job(self, job_id):
"""
Gets metadata about a topic modeling job.
:param job_id: The ID of the job to look up.
:return: Metadata about the job.
"""
try:
response = self.comprehend_client.describe_topics_detection_job(
JobId=job_id
)
job = response["TopicsDetectionJobProperties"]
logger.info("Got topic detection job %s.", job_id)
except ClientError:
logger.exception("Couldn't get topic detection job %s.", job_id)
raise
else:
return job
def list_jobs(self):
"""
Lists topic modeling jobs for the current account.
:return: The list of jobs.
"""
try:
response = self.comprehend_client.list_topics_detection_jobs()
jobs = response["TopicsDetectionJobPropertiesList"]
logger.info("Got %s topic detection jobs.", len(jobs))
except ClientError:
logger.exception("Couldn't get topic detection jobs.")
raise
else:
return jobs
```
使用包裝函式類別來執行主題建模任務，並取得任務資料。
```
def usage_demo():
print("-" * 88)
print("Welcome to the Amazon Comprehend topic modeling demo!")
print("-" * 88)
logging.basicConfig(level=logging.INFO, format="%(levelname)s: %(message)s")
input_prefix = "input/"
output_prefix = "output/"
demo_resources = ComprehendDemoResources(
boto3.resource("s3"), boto3.resource("iam")
)
topic_modeler = ComprehendTopicModeler(boto3.client("comprehend"))
print("Setting up storage and security resources needed for the demo.")
demo_resources.setup("comprehend-topic-modeler-demo")
print("Copying sample data from public bucket into input bucket.")
demo_resources.bucket.copy(
{"Bucket": "public-sample-us-west-2", "Key": "TopicModeling/Sample.txt"},
f"{input_prefix}sample.txt",
)
print("Starting topic modeling job on sample data.")
job_info = topic_modeler.start_job(
"demo-topic-modeling-job",
demo_resources.bucket.name,
input_prefix,
JobInputFormat.per_line,
demo_resources.bucket.name,
output_prefix,
demo_resources.data_access_role.arn,
)
print(
f"Waiting for job {job_info['JobId']} to complete. This typically takes "
f"20 - 30 minutes."
)
job_waiter = JobCompleteWaiter(topic_modeler.comprehend_client)
job_waiter.wait(job_info["JobId"])
job = topic_modeler.describe_job(job_info["JobId"])
print(f"Job {job['JobId']} complete:")
pprint(job)
print(
f"Getting job output data from the output Amazon S3 bucket: "
f"{job['OutputDataConfig']['S3Uri']}."
)
job_output = demo_resources.extract_job_output(job)
lines = 10
print(f"First {lines} lines of document topics output:")
pprint(job_output["doc-topics.csv"]["data"][:lines])
print(f"First {lines} lines of terms output:")
pprint(job_output["topic-terms.csv"]["data"][:lines])
print("Cleaning up resources created for the demo.")
demo_resources.cleanup()
print("Thanks for watching!")
print("-" * 88)
```
+ 如需 API 詳細資訊，請參閱《適用於 Python (Boto3) 的AWS SDK API 參考》**中的下列主題。
+ [DescribeTopicsDetectionJob](https://docs.aws.amazon.com/goto/boto3/comprehend-2017-11-27/DescribeTopicsDetectionJob)
+ [ListTopicsDetectionJobs](https://docs.aws.amazon.com/goto/boto3/comprehend-2017-11-27/ListTopicsDetectionJobs)
+ [StartTopicsDetectionJob](https://docs.aws.amazon.com/goto/boto3/comprehend-2017-11-27/StartTopicsDetectionJob)
------
如需 AWS SDK 開發人員指南和程式碼範例的完整清單，請參閱 [搭配 SDK 使用 Amazon Comprehend AWS](sdk-general-information-section.md)。此主題也包含有關入門的資訊和舊版 SDK 的詳細資訊。
# 訓練自訂 Amazon Comprehend 分類器，並使用 AWS SDK 分類文件
以下程式碼範例顯示做法：
+ 建立 Amazon Comprehend 多標籤分類器。
+ 根據範例資料訓練分類器。
+ 在第二組資料上執行分類任務。
+ 從 Amazon S3 擷取任務輸出資料。
------
#### [ Python ]
**適用於 Python 的 SDK (Boto3)**
GitHub 上提供更多範例。尋找完整範例，並了解如何在 [AWS 程式碼範例儲存庫](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/python/example_code/comprehend#code-examples)中設定和執行。
建立包裝函式類別以呼叫 Amazon Comprehend 文件分類器動作。
```
class ComprehendClassifier:
"""Encapsulates an Amazon Comprehend custom classifier."""
def __init__(self, comprehend_client):
"""
:param comprehend_client: A Boto3 Comprehend client.
"""
self.comprehend_client = comprehend_client
self.classifier_arn = None
def create(
self,
name,
language_code,
training_bucket,
training_key,
data_access_role_arn,
mode,
):
"""
Creates a custom classifier. After the classifier is created, it immediately
starts training on the data found in the specified Amazon S3 bucket. Training
can take 30 minutes or longer. The `describe_document_classifier` function
can be used to get training status and returns a status of TRAINED when the
classifier is ready to use.
:param name: The name of the classifier.
:param language_code: The language the classifier can operate on.
:param training_bucket: The Amazon S3 bucket that contains the training data.
:param training_key: The prefix used to find training data in the training
bucket. If multiple objects have the same prefix, all
of them are used.
:param data_access_role_arn: The Amazon Resource Name (ARN) of a role that
grants Comprehend permission to read from the
training bucket.
:return: The ARN of the newly created classifier.
"""
try:
response = self.comprehend_client.create_document_classifier(
DocumentClassifierName=name,
LanguageCode=language_code,
InputDataConfig={"S3Uri": f"s3://{training_bucket}/{training_key}"},
DataAccessRoleArn=data_access_role_arn,
Mode=mode.value,
)
self.classifier_arn = response["DocumentClassifierArn"]
logger.info("Started classifier creation. Arn is: %s.", self.classifier_arn)
except ClientError:
logger.exception("Couldn't create classifier %s.", name)
raise
else:
return self.classifier_arn
def describe(self, classifier_arn=None):
"""
Gets metadata about a custom classifier, including its current status.
:param classifier_arn: The ARN of the classifier to look up.
:return: Metadata about the classifier.
"""
if classifier_arn is not None:
self.classifier_arn = classifier_arn
try:
response = self.comprehend_client.describe_document_classifier(
DocumentClassifierArn=self.classifier_arn
)
classifier = response["DocumentClassifierProperties"]
logger.info("Got classifier %s.", self.classifier_arn)
except ClientError:
logger.exception("Couldn't get classifier %s.", self.classifier_arn)
raise
else:
return classifier
def list(self):
"""
Lists custom classifiers for the current account.
:return: The list of classifiers.
"""
try:
response = self.comprehend_client.list_document_classifiers()
classifiers = response["DocumentClassifierPropertiesList"]
logger.info("Got %s classifiers.", len(classifiers))
except ClientError:
logger.exception(
"Couldn't get classifiers.",
)
raise
else:
return classifiers
def delete(self):
"""
Deletes the classifier.
"""
try:
self.comprehend_client.delete_document_classifier(
DocumentClassifierArn=self.classifier_arn
)
logger.info("Deleted classifier %s.", self.classifier_arn)
self.classifier_arn = None
except ClientError:
logger.exception("Couldn't deleted classifier %s.", self.classifier_arn)
raise
def start_job(
self,
job_name,
input_bucket,
input_key,
input_format,
output_bucket,
output_key,
data_access_role_arn,
):
"""
Starts a classification job. The classifier must be trained or the job
will fail. Input is read from the specified Amazon S3 input bucket and
written to the specified output bucket. Output data is stored in a tar
archive compressed in gzip format. The job runs asynchronously, so you can
call `describe_document_classification_job` to get job status until it
returns a status of SUCCEEDED.
:param job_name: The name of the job.
:param input_bucket: The Amazon S3 bucket that contains input data.
:param input_key: The prefix used to find input data in the input
bucket. If multiple objects have the same prefix, all
of them are used.
:param input_format: The format of the input data, either one document per
file or one document per line.
:param output_bucket: The Amazon S3 bucket where output data is written.
:param output_key: The prefix prepended to the output data.
:param data_access_role_arn: The Amazon Resource Name (ARN) of a role that
grants Comprehend permission to read from the
input bucket and write to the output bucket.
:return: Information about the job, including the job ID.
"""
try:
response = self.comprehend_client.start_document_classification_job(
DocumentClassifierArn=self.classifier_arn,
JobName=job_name,
InputDataConfig={
"S3Uri": f"s3://{input_bucket}/{input_key}",
"InputFormat": input_format.value,
},
OutputDataConfig={"S3Uri": f"s3://{output_bucket}/{output_key}"},
DataAccessRoleArn=data_access_role_arn,
)
logger.info(
"Document classification job %s is %s.", job_name, response["JobStatus"]
)
except ClientError:
logger.exception("Couldn't start classification job %s.", job_name)
raise
else:
return response
def describe_job(self, job_id):
"""
Gets metadata about a classification job.
:param job_id: The ID of the job to look up.
:return: Metadata about the job.
"""
try:
response = self.comprehend_client.describe_document_classification_job(
JobId=job_id
)
job = response["DocumentClassificationJobProperties"]
logger.info("Got classification job %s.", job["JobName"])
except ClientError:
logger.exception("Couldn't get classification job %s.", job_id)
raise
else:
return job
def list_jobs(self):
"""
Lists the classification jobs for the current account.
:return: The list of jobs.
"""
try:
response = self.comprehend_client.list_document_classification_jobs()
jobs = response["DocumentClassificationJobPropertiesList"]
logger.info("Got %s document classification jobs.", len(jobs))
except ClientError:
logger.exception(
"Couldn't get document classification jobs.",
)
raise
else:
return jobs
```
建立類別以協助執行案例。
```
class ClassifierDemo:
"""
Encapsulates functions used to run the demonstration.
"""
def __init__(self, demo_resources):
"""
:param demo_resources: A ComprehendDemoResources class that manages resources
for the demonstration.
"""
self.demo_resources = demo_resources
self.training_prefix = "training/"
self.input_prefix = "input/"
self.input_format = JobInputFormat.per_line
self.output_prefix = "output/"
def setup(self):
"""Creates AWS resources used by the demo."""
self.demo_resources.setup("comprehend-classifier-demo")
def cleanup(self):
"""Deletes AWS resources used by the demo."""
self.demo_resources.cleanup()
@staticmethod
def _sanitize_text(text):
"""Removes characters that cause errors for the document parser."""
return text.replace("\r", " ").replace("\n", " ").replace(",", ";")
@staticmethod
def _get_issues(query, issue_count):
"""
Gets issues from GitHub using the specified query parameters.
:param query: The query string used to request issues from the GitHub API.
:param issue_count: The number of issues to retrieve.
:return: The list of issues retrieved from GitHub.
"""
issues = []
logger.info("Requesting issues from %s?%s.", GITHUB_SEARCH_URL, query)
response = requests.get(f"{GITHUB_SEARCH_URL}?{query}&per_page={issue_count}")
if response.status_code == 200:
issue_page = response.json()["items"]
logger.info("Got %s issues.", len(issue_page))
issues = [
{
"title": ClassifierDemo._sanitize_text(issue["title"]),
"body": ClassifierDemo._sanitize_text(issue["body"]),
"labels": {label["name"] for label in issue["labels"]},
}
for issue in issue_page
]
else:
logger.error(
"GitHub returned error code %s with message %s.",
response.status_code,
response.json(),
)
logger.info("Found %s issues.", len(issues))
return issues
def get_training_issues(self, training_labels):
"""
Gets issues used for training the custom classifier. Training issues are
closed issues from the Boto3 repo that have known labels. Comprehend
requires a minimum of ten training issues per label.
:param training_labels: The issue labels to use for training.
:return: The set of issues used for training.
"""
issues = []
per_label_count = 15
for label in training_labels:
issues += self._get_issues(
f"q=type:issue+repo:boto/boto3+state:closed+label:{label}",
per_label_count,
)
for issue in issues:
issue["labels"] = issue["labels"].intersection(training_labels)
return issues
def get_input_issues(self, training_labels):
"""
Gets input issues from GitHub. For demonstration purposes, input issues
are open issues from the Boto3 repo with known labels, though in practice
any issue could be submitted to the classifier for labeling.
:param training_labels: The set of labels to query for.
:return: The set of issues used for input.
"""
issues = []
per_label_count = 5
for label in training_labels:
issues += self._get_issues(
f"q=type:issue+repo:boto/boto3+state:open+label:{label}",
per_label_count,
)
return issues
def upload_issue_data(self, issues, training=False):
"""
Uploads issue data to an Amazon S3 bucket, either for training or for input.
The data is first put into the format expected by Comprehend. For training,
the set of pipe-delimited labels is prepended to each document. For
input, labels are not sent.
:param issues: The set of issues to upload to Amazon S3.
:param training: Indicates whether the issue data is used for training or
input.
"""
try:
obj_key = (
self.training_prefix if training else self.input_prefix
) + "issues.txt"
if training:
issue_strings = [
f"{'|'.join(issue['labels'])},{issue['title']} {issue['body']}"
for issue in issues
]
else:
issue_strings = [
f"{issue['title']} {issue['body']}" for issue in issues
]
issue_bytes = BytesIO("\n".join(issue_strings).encode("utf-8"))
self.demo_resources.bucket.upload_fileobj(issue_bytes, obj_key)
logger.info(
"Uploaded data as %s to bucket %s.",
obj_key,
self.demo_resources.bucket.name,
)
except ClientError:
logger.exception(
"Couldn't upload data to bucket %s.", self.demo_resources.bucket.name
)
raise
def extract_job_output(self, job):
"""Extracts job output from Amazon S3."""
return self.demo_resources.extract_job_output(job)
@staticmethod
def reconcile_job_output(input_issues, output_dict):
"""
Reconciles job output with the list of input issues. Because the input issues
have known labels, these can be compared with the labels added by the
classifier to judge the accuracy of the output.
:param input_issues: The list of issues used as input.
:param output_dict: The dictionary of data that is output by the classifier.
:return: The list of reconciled input and output data.
"""
reconciled = []
for archive in output_dict.values():
for line in archive["data"]:
in_line = int(line["Line"])
in_labels = input_issues[in_line]["labels"]
out_labels = {
label["Name"]
for label in line["Labels"]
if float(label["Score"]) > 0.3
}
reconciled.append(
f"{line['File']}, line {in_line} has labels {in_labels}.\n"
f"\tClassifier assigned {out_labels}."
)
logger.info("Reconciled input and output labels.")
return reconciled
```
使用已知標籤在一組 GitHub 問題上訓練分類器，然後將第二組 GitHub 問題傳送到分類器，以便針對問題進行標記。
```
def usage_demo():
print("-" * 88)
print("Welcome to the Amazon Comprehend custom document classifier demo!")
print("-" * 88)
logging.basicConfig(level=logging.INFO, format="%(levelname)s: %(message)s")
comp_demo = ClassifierDemo(
ComprehendDemoResources(boto3.resource("s3"), boto3.resource("iam"))
)
comp_classifier = ComprehendClassifier(boto3.client("comprehend"))
classifier_trained_waiter = ClassifierTrainedWaiter(
comp_classifier.comprehend_client
)
training_labels = {"bug", "feature-request", "dynamodb", "s3"}
print("Setting up storage and security resources needed for the demo.")
comp_demo.setup()
print("Getting training data from GitHub and uploading it to Amazon S3.")
training_issues = comp_demo.get_training_issues(training_labels)
comp_demo.upload_issue_data(training_issues, True)
classifier_name = "doc-example-classifier"
print(f"Creating document classifier {classifier_name}.")
comp_classifier.create(
classifier_name,
"en",
comp_demo.demo_resources.bucket.name,
comp_demo.training_prefix,
comp_demo.demo_resources.data_access_role.arn,
ClassifierMode.multi_label,
)
print(
f"Waiting until {classifier_name} is trained. This typically takes "
f"30–40 minutes."
)
classifier_trained_waiter.wait(comp_classifier.classifier_arn)
print(f"Classifier {classifier_name} is trained:")
pprint(comp_classifier.describe())
print("Getting input data from GitHub and uploading it to Amazon S3.")
input_issues = comp_demo.get_input_issues(training_labels)
comp_demo.upload_issue_data(input_issues)
print("Starting classification job on input data.")
job_info = comp_classifier.start_job(
"issue_classification_job",
comp_demo.demo_resources.bucket.name,
comp_demo.input_prefix,
comp_demo.input_format,
comp_demo.demo_resources.bucket.name,
comp_demo.output_prefix,
comp_demo.demo_resources.data_access_role.arn,
)
print(f"Waiting for job {job_info['JobId']} to complete.")
job_waiter = JobCompleteWaiter(comp_classifier.comprehend_client)
job_waiter.wait(job_info["JobId"])
job = comp_classifier.describe_job(job_info["JobId"])
print(f"Job {job['JobId']} complete:")
pprint(job)
print(
f"Getting job output data from Amazon S3: "
f"{job['OutputDataConfig']['S3Uri']}."
)
job_output = comp_demo.extract_job_output(job)
print("Job output:")
pprint(job_output)
print("Reconciling job output with labels from GitHub:")
reconciled_output = comp_demo.reconcile_job_output(input_issues, job_output)
print(*reconciled_output, sep="\n")
answer = input(f"Do you want to delete the classifier {classifier_name} (y/n)? ")
if answer.lower() == "y":
print(f"Deleting {classifier_name}.")
comp_classifier.delete()
print("Cleaning up resources created for the demo.")
comp_demo.cleanup()
print("Thanks for watching!")
print("-" * 88)
```
+ 如需 API 詳細資訊，請參閱《適用於 Python (Boto3) 的AWS SDK API 參考》**中的下列主題。
+ [CreateDocumentClassifier](https://docs.aws.amazon.com/goto/boto3/comprehend-2017-11-27/CreateDocumentClassifier)
+ [DeleteDocumentClassifier](https://docs.aws.amazon.com/goto/boto3/comprehend-2017-11-27/DeleteDocumentClassifier)
+ [DescribeDocumentClassificationJob](https://docs.aws.amazon.com/goto/boto3/comprehend-2017-11-27/DescribeDocumentClassificationJob)
+ [DescribeDocumentClassifier](https://docs.aws.amazon.com/goto/boto3/comprehend-2017-11-27/DescribeDocumentClassifier)
+ [ListDocumentClassificationJobs](https://docs.aws.amazon.com/goto/boto3/comprehend-2017-11-27/ListDocumentClassificationJobs)
+ [ListDocumentClassifiers](https://docs.aws.amazon.com/goto/boto3/comprehend-2017-11-27/ListDocumentClassifiers)
+ [StartDocumentClassificationJob](https://docs.aws.amazon.com/goto/boto3/comprehend-2017-11-27/StartDocumentClassificationJob)
------
如需 AWS SDK 開發人員指南和程式碼範例的完整清單，請參閱 [搭配 SDK 使用 Amazon Comprehend AWS](sdk-general-information-section.md)。此主題也包含有關入門的資訊和舊版 SDK 的詳細資訊。