Settings for the model speculative decoding technique that's applied by a model optimization job.
Inheritance Hierarchy
Namespace: Amazon.SageMaker.Model
Assembly: AWSSDK.SageMaker.dll
Version: 3.x.y.z
Syntax
public class ModelSpeculativeDecodingConfig
The ModelSpeculativeDecodingConfig type exposes the following members
Constructors
| Name | Description | |
|---|---|---|
|
ModelSpeculativeDecodingConfig() |
Properties
| Name | Type | Description | |
|---|---|---|---|
|
|
Technique | Amazon.SageMaker.ModelSpeculativeDecodingTechnique |
Gets and sets the property Technique. The speculative decoding technique to apply during model optimization. |
|
|
TrainingDataSource | Amazon.SageMaker.Model.ModelSpeculativeDecodingTrainingDataSource |
Gets and sets the property TrainingDataSource. The location of the training data to use for speculative decoding. The data must be formatted as ShareGPT, OpenAI Completions or OpenAI Chat Completions. The input can also be unencrypted captured data from a SageMaker endpoint as long as the endpoint uses one of the above formats. |
Version Information
.NET:
Supported in: 8.0 and newer, Core 3.1
.NET Standard:
Supported in: 2.0
.NET Framework:
Supported in: 4.5 and newer, 3.5