PromptModelInferenceConfiguration

Contains inference configurations related to model inference for a prompt. For more information, see Inference parameters.

Types

The maximum number of tokens to return in the response.

A list of strings that define sequences after which the model will stop generating.

Controls the randomness of the response. Choose a lower value for more predictable outputs and a higher value for more surprising outputs.

The percentage of most-likely candidates that the model considers for the next token.

open operator override fun equals(other: Any?): Boolean

open override fun hashCode(): Int

open override fun toString(): String