Interface InferenceConfiguration.Builder

All Superinterfaces:
Buildable, CopyableBuilder<InferenceConfiguration.Builder,InferenceConfiguration>, SdkBuilder<InferenceConfiguration.Builder,InferenceConfiguration>, SdkPojo
Enclosing class:
InferenceConfiguration
@Mutable @NotThreadSafe public static interface InferenceConfiguration.Builder extends SdkPojo, CopyableBuilder<InferenceConfiguration.Builder,InferenceConfiguration>

  • Method Details

    • maxTokens

      InferenceConfiguration.Builder maxTokens(Integer maxTokens)

      The maximum number of tokens to generate in the model response during evaluation.

      Parameters:
      maxTokens - The maximum number of tokens to generate in the model response during evaluation.
      Returns:
      Returns a reference to this object so that method calls can be chained together.

    • temperature

      InferenceConfiguration.Builder temperature(Float temperature)

      The temperature value that controls randomness in the model's responses. Lower values produce more deterministic outputs.

      Parameters:
      temperature - The temperature value that controls randomness in the model's responses. Lower values produce more deterministic outputs.
      Returns:
      Returns a reference to this object so that method calls can be chained together.

    • topP

      InferenceConfiguration.Builder topP(Float topP)

      The top-p sampling parameter that controls the diversity of the model's responses by limiting the cumulative probability of token choices.

      Parameters:
      topP - The top-p sampling parameter that controls the diversity of the model's responses by limiting the cumulative probability of token choices.
      Returns:
      Returns a reference to this object so that method calls can be chained together.

    • stopSequences

      InferenceConfiguration.Builder stopSequences(Collection<String> stopSequences)

      The list of sequences that will cause the model to stop generating tokens when encountered.

      Parameters:
      stopSequences - The list of sequences that will cause the model to stop generating tokens when encountered.
      Returns:
      Returns a reference to this object so that method calls can be chained together.

    • stopSequences

      InferenceConfiguration.Builder stopSequences(String... stopSequences)

      The list of sequences that will cause the model to stop generating tokens when encountered.

      Parameters:
      stopSequences - The list of sequences that will cause the model to stop generating tokens when encountered.
      Returns:
      Returns a reference to this object so that method calls can be chained together.