Skip to main content

Chat Models

Chat models are specialized versions of large language models designed to handle conversational contexts. They excel at generating human-like responses in interactive dialogue and maintaining context throughout a conversation, making them ideal for tasks like customer support, virtual assistance, and interactive applications. In contrast, Completion models are more suited for single-turn tasks that don't require maintaining a conversational history, such as text summarization, translation, or code generation. While both types of models can generate text based on the input they receive, Chat models are optimized for multi-turn dialogue and often exhibit a better understanding of nuanced conversational cues compared to their Completion counterparts.




Parameter NameStreaming SupportedTypeDescriptionMinMaxDefault
promptsTrueChatSessionText input to generate model response.
temperatureTruefloatThe temperature is used for sampling during the response generation. Controls the degree of randomness in token selection. Lower temperatures result in less randomness. Higher temperatures can lead to more diverse or creative results.010
maxOutputTokensTrueintMaximum number of tokens that can be generated in the response. Specify a lower value for shorter responses and a higher value for longer responses. The maximum value may be lower for certain models.120481024
topKTrueintTop-K changes how the model selects tokens for output. Specify a lower value for less random responses and a higher value for more random responses.0400
candidateCountFalseintTokens are selected from most probable to least until the sum of their probabilities equals the top_p value.181
stopSequencesTruearray of stringsSpecifies a list of strings that tells the model to stop generating text if one of the strings is encountered in the response. Strings are case-sensitive.
topPTruefloatTop-p changes how the model selects tokens for output. Tokens are selected from the most (see top-K) to least probable until the sum of their probabilities equals the top-P value.010.95
Was this helpful?