Skip to content

Commit 02c4434

Browse files
committed
Reduce default max output tokens for OpenAI
Changed the default value for OPENAI_MAX_OUTPUT_TOKENS from 128000 to 500 in AIModerator. This likely addresses performance or cost concerns by limiting the maximum number of output tokens generated.
1 parent 1bbd830 commit 02c4434

1 file changed

Lines changed: 1 addition & 1 deletion

File tree

app/services/ai/ai_moderator.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@ def __init__(self):
2121
self.model_context_window = int(
2222
cfg.get('OPENAI_CONTEXT_WINDOW', 272000))
2323
self.max_output_tokens = int(
24-
cfg.get('OPENAI_MAX_OUTPUT_TOKENS', 128000))
24+
cfg.get('OPENAI_MAX_OUTPUT_TOKENS', 500))
2525

2626
# Initialize tokenizer; prefer model-specific, fallback to cl100k_base
2727
try:

0 commit comments

Comments
 (0)