Chat completion stream returns empty total usage
Problem
Confirm this is a Node library issue and not an underlying OpenAI API issue - [X] This is an issue with the Node library Describe the bug Using openai.beta.chat.completions.stream() and then calling totalUsage function returns the object with all values set to zero. To Reproduce 1. Call the chat completions API with response stream 2. Log the result of totalUsage function 3. Check that every value is zero Code snippets [code block] OS Ubuntu Node version Node v16.13.0 Library version open v4.19.0
Unverified for your environment
Select your OS to check compatibility.
1 Fix
Fix Empty Total Usage in Chat Completion Stream
The issue arises from the way the Node library handles the streaming response from the OpenAI API. When using the stream method, the total usage metrics may not be correctly populated because the library does not aggregate usage data during the streaming process. This results in the totalUsage function returning all zero values.
Awaiting Verification
Be the first to verify this fix
- 1
Update Library to Latest Version
Ensure you are using the latest version of the OpenAI Node library, as updates may include fixes for streaming issues.
bashnpm install openai@latest - 2
Modify Stream Handling to Capture Usage
Adjust the implementation of the chat completion stream to ensure that usage metrics are captured correctly. This can be done by listening to the stream events and manually aggregating the usage data.
javascriptconst { ChatCompletionStream } = require('openai'); let totalUsage = { prompt_tokens: 0, completion_tokens: 0, total_tokens: 0 }; const stream = ChatCompletionStream({ model: 'gpt-3.5-turbo', messages: [{ role: 'user', content: 'Hello!' }], stream: true }); stream.on('data', (data) => { totalUsage.prompt_tokens += data.usage.prompt_tokens; totalUsage.completion_tokens += data.usage.completion_tokens; totalUsage.total_tokens += data.usage.total_tokens; }); - 3
Log Total Usage After Stream Completion
Ensure that you log the total usage after the stream has completed to verify that the values are now populated correctly.
javascriptstream.on('end', () => { console.log('Total Usage:', totalUsage); }); - 4
Test with Different Inputs
Run tests with various input messages to ensure that the total usage metrics are being captured correctly across different scenarios.
javascript// Example test stream.write({ role: 'user', content: 'Test message' });
Validation
To confirm the fix worked, run the modified code and check the console logs for total usage values. They should reflect the actual usage metrics instead of zeros. Additionally, test with multiple messages to ensure consistent behavior.
Sign in to verify this fix
Environment
Submitted by
Alex Chen
2450 rep