FG
💻 Software🤖 AI & LLMsAnthropic

Wrong request options in retries

Fresh3 days ago
Mar 14, 20260 views
Confidence Score47%
47%

Problem

We are using `AsyncAnthropicBedrock` class. We noticed that when a request is automatically retried, the request options do not have the correct values, resulting in 400 - Bad request. Example logs (actual content redacted): [code block] This started happening with version 0.31.0. Doesn't happen with version 0.30.1. Maybe has something to do with https://github.com/anthropics/anthropic-sdk-python/pull/580 ?

Error Output

Exception	
2024-07-12 12:56:25.665	Traceback (most recent call last):

Unverified for your environment

Select your OS to check compatibility.

1 Fix

Canonical Fix
Unverified Fix
New Fix – Awaiting Verification

Fix Request Options in Retries for AsyncAnthropicBedrock

Medium Risk

The issue arises from changes introduced in version 0.31.0 of the AsyncAnthropicBedrock class, where the retry mechanism does not correctly preserve the original request options. This leads to incorrect parameters being sent during retries, resulting in a 400 - Bad Request error. The change in pull request #580 may have altered how request options are stored or retrieved during the retry process.

Awaiting Verification

Be the first to verify this fix

  1. 1

    Review Retry Logic Implementation

    Examine the retry logic in the AsyncAnthropicBedrock class to identify how request options are being handled. Ensure that the original request options are retained and reused during retries.

    python
    def retry_request(self, request_options):
    original_options = request_options.copy()  # Preserve original options
    for attempt in range(self.max_retries):
        response = self.send_request(original_options)
        if response.is_successful():
            return response
        self.handle_retry(response)
    raise Exception('Max retries exceeded')
  2. 2

    Modify Request Options Handling

    Update the code to ensure that the request options are correctly passed during retries. This may involve modifying how the request options are stored in the class or how they are retrieved during a retry.

    python
    def send_request(self, request_options):
    # Ensure request_options are correctly formatted
    if not self.validate_options(request_options):
        raise ValueError('Invalid request options')
    return self.api_client.send(request_options)
  3. 3

    Implement Unit Tests for Retry Logic

    Create unit tests that specifically test the retry logic to ensure that the correct request options are used on retries. This will help catch any future regressions related to this issue.

    python
    def test_retry_request_correct_options():
    original_options = {'param1': 'value1'}
    bedrock = AsyncAnthropicBedrock()
    response = bedrock.retry_request(original_options)
    assert response.request_options == original_options, 'Request options do not match expected values'
  4. 4

    Deploy and Monitor Changes

    Deploy the changes to a staging environment and monitor the logs for any 400 errors related to request options. Ensure that the fix resolves the issue without introducing new problems.

Validation

Confirm that the changes have resolved the issue by running the application and triggering retries. Monitor the logs for any 400 - Bad Request errors. Additionally, run the unit tests created in step 3 to ensure that the retry logic behaves as expected.

Sign in to verify this fix

Environment

Submitted by

AC

Alex Chen

2450 rep

Tags

claudeanthropicllmapibug