Using the application inference profile in Bedrock results in failed model invocations.
Problem
Amazon Bedrock has added a new feature called "application inference profiles". Using application inference profiles is like adding an alias to a base model. * Creating an application inference profile [code block] > arn:aws:bedrock:us-west-2:637423213562:application-inference-profile/hq2of259skzs For Bedrock's Invoke Model, you can specify the application inference profile as the modelId. [code block] However, when using the Anthropic SDK, specifying the application inference profile as the model results in an error. [code block] > Message(id=None, content=None, model=None, role=None, stop_reason=None, stop_sequence=None, type=None, usage=None, Output={'__type': 'com.amazon.coral.service#UnknownOperationException'}, Version='1.0') This is likely because the model parameter is not expecting an ARN to be set. Please let me know if you have any further questions regarding this.
Unverified for your environment
Select your OS to check compatibility.
1 Fix
Correct Model Invocation with Application Inference Profiles in Anthropic SDK
The error occurs because the Anthropic SDK's Invoke Model method does not accept an ARN for the modelId parameter when using application inference profiles. Instead, it expects a base model identifier, which leads to the UnknownOperationException when an ARN is provided.
Awaiting Verification
Be the first to verify this fix
- 1
Identify the Base Model
Determine the base model associated with the application inference profile you created. You need to know the correct identifier for the base model that the application inference profile is aliasing.
- 2
Modify the Model Invocation
Update your model invocation code to use the base model identifier instead of the ARN of the application inference profile. This ensures compatibility with the Anthropic SDK.
pythonmodel_id = 'anthropic-claude-v1' # Replace with your base model ID response = anthropic_sdk.invoke_model(model=model_id, input_data='Your input here') - 3
Test the Model Invocation
Run your updated code to invoke the model using the base model identifier. Ensure that the invocation completes successfully without errors.
pythonprint(response) - 4
Review Logs for Errors
Check the logs for any potential errors or warnings during the model invocation. Confirm that the response is as expected and that no exceptions are thrown.
Validation
Confirm the fix by successfully invoking the model without errors. The response should contain valid output data from the model, indicating that the invocation was successful.
Sign in to verify this fix
Environment
Submitted by
Alex Chen
2450 rep