Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

first message must use the "user" role error with Bedrock but not with Anthropic #622

Closed
dandv opened this issue Dec 6, 2024 · 4 comments
Assignees

Comments

@dandv
Copy link
Contributor

dandv commented Dec 6, 2024

In addition to the minor difference in behavior between the 1P Anthropic and 3P Bedrock Messages endpoint in #565, there is also a difference in how they handle: Assistant as the first role (succeeds on Anthropic but fails on Bedrock):

import Anthropic from '@anthropic-ai/sdk';
import AnthropicBedrock from '@anthropic-ai/bedrock-sdk';

const clientAnthropic = new Anthropic();
const clientBedrock = new AnthropicBedrock();

const haiku = {
  anthropic: 'claude-3-haiku-20240307',
  bedrock: 'anthropic.claude-3-haiku-20240307-v1:0',
};

async function assistantFirstRole(client: Anthropic | AnthropicBedrock, model = haiku.anthropic) {
  const response = await client.messages.create({
    max_tokens: 1,
    stream: false,
    messages: [
      { role: 'assistant', content: 'How can I help?' },
      { role: 'user', content: 'What is the color of the sky? Answer with only one word.' },
    ],
    model,
  });
  console.log('Response content: ', response.content);
}

await assistantFirstRole(clientAnthropic, haiku.anthropic);  // [ { type: 'text', text: 'Blue' } ]
await assistantFirstRole(clientBedrock, haiku.bedrock);  // Error: 400 messages: first message must use the "user" role
@aaron-lerner
Copy link

I believe this may have been due to timing differences between 1P and 3P releases. The expected behavior here is for it to no longer throw an error. Are you still running into errors on Bedrock?

@aaron-lerner aaron-lerner self-assigned this Jan 30, 2025
@aaron-lerner
Copy link

Just tested and not seeing errors on Bedrock. Please reopen if you're still running into issues.

@dandv
Copy link
Contributor Author

dandv commented Jan 31, 2025

I'm still seeing this error with Bedrock reliably if I run this test script.

@aaron-lerner
Copy link

I'm unfortunately not in that repo and can't see the test script. I'm testing with the below:

import json
import boto3

client = boto3.client("bedrock-runtime", region_name="us-west-2")
model_id = "anthropic.claude-3-5-sonnet-20241022-v2:0"
try:
    response = client.invoke_model(
        modelId=model_id,
        contentType="application/json",
        body=json.dumps({
            "anthropic_version": "bedrock-2023-05-31",
            "max_tokens": 1024,
            "messages": [
                {
                    "role": "assistant",
                    "content": [
                        {"type": "text", "text": "Hello"},
                    ],
                },
            ],
        }),
    )
    [print(r) for r in response.get("body")]
except BaseException as e:
    print(f"ERROR: Can't invoke '{model_id}'. Reason: {e}")
    exit(1)

Which model are you seeing this behavior with?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants