AI & LLMs
228 verified issues
Support for zod 4
Confirm this is a Node library issue and not an underlying OpenAI API issue - [x] This is an issue with the Node library Describe the bug Hey OpenAI team ๐ After updating to Zod v4 Iโm hitting this runtime error: [code block] Looks like the vendored `zod-to-json-schema` in `openai/helpers/zod` still expects the `ZodFirstPartyTypeKind` export, which was removed in Zod v4. Notes / ideas Pinning zod@3 is a temporary workaround, but it blocks upgrading the whole stack. A quick fix might be bumping the vendored zod-to-json-schema to a version that supports Zod v4 or swapping it out for the maintained `@asteasolutions/zod-to-openapi` which already handles the new internals. Thanks for taking a look! Let me know if I can help with more info or a PR. To Reproduce 1. Fresh project with pnpm 1. pnpm add openai@5.8.3 zod@^4 1. Add a simple script: `import { zodResponseFormat } from 'openai/helpers/zod'` 1. pnpm tsx index.ts (or just bundle with esbuild) โ error above. Code snippets [code block] OS macOS Node version Node v22 Library version Latest
[Community contributions] Model cards
Hey friends! ๐ We are currently in the process of improving the Transformers model cards by making them more directly useful for everyone. The main goal is to: 1. Standardize all model cards with a consistent format so users know what to expect when moving between different model cards or trying to learn how to use a new model. 2. Include a brief description of the model (what makes it unique/different) written in a way that's accessible to everyone. 3. Provide ready to use code examples featuring the `Pipeline`, `AutoModel`, and `transformers-cli` with available optimizations included. For large models, provide a quantization example so its easier for everyone to run the model. 4. Include an attention mask visualizer for currently supported models to help users visualize what a model is seeing (refer to #36630) for more details. Compare the before and after model cards below: With so many models in Transformers, we could really use some a hand with standardizing the existing model cards. If you're interested in making a contribution, pick a model from the list below and then you can get started! Steps Each model card should follow the format below. You can copy the text exactly as it is! [code block]py from transformers.utils.attention_visualizer import AttentionMaskVisualizer visualizer = AttentionMaskVisualizer("google/gemma-3-4b-it") visualizer("<img>What is shown in this image?") \[code block]py <insert relevant code snippet here related to the note if its
"lib/IsolatedGPT35TurboMutation/deleteFineTuneModel: AbortController is not defined"
Confirm this is a Node library issue and not an underlying OpenAI API issue - [X] This is an issue with the Node library Describe the bug What is this error? It works fine locally, but when deploying I get this error..: "lib/IsolatedGPT35TurboMutation/deleteFineTuneModel: AbortController is not defined" To Reproduce if (job.model_id) { if (!src.openAiKey) throw new Error('OpenAiKey not found'); const openai = new OpenAI({ apiKey: src.openAiKey }); const model = await openai.models.del(job.model_id); Code snippets _No response_ OS macOS Node version Node 18 Library version openai 4.0.8
Internal Server Error when sending multiple PDFs in conversation
Description When attempting to have a conversation that includes multiple PDF documents with Claude, the API returns a 500 Internal Server Error. The error occurs specifically when trying to send a second PDF after receiving a response for the first one. Steps to Reproduce 1. Initialize a conversation with Claude API 2. Send first message containing: - PDF document (base64 encoded) - Text prompt ("1์ค ์์ฝ") 3. Receive assistant's response 4. Send second message with: - Different PDF document (base64 encoded) - Text prompt ("1์ค์์ฝ") 5. API returns 500 Internal Server Error Code Example [code block] Error Message [code block] Environment - Python SDK Version: (version number) - Claude Model: claude-3-5-sonnet-20241022 - Beta Features: ["pdfs-2024-09-25"] Expected Behavior The API should handle multiple PDF documents in a conversation, allowing for sequential analysis of different documents. Additional Context - The first PDF upload and response works correctly - The error occurs specifically when trying to send a second PDF in the conversation - Using the latest beta PDF feature as indicated in the `betas` parameter Questions 1. Is there a limitation on the number of PDFs that can be processed in a single conversation? 2. Is there a specific way to handle multiple PDF documents in a conversation that I'm missing? 3. Could this be related to the beta status of the PDF feature?
Can't build NextJS project with openai library. Getting: Type error: Private identifiers are only available when targeting ECMAScript 2015 and higher.
Confirm this is a Node library issue and not an underlying OpenAI API issue - [X] This is an issue with the Node library Describe the bug Getting this kind of error on build time on NextJS 14 and I don't know why this is my tsconfig.json [code block] Thanks for help. To Reproduce 1) Install and use library on nextjs 2) import something like `import typoe { Message } from 'openai/resources/beta/threads/messages';` Code snippets _No response_ OS macOS Node version v22.3.0 Library version 4.52.3
ImportError: cannot import name 'ModelField' from 'pydantic.fields'
Hello. I have import error when I am trying to use Anthropic. [code block] [code block] [code block]
Vacuums extremely slow for HNSW indices?
We recently deleted a large part of a ~20 million row table (with an HNSW index size of ~31GB). Attempting to manually vacuum the table took us 10 hours (before we cancelled it) - vacuum got stuck on vacuuming the HNSW index. We tried doing a parallel vacuum and that also didn't seem to help. Our metrics showed that we weren't limited by CPU or memory at any point. Eventually we gave up, dropped the index, vacuumed the table (took <10 mins to complete), and recreated the index. Any guidance as to what we were doing wrong and/or should be doing better in future?
Invalid API Key when using claude 2.0
Windows 11 with Pythno 3.10 and used these codes . It turned out to an "Invalid API Key" error. But I'm sure the api_key is good because I could get good response via unofficial API call (from other github repository). from anthropic import Anthropic, HUMAN_PROMPT, AI_PROMPT anthropic = Anthropic(api_key = "sk-ant-XXXXXX") def getResponse(prompt): msg=f"{HUMAN_PROMPT} {prompt} {AI_PROMPT}" print(msg) completion = anthropic.completions.create( model = "claude-2", max_tokens_to_sample = 30000, prompt = msg, ) res = completion.completion print(res) return res if __name__ == "__main__": getResponse("Hello, Claude") the last 3 lines of error messages: File "D:\Python310\anthropic\lib\site-packages\anthropic\_base_client.py", line 761, in _request raise self._make_status_error_from_response(request, err.response) from None anthropic.AuthenticationError: Error code: 401 - {'error': {'type': 'authentication_error', 'message': 'Invalid API Key'}} Appreciate your help. Thanks.
ChatCompletionStream.fromReadableStream errors due to missing finish_reason for choice
Confirm this is a Node library issue and not an underlying OpenAI API issue - [X] This is an issue with the Node library Describe the bug When trying to use the API described here https://github.com/openai/openai-node/blob/2242688f14d5ab7dbf312d92a99fa4a7394907dc/examples/stream-to-client-browser.ts I'm getting the following an error at the following point: where the actual choices look like this: Looks like the code expects `finish_reason` to be populated but the finish details are now in a property called `finish_details`? To Reproduce Setup a server that responds with chat completion streams Then in the client try to use the `ChatCompletionStream.fromReadableStream` API, e.g.: [code block] Code snippets _No response_ OS Windows Node version 18.12.1 Library version 4.16.1
Support functions
Currently, IvfflatGetType and HnswGetType functions do a syscache lookup to get the datatype, and then there is a bunch of code that do things like this based on the type: [code block] That's a bit of an unusual pattern in indexes, the usual pattern would be to have support function in the opclass to encapsulate any type-specific differences. To refactor this to use support functions, the minimal change to what's in 'master' would be to define one new support function, something like 'get_vector_type()', which returns a HnswType or IvfflatType. The HnswGetType()/IvfflatGetType() function would then just call the support function. Those if-statements would remain unchanged. A more proper way to use a support function would be to have support functions like 'hnsw_get_max_dimensions' and 'hnsw_check_value', to replace the places where we currently check the type (GetMaxDimensions and HnswCheckValue). A third approach would be to have just one support function like 'hnsw_type_support' that returns a struct like: [code block] That might be more handy than having a lot of support functions, and you get better type checking from the compiler as you don't need to convert all arguments to Datums. Ivfflat has "if (type == IVFFLAT_TYPE_VECTOR) ..." kind of checks, so it would need more support function, something like: [code block]
Build failure on PostgreSQL 18 beta 1
Build failure: [code block]
src\bitvec.c(43): warning C4141: 'dllexport': used more than once
I'm installing it on Windows 11 with PostgreSQL 16. I got the git clone to work, but when I ran the nmake command (nmake /F Makefile.win) I'm getting a few errors: src\bitvec.c(43): warning C4141: 'dllexport': used more than once C:\Program Files\PostgreSQL\16\include\server\access/tupmacs.h(65): error C2196: case value '4' already used C:\Program Files\PostgreSQL\16\include\server\access/tupmacs.h(197): error C2196: case value '4' already used src\hnsw.c(190): warning C4141: 'dllexport': used more than once NMAKE : fatal error U1077: 'cl /nologo /I"C:\Program Files\PostgreSQL\16\include\server\port\win32_msvc" /I"C:\Program Files\PostgreSQL\16\include\server\port\win32" /I"C:\Program Files\PostgreSQL\16\include\server" /I"C:\Program Files\PostgreSQL\16\include" /O2 /fp:fast /c src\hnsw.c /Fosrc\hnsw.obj' : return code '0x2' Stop. Can anyone please provide assistance? Thank you. Aaron
How to stop streaming
Confirm this is a Node library issue and not an underlying OpenAI API issue - [X] This is an issue with the Node library Describe the bug I use stream.abort(); to stop receive from api , but i have exception below [code block] I have been follow to guide in the document > If you need to cancel a stream, you can break from a for await loop or call `stream.abort()`. To Reproduce [code block] Nope Code snippets _No response_ OS Ubuntu Node version 16.15.1 Library version v4.28.0
Batch API does not support cache_control
Batch API failed with error message > messages.5.content.0.text.cache_control: Extra inputs are not permitted I have both indicated `betas` in `params` and `client.beta.messages.batches.create` My prompts look like [code block]
No results when using index
I have a table of quotes with vector embeddings. There are two columns in the table, one is indexed one is not, they are otherwise identical. When I query against the unindexed column for the 5 nearest neighbors to a given embedding, I get 5 results. However, when I run the same query using the indexed column I get no results. [code block] Produces: [code block] But changing to use the indexed column, embedding2, in the order by clause, and setting the probes parameter [code block] Returns no results. How can using an index in an order by clause lead to no results being returned? This is entirely reproducible, so if there is any debugging I can do to help diagnose the issue, let me know.
Importing & using AssistantStream breaks Angular SSR
Confirm this is a Node library issue and not an underlying OpenAI API issue - [X] This is an issue with the Node library Describe the bug Just by adding the following lines in my new Angular project, I get an error. Lines: [code block] On compiling the code I get the following error: [code block] Now I'm unsure whether or not this is something you guys can fix, or it should be somehow reported to Angular or the polyfill... but I thought I'd start here, because I really have no clue what's going on. The internals of the AssistantStream are somewhat beyond me and I have no experience with handling streams in Typescript. Feel free to close this issue and report it to the right place - or I can do it with the right info. To Reproduce 1. Install Angular normally with [code block] As part of the installation prompts, choose to use SSR 2. Add this lib [code block] 3. Change the app.component.ts to include the openai AssistantStream [code block] 4. Run angular and watch it crash [code block] Code snippets [code block] OS Linux Mint Node version 20.9.0 and 22.1.0 Library version 4.51.0
createTranscription() doesn't work as expected in NodeJS
Describe the bug Hi! I tried to follow the documentation when I was writing a transcription script using NodeJS. And I wanted to get a response in .srt format. But it returns an error. I tried to use the argument `response_format` as well as `responseFormat()`. But that didn't work. Also, there is only one way to communicate with OpenAI API: [code block] But anyway, it doesn't work if I would like to specify the output file format. To Reproduce 1. Run the function (one of them) 2. Get `Required parameter model was null or undefined` error Code snippets [code block] [code block] ``` OS macOS Node version Node v.16.13.0 Library version openai v.3.2.1
TS7030: Not all code paths return a value
Confirm this is a Node library issue and not an underlying OpenAI API issue - [X] This is an issue with the Node library Describe the bug When building using `tsc` with `openai` as a dependency, I get the following error: node_modules/openai/src/lib/AbstractChatCompletionRunner.ts(224,28): error TS7030: Not all code paths return a value. I also had to disable the `noImplicitAny` rule since `openai` is not built, and then my app build inherits your source code basically. To Reproduce `const OpenAI = require('openai');` and build with `tsc` (Ideally, I would like this to be an automated test for this library, so build errors do not repeat.) Code snippets _No response_ OS macOS, ubuntu Node version Node 18 Library version 4.15.1
Incremental Streaming vs Culimative Streaming
I noticed that Anthropics APIs use cumulative streaming in the completion endpoint, resulting in repetitive data being sent over the wire. Is there a reason for this design choice? I imagine Incremental streaming is typically preferred for efficiency. I see this is on the endpoint and not a python or ts issue. so this could impact existing customers.
fetch is not exported from 'openai/_shims/fetch'
Confirm this is a Node library issue and not an underlying OpenAI API issue - [X] This is an issue with the Node library Describe the bug I am getting the following error when trying to instantiate my OpenAI client. [code block] To Reproduce Instantiate openAI with the following: [code block] Code snippets The OpenAI definition in my package.json is `"openai": "^4.2.0"` My `tsconfig.json` file: [code block] ``` OS macOS Node version v20.2.0 Library version 4.2.0
Add a way to avoid printing the finalContent when using runFunctions
Confirm this is a feature request for the Node library and not the underlying OpenAI API. - [X] This is a feature request for the Node library Describe the feature or improvement you're requesting I have a routine that prints alphabet A-Z. One function prints one letter at a time. The problem is that OpenAI SDK also outputs the entire alphabet _again_, at the end. Is there a way that would force it to return early without bothering with producing the final output? [code block] Here the last `console.log` will print something along the lines of: [code block] I tried tapping into `.on('finalFunctionCall', ...` event, but that happens after `finalContent()` already has a result. I really just need some sort of event that fires before it starts generating the final output, so I could abort early. The example uses alphabet listing, which is quick. But in real-world scenario, that final content function can a long time/many tokens to generate.
Please support vector sizes of 20,000 like PinconeDb
I would much rather use pgVector with PostgreSQL than Pinecone. Any suggestions? Thanks.
Proposal: Explicitly set HNSW build workers
Problem Selecting parallel workers for HNSW follows a similar method to IVFFLAT, i.e. leveraging the PostgreSQL `plan_create_index_workers` function, which uses the number of heap (table) pages return from `estimate_rel_size` to determine the number of parallel workers to use. This makes sense for B-tree / IVFFLAT as the time spent is in the loading the data from the table vs. calculations. But for HNSW, we're likely underestimating the number of workers we could use to speed up the builds as the HNSW process is much more CPU heavy. For example, review the [charts in this blog post][1] that show how we can increase concurrent inserts and can continue scaling the amount of inserts per second. We can see this play out in practice. Using a similar sample to [aforementioned blog post][1] on a 64-core m7gd.16xlarge instance, I created a table with 1,000,000 128-dim vectors. With enabling parallel builds, PostgreSQL elected to spawn 4 parallel workers (+ leader). [code block] For the same data set, I hardcoded to use 8 parallel workers (+leader), and saw a significant speedup: [code block] We're still likely falling to [underestimating parallel workers due to TOAST][2], but this case is not affected by that as a 128-dim vector is not TOAST'd. Additionally, as mentioned, HNSW index builds are CPU bound, so we'd want to maximize the number of cores we can use for the process. (There's still considerations around shared/temporary memory, but I'm seeing the brunt of the issue a
[Question]Need help with the pgvector Installation with pgAdmin & version mismatch error
Hello, I have installed 'pgAdmin 4 v8.2' on macOS. After successfully installing the software, I proceeded to run the command CREATE EXTENSION vector; However, I encountered an error: so I attempted to solve the issue by using the following commands in my terminal: `cd /tmp git clone --branch v0.5.1 https://github.com/pgvector/pgvector.git cd pgvector make make install ` The error is still the same, i've tried `brew install pgvector`too. In the folder "pgvector", I also tried this line of command `export PG_CONFIG=/Library/PostgreSQL/16/bin/` and copy "pgvector/sql/vector--0.5.1.sql" and "vector.control" into the folder "extension" <img width="730" alt="Screenshot 2024-01-15 at 16 26 26" src="https://github.com/pgvector/pgvector/assets/74989793/40a29e2d-0824-4a88-9cf8-d7444b0066af"> Unfortunately, the error evolved to I am reaching out for assistance. Could anyone provide guidance or insights into resolving this matter? Thanks
ImportError: cannot import name 'Anthropic' from 'anthropic'
Trying to run the basic code for Anthropic and getting this error: AttributeError: module 'anthropic' has no attribute 'Anthropic' Using anthropic == 0.3.6 Code in my notebook: from anthropic import Anthropic, HUMAN_PROMPT, AI_PROMPT anthropic = Anthropic( defaults to os.environ.get("ANTHROPIC_API_KEY") api_key='replaced with my actual api key', ) completion = anthropic.completions.create( model="claude-2", max_tokens_to_sample=300, prompt=f"{HUMAN_PROMPT} how does a court case get to the Supreme Court? {AI_PROMPT}", ) print(completion.completion)
Model seems to ignore `.optional()` fields, and instead uses fasly values such as 0, empty strings etc'
Confirm this is a Node library issue and not an underlying OpenAI API issue - [X] This is an issue with the Node library Describe the bug Model seems to ignore `.optional()` fields, and instead uses nullary values, empty strings etc' I'm not sure if it's a library issue or chatgpt model issue. To Reproduce I suspect this is a KI or an upstream problem, but let me know if not I'll send a repro Code snippets _No response_ OS ubuntu Node version 22.7.0 Library version 4.57.0
"warning":"This model version is deprecated. Migrate before January 4, 2024
Describe the bug Im using v3.3.0 attempting to make a call with text-davinci-003 [code block] I get this sort of return: {"warning":"This model version is deprecated. Migrate before January 4, 2024 to avoid disruption of service. Learn more https://platform.openai.com/docs/deprecations"...... I go to my account and try then use - gpt-3.5-turbo-instruct for the model, and promptly get a response of Model does not exist. The information provided in the blog posts, says this is a drop in replacement. I'm kinda stuck, and I don't believe an API call should return a warning as the object and a 200 response. To Reproduce 1. Create a Node application 2. Add the code to call the API 3. Make the call and watch the result Code snippets [code block] ``` OS Windows 11 Node version Node 18.12.1 Library version v3.3.0
consider supporting an `enable_approxscan` option
problem When monitoring the recall of a vector query, you sometimes want to disable all approximate indexes to get the ground_truth and monitor recall. Yet `enable_indexscan` isn't ideal since it turns off all indexes, causing problems in complex queries that make use of multiple tables/indexes. solution Introduce a new guc convention, that if `enable_approxscan` is set and marked as off, turns off approximate indexes (by setting the cost really high as other enable_* gucs do). This guc wouldn't be defined in pgvector, but pgvector would check if it's set (just by name) and act accordingly. I propose `enable_approxscan` as the name but open to other suggestions. I think this convention would ideally, in time apply to all approximate Postgres indexes, certainly we'd also implement it in pgvectorscale's diskann. question Curious about thoughts on whether or not this is a good idea.
Trouble on installing with mac (M1)
I have followed the methods for installation of pgvector. I changed the path for Postgres accordingly but I received the error message: 2024-04-10 10:08:58.179 GMT [53653] FATAL: --pgxs requires a value cp sql/vector.sql sql/vector--0.6.2.sql Is there anything that I need to install to resolve the error ?
Memory leak while using anthropic python sdk 0.3.10
We have a server implemented with FastAPI to cal Anthropic through Python, but when run the following experiments, the memory kept increasing and not released after stop sending request to the server. - a client sending http requests continuously to the server - concurrency 20 - running duration 2mins Sample server code logic [code block] When I replaced the Anthropic python sdk with httpx request directly, the memory kept low and stable during the same experiment. Same issue when using 0.3.11 as well. Would be great if you can help to take a look.