Add support for Cloudflare Workers / Vercel Edge Functions
Problem
So far it seems like there are 3 culprits which may block us from running `Langchainjs` on Edge. - [x] 1) OpenAI and it's usage of `axios` #118 - Switch to a different library / create a upstream PR for the official library (the pending MR seems to be doing way too much IMO) - [x] 2) `**/index.ts` files might include other modules, which aren't needed - better treeshaking support, which - [x] 3) `langchain/util/hub` and the general usage of NodeJS APIs (`fs`, etc.) #97 - [x] 4) Have at least 1 vector store which 1. doesn't need Node native modules and 2. stores metadata for each document #73 - [ ] Add tag in the docs for node-only things (eg hnswlib) - [x] Create and test cloudlfare workers and vercel edge templates - [x] Remove usage of node-fetch (which uses node-only APIs) #118 - [x] Remove all usage of require #124 - [x] Get TS to output ESM #124 - [x] Remove usage of node `crypto` #213 - [x] Replace node-only tokeniser with one that works in browser-like envs #214 This is the error output when attempting to deploy the `examples/src/chains/llm_chain.ts` via Vercel: <img width="582" alt="CleanShot 2023-02-19 at 19 28 23@2x" src="https://user-images.githubusercontent.com/1443449/219971669-827be1f6-fd2c-4b3c-995c-b5d572a0847d.png">
Error Output
error output when attempting to deploy the `examples/src/chains/llm_chain.ts` via Vercel:
Unverified for your environment
Select your OS to check compatibility.
1 Fix
Implement Edge Compatibility for Langchainjs
The current implementation of Langchainjs relies on Node.js-specific APIs and libraries that are incompatible with Edge environments like Cloudflare Workers and Vercel Edge Functions. This includes the usage of libraries such as axios, node-fetch, and Node.js built-in modules like fs and crypto, which are not available in the Edge runtime. Additionally, the current module structure does not support optimal tree-shaking, leading to unnecessary code being included in the final bundle.
Awaiting Verification
Be the first to verify this fix
- 1
Replace axios with Fetch API
Refactor the code to use the native Fetch API instead of axios for making HTTP requests. This ensures compatibility with Edge environments.
typescriptconst response = await fetch(url, { method: 'GET', headers: { 'Authorization': `Bearer ${token}` } }); - 2
Implement Tree-shaking
Review the `index.ts` files and ensure that only necessary modules are exported. Use ES modules to facilitate better tree-shaking during the build process.
typescript// Example of exporting only necessary functions export { functionA } from './moduleA'; - 3
Remove Node.js APIs
Identify and remove all usages of Node.js-specific APIs such as fs and crypto. Replace them with browser-compatible alternatives or refactor the logic to avoid their use.
typescript// Replace fs usage with an alternative const data = await fetch('/path/to/data.json').then(res => res.json()); - 4
Add Vector Store Compatibility
Implement at least one vector store that does not rely on Node.js native modules and can store metadata for each document. Consider using a browser-compatible library or service.
typescript// Example of a simple in-memory vector store class VectorStore { constructor() { this.store = {}; } } - 5
Document Node-Only Features
Update the documentation to clearly indicate which features and modules are Node.js-specific. Tag these features appropriately to guide users in Edge environments.
markdown// Add tags in documentation // @node-only: hnswlib
Validation
Deploy the updated Langchainjs code to Vercel and Cloudflare Workers. Verify that the deployment completes successfully without errors related to Node.js APIs or incompatible libraries. Test the functionality of the application in both environments to ensure everything operates as expected.
Sign in to verify this fix
Environment
Submitted by
Alex Chen
2450 rep