Streaming Structured JSON strings with OpenAIStream / ReadableStream

[moisestech]

How would one go about streaming JSON stringified data by using OpenAIStream and ReadableStream?

Why Stream Structure Data?

Streaming helps me resolve Vercel Timeout Code Limits that happen with regular Fetch Req/Resp.

Goal

Create structured data streams of valid JSON. Ex.

{ id: "id1", name: "joe", location: "new york", age: 30 }

---

Progress so far

It was been tricky since the data may not be valid JSON. Sometimes the model forgets to returns a { bracket for example.
After testing many times it is inconsistenly not sending the first bracket in the first stream payload. Sending an empty text string, yet it is not missing any other parts of the JSON.
{chunkValue: 'data: {"text":""}\n\ndata: {"text":" "}\n\n'}

Here is the full buffer when parsing the text key

Option 1) I've started looking into FrontEnd streaming libraries such as:

1. Clarinet
2. Oboe

Option 2) Starting to look into Function Calls OpenAI API

• This could be the solution since I'm trying to stream structured data from the language mode.
  

---

Reference Code

• You can reproduce by cloning the Twitter Bio Generator Vercel Template and changing the prompt to request JSON. If needed I can create a public repo and link to this discussion. 🙏
• OpenAIStream.ts

Thanks for any advice and guidance. Would love to resolve and make a solution that helps others in the community. Cheers.

[moisestech]

Here is a ⚛️ public repo to replicate the scenario.

I think there may have been changes in the Twitter Bio Generator Vercel Template, the updated repo is using:

1. the app directory

2. /app/page.tsx
import { useChat } from 'ai/react' link

3. /app/api/chat/route.ts
   import { Configuration, OpenAIApi } from 'openai-edge';
   import { OpenAIStream, StreamingTextResponse } from 'ai';