Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature] Realtime API from OpenAI working #545

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open

[Feature] Realtime API from OpenAI working #545

wants to merge 4 commits into from
+58 −0

Conversation

franpb14
Copy link

@franpb14 franpb14 commented Nov 4, 2024 •
edited
Loading

WIP

This is a very basic PR that we can use to iterate. Right now I'm using it with ActionCable in a rails app like this:

class OpenAiChannel < ApplicationCable::Channel
  def subscribed
    stream_from "open_ai_channel"
    @client = OpenAI::Client.new(access_token: ENV['OPENAI_API_KEY'])
    @client.real_time.on_message do |event|
      ActionCable.server.broadcast 'open_ai_channel', { message: event.data.force_encoding('UTF-8') }
    end
    @client.real_time.connect
  end

  def send_message(data)
    @client.real_time.send_event(data['event'])
  end
end

In the example data['event'] can be something like this:

{
  type: "response.create",
  response: {
    modalities: ["text", "audio"],
    instructions: "Please assist the user.",
  }
}

Maybe we could add more functions in order to facilitate event management. Something like I said in this comment.

Dependencies

  • faye-websocket
  • eventmachine

All Submissions:

  • Have you followed the guidelines in our Contributing document?
  • Have you checked to ensure there aren't other open Pull Requests for the same update/change?
  • Have you added an explanation of what your changes do and why you'd like us to include them?

Closes #524

franpb14
franpb14 commented Nov 4, 2024
View reviewed changes
lib/openai/real_time.rb
end

def connect(model: "gpt-4o-realtime-preview-2024-10-01")
uri = "#{File.join(@client.websocket_uri_base, @client.api_version, 'realtime')}?model=#{model}"
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

probably this uri shouldn't be here but I was not sure where to put it

@drnic
Copy link
Contributor

drnic commented Nov 7, 2024 •
edited
Loading

Is it out of scope to add a sample sinatra app into the repo with some stimulusjs that demos the client-side setup of the websocket/connection to feed/receive messages to the backend?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Realtime
2 participants
@franpb14 @drnic