FlexFlow Serve Gradio API


Users can also set up the API endpoints with a Gradio Chatbot Interface.


  • FlexFlow Serve setup with necessary configurations.

  • Running the gradio chatbot interface.


In a running gradio chatbot interface, hit the “Use via API” button on the bottom left.

Gradio Chatbot Interface

Users can easily access an API endpoint for sending prompts to the model.

Gradio API