FlexFlow Serve Gradio API

Introduction

Users can also set up the API endpoints with a Gradio Chatbot Interface.

Requirements

  • FlexFlow Serve setup with necessary configurations.

  • Running the gradio chatbot interface.

Example

In a running gradio chatbot interface, hit the “Use via API” button on the bottom left.

Gradio Chatbot Interface

Users can easily access an API endpoint for sending prompts to the model.

Gradio API