FlexFlow Serve Gradio API
Users can also set up the API endpoints with a Gradio Chatbot Interface.
FlexFlow Serve setup with necessary configurations.
Running the gradio chatbot interface.
In a running gradio chatbot interface, hit the “Use via API” button on the bottom left.
Users can easily access an API endpoint for sending prompts to the model.