Images Bot Integration

This document explains how to integrate image handling in bots that use the OpenAI API, following a clear workflow from image upload to API request handling.

Updated Workflow Overview

1. User uploads an image to the chat
2. System detects the content type and routes to a specific trigger
3. Bot processes the image using the OpenAI API
4. AI generates a response based on the image content

Implementation Example

When an image is uploaded, the JSON payload for OpenAI looks like:

In Live Helper Chat pictures are uploaded instantly and there is no possibility to write user question within image before uploading. So we hint AI that we will ask our question afterwards.

{
    "role": "user",
    "content": [
        { "type": "text", "text": "I'll ask my question about uploaded picture in the next message" },
        {
            "type": "image_url",
            "image_url": {
                "url": "{args.msg.file.file_body_embed}"
            }
        }
    ]
}

Sample Configuration Files

You can download the necessary files to implement this functionality:

• REST API Configuration - Remember to set your OPENAI_API_KEY in the configuration
• Bot Configuration - Import this after setting up the REST API

Integration Points

The system handles images through specific triggers that activate when image content is detected.

Technical Notes

• Image content is embedded using the file_body_embed parameter
• The OpenAI API processes multimodal content through the content array
• Both text and image data can be sent in the same request