Image Moderation
Use image moderation in OCI Generative AI Guardrails to evaluate images for unsafe content.
Image moderation extends the existing content moderation feature to image inputs. Starting with Guardrails system version 1.1.0, you can use the ApplyGuardrails API to evaluate standalone images or multimodal requests that include both text and images.
Image moderation helps identify unsafe content in user-uploaded images, generated images, screenshots, and images that contain embedded text.
For version selection, see Version Selection Workflow. For the shared request configuration, see Using the ApplyGuardrails API.
Using Image Inputs
To evaluate image content, use multimodalInput instead of input, and specify a Guardrails system version that supports image moderation, such as 1.1.0 or a later version.
Requests can include image-only content or a combination of text and images. When both text and image content are included in the same request, each modality is evaluated independently.
The multimodalInput field can include items with the following type values:
TEXTIMAGE
Supported image formats include:
- JPEG
- PNG
- WebP
A single request can include a maximum of five images. When using text with images, include only one TEXT item in multimodalInput. If you have multiple text values, combine them into a single TEXT item before submitting the request.
Moderation Results
Image moderation doesn’t introduce a separate image moderation response object. Instead, image moderation results are returned as part of the existing contentModeration result, including the existing OVERALL score.
The flaggedModalities field identifies which input modality was detected or contributed to the moderation result.
Supported modality values are:
TEXTIMAGE
For example, if unsafe content is detected only in an image, the OVERALL category includes:
"flaggedModalities": ["IMAGE"]If both text and image content contribute to the moderation result, the response includes both modalities:
"flaggedModalities": ["TEXT", "IMAGE"]Use the returned content moderation results to take action in the application, such as logging detections, warning users, or blocking requests.
Image Moderation Limits and Validation
Image inputs are subject to image-token throttling. The default image-token limit is about 200,000 image tokens per minute. If you need more, request a service limit increase.
Each image input can contain up to 170 million pixels. Image moderation requests are validated before processing. A request can fail when multimodalInput doesn’t meet the supported input requirements.
| Condition | Error Detail | Action |
|---|---|---|
| More than five images are provided | Guardrails API doesn’t support more than five images in multimodalInput. |
Submit no more than five images in a single request. |
More than one TEXT item is provided in multimodalInput with images |
More than one text input along with images isn’t supported. | Combine all text into a single TEXT item before submitting the request. |
| The selected Guardrails system version doesn’t support image moderation | Guardrails version is missing or the specified version doesn’t include image moderation support. | Include guardrailVersionConfig and set guardrailVersion to 1.1.0 or a later supported version. |
| Image input exceeds 170 million pixels | Image size exceeds the maximum pixel limit. | Resize the image and resubmit the request. |