# Multimodal Support

Choose the right Mighty settings for text, images, PDFs, documents, OCR output, model output, and PII-heavy workflows.

Source URL: https://trymighty.ai/docs/integrate/multimodal-support

Mighty uses one scan API across many kinds of material. The important decision is not the endpoint. The important decision is what your app is about to trust.

Mighty supports text, images, PDFs, documents, OCR output, model output, and agent output through one scan contract.

## Mental Model

Send untrusted material to Mighty before it reaches storage, OCR, AI extraction, model context, workflow automation, or a final user-visible answer.

Use these three fields first:

| Field | What to decide |
| --- | --- |
| `content_type` | What kind of material is this? |
| `scan_phase` | Is this submitted input or generated output? |
| `data_sensitivity` | Should normal PII be tolerated or treated as strict risk? |

## Modality Guide

| What you have | Send as | Common settings |
| --- | --- | --- |
| Chat prompt, support message, note, or form text | JSON `content` | `content_type=text`, `scan_phase=input` |
| OCR text, extracted fields, IDP output | JSON `content` | `content_type=text`, `data_sensitivity=tolerant` |
| Model response, summary, agent answer, generated decision | JSON `content` | `content_type=text`, `scan_phase=output`, `profile=ai_safety` |
| Damage photo, ID photo, receipt photo, screenshot | multipart file or raw binary | `content_type=image`, `focus=both` |
| Claim packet, invoice, estimate, signed form | multipart file | `content_type=pdf`, `document`, or `auto` |
| Audio transcript | JSON `content` | `content_type=text`, `metadata[source]=audio_transcript` |
| Audio file | Closed beta | Ask Mighty for beta access before sending audio files. |
| Unknown upload type | multipart file | `content_type=auto` |

## What Each Modality Checks

| Modality | Detection surface | Typical risk |
| --- | --- | --- |
| Text | Raw text, prompt text, field values, agent tool output. | Prompt injection, unsafe instruction, secret exposure, PII leakage. |
| Images | Visual evidence and image metadata when available. | AI-generated evidence signals, edits, reposts, metadata mismatch. |
| PDFs | Page text, embedded images, extracted text, per-page signals. | Hidden instructions, altered invoices, poisoned extraction output. |
| Documents | Business documents, estimates, forms, uploaded packets. | Hidden text, suspicious document instructions, unsafe workflow data. |
| OCR and IDP output | Extracted text and structured fields. | Poisoned OCR, field manipulation, extraction output becoming trusted too early. |
| Model and agent output | Generated text, summaries, recommendations, tool results. | Unsafe output, secret leakage, tool-result injection, bad autonomous action. |
| Audio | Closed beta audio or transcript evidence. | Synthetic voice evidence signals, transcript injection, sensitive disclosure. |

For the attack taxonomy, see [Modalities And Attacks](/docs/concepts/modalities-attacks).

## PII And Sensitive Data

Business workflows often contain normal PII. A claim, invoice, health intake, or identity workflow may include names, addresses, phone numbers, account numbers, policy IDs, and claim IDs.

Use:

| Setting | Use when |
| --- | --- |
| `data_sensitivity=standard` | Default for normal apps. |
| `data_sensitivity=tolerant` | PII is expected and should not block by itself. |
| `data_sensitivity=strict` | Secrets, credentials, regulated output, or public AI responses need aggressive handling. |

If Mighty returns `redacted_output`, use it only when your product policy allows a safer replacement. If the action is `BLOCK` and no redaction exists, do not show the original output.

## AI Involvement

Use `focus=both` when the material will be used by an AI system or when AI authenticity matters.

Use metadata for context your app knows:

```json
{
  "metadata": {
    "workflow": "claims_intake",
    "ai_involved": "true",
    "submitted_as_ai_generated": "unknown"
  }
}
```

`submitted_as_ai_generated` is your app's claim about what the submitter said. It is not a Mighty verdict. Read Mighty response signals like `authenticity`, `forensics`, `threats`, and `risk_score` separately.

## Example Request

```bash
curl -X POST https://gateway.trymighty.ai/v1/scan \
  -H "Authorization: Bearer $MIGHTY_API_KEY" \
  -F "file=@./claim-packet.pdf" \
  -F "content_type=pdf" \
  -F "scan_phase=input" \
  -F "mode=secure" \
  -F "focus=both" \
  -F "data_sensitivity=tolerant" \
  -F "metadata[workflow]=claims_intake" \
  -F "metadata[ai_involved]=true" \
  -F "metadata[submitted_as_ai_generated]=unknown"
```

## Routing

| Action | Product behavior |
| --- | --- |
| `ALLOW` | Continue the workflow. |
| `WARN` | Continue with friction, request more evidence, or queue review. |
| `BLOCK` | Stop automation. Use `redacted_output` only when returned and allowed. |

## Production Checklist

- Keep the API key on the server.
- Scan original files before OCR when possible.
- Scan extracted text after OCR when the extracted text will be trusted.
- Scan model or agent output before users see strict workflow answers.
- Use `scan_group_id` to connect file, OCR, and output scans.
- Store `content_type_detected`, `action`, `risk_score`, `threats`, `authenticity`, `forensics`, and `redacted_output` when returned.
- Treat AI authenticity signals as review evidence, not proof.

## AI-Agent Prompt

### Add multimodal Mighty support

```text
Add Mighty to the product surfaces that handle text, files, images, OCR output, and model output.

Requirements:
- Keep MIGHTY_API_KEY on the server.
- Use POST https://gateway.trymighty.ai/v1/scan.
- Use content_type=text for chat, OCR text, extracted fields, model output, and agent output.
- Use content_type=image for image evidence.
- Use content_type=pdf, document, or auto for uploads.
- Treat audio as closed beta. If the app only has transcripts, scan transcripts as content_type=text.
- Use scan_phase=input for submitted material.
- Use scan_phase=output for model, OCR, IDP, agent, or automation output.
- Use data_sensitivity=tolerant when normal business PII is expected.
- Use data_sensitivity=strict for public AI output or secret exposure risk.
- Add metadata for workflow, ai_involved, and submitted_as_ai_generated when known.
- Route ALLOW, WARN, BLOCK.
- Use redacted_output only when returned.

Acceptance criteria:
- Every modality has a server-side scan before trust.
- PII handling is explicit.
- Output scans reuse scan_group_id from the related input.
- Tests cover text, image, PDF, OCR output, model output, WARN, BLOCK, redacted_output, 402, 413, and 429.
```