Jev Can See Now: Working With Images in Jev

Jev is TypeSafe AI’s model for answering small, typed questions about some input. You give it a state and a set of questions, and it returns typed answers with probabilities and a confidence. The question types are Choice (pick one option), Score (rate against criteria) and Noul (a probability that a statement is true).

If Jev is new to you, start with the Jev tutorial, which covers the three question types and how a request works.

Until recently the state could only be text. Jev 1.14, released on October 14th, 2026, accepts images. This post shows how to use them.

The full code is at github.com/nononsense-ai/jev-image.

A warning first

TypeSafe calls image support dreamer-ready, which means it’s fine for prototypes but not for production yet. The request format and the answers can change between releases, so keep each question small and simple.

What we’re building

Imagine a camera at your front door. We’ll ask Jev whether a person is standing there and what they’re carrying. After that we’ll compare two photos, use pictures as the options of a Choice question, and finish with the new Box question type, which finds an object in a picture.

1. Initialize the project

uv init --lib jev
cd jev
uv add "typesafe-sdk[pil]" python-dotenv

We’re not publishing a library. The --lib flag is only there for the src/jev layout, and that layout is why our files can import each other as jev.helpers. So create the project this way and don’t rename it afterwards.

The pil extra lets us send Pillow images (Pillow is Python’s image library) to the SDK, and python-dotenv loads our .env file.

From here on, every file runs with uv run followed by its path:

uv run src/jev/<file name>.py

2. Set your API key

Create a file called .env in the project root with one line:

TYPESAFE_API_KEY=your-key-from-the-typesafe-console

Get the key from the TypeSafe console and treat it like a password.

Add .env to .gitignore

Do this before your first commit, so the key never ends up on GitHub. uv init already created a .gitignore. Add these lines to it:

# Secrets
.env

# macOS
.DS_Store

If you’ve already committed the file by accident, adding it to .gitignore isn’t enough. Remove it from git with git rm --cached .env and rotate the key.

3. Directory structure

Put an assets/ folder in the project root for the pictures. This is what the project looks like when we’re done:

jev/
├── .env                  # TYPESAFE_API_KEY (git-ignored)
├── .gitignore
├── .python-version
├── pyproject.toml
├── README.md
├── assets/
│   ├── badge.png
│   ├── fist.png
│   ├── in-door-person-with-package.png
│   ├── in-door-person-with-bag.png
│   ├── kitchen.png
│   ├── open_hand.png
│   ├── person-in-door.png
│   ├── thumbs_up.png
│   └── thumbs_up_test.png    # a second, different thumbs-up photo
└── src/
    └── jev/
        ├── __init__.py
        ├── helpers.py            # make_client(), fit(), load()
        ├── image.py              # Example 1: one image
        ├── compare_two_images.py # Example 2: two images
        ├── images_as_choice.py   # Example 3: images as options
        └── box_mug.py            # Box question

Use any photos you like, as long as the file names match what the code expects.

4. The rules for images

These four rules cover most of the failures.

Rule 1: turn on the header. Image requests need the header TypeSafe-Danger-Zone-Images: true. Without it, every image request fails with a 400 error.

Rule 2: wrap each picture in Image. An Image can go anywhere text can: in the state, in the instructions or in the criteria. The SDK uploads the file for you.

Rule 3: share images once. Every place an image appears adds another copy for the model to read, and each copy counts against your size limits. Images are billed as input tokens, at the same price as text. Output tokens are free, so input is what you pay for. If several questions use the same picture, put it in the state once and refer to it by name. You can see what a request used in result.usage.input_tokens.

Rule 4: names stay plain text. Question IDs and similar names are always text.

Size limits

• PNG or JPEG only, and not animated.
• Width and height must each be between 280 and 1568 pixels.
• Jev won’t resize anything for you. Too small is rejected, and so is too big.
• One bad image fails the whole request, with no partial answers.

5. The helper file

The images header, the Image wrapper and the size limits live in one small file that every example shares.

src/jev/helpers.py:

from dotenv import load_dotenv
from PIL import Image as PILImage
from PIL import ImageOps
from typesafe_sdk import Image, TypeSafeClient


def make_client() -> TypeSafeClient:
    load_dotenv()
    return TypeSafeClient(headers={"TypeSafe-Danger-Zone-Images": "true"})


def fit(picture: PILImage.Image) -> Image:
    picture = ImageOps.exif_transpose(picture).convert("RGB")
    picture.thumbnail((1568, 1568))
    if min(picture.size) < 280:
        picture = ImageOps.scale(picture, 280 / min(picture.size))
    if max(picture.size) > 1568:
        raise ValueError(f"{picture.size} is too elongated. Crop or pad it first")
    return Image.from_pil(picture)


def load(path) -> Image:
    return fit(PILImage.open(path))

make_client() loads the .env file and creates the client with the images header set (Rule 1).

fit() straightens the picture’s rotation, converts it to RGB and shrinks anything over 1568 pixels. If a side is under 280 it scales the picture up. If the shape is too stretched to fix, it raises an error so you can crop it first. It returns an Image (Rule 2).

load() opens a file and runs fit() on it.

6. Example 1: ask about one image

The state holds one picture, the door camera photo, under the name door. We ask two questions that point at it with backticks: a Noul (“Is there a person in door?”) and a Choice (“What is the person carrying?”) with a package, a bag or nothing as the options.

src/jev/image.py:

from typing import cast

from typesafe_sdk import Choice, ChoiceAnswer, Noul, NoulAnswer

from jev.helpers import load, make_client


def main() -> None:
    with make_client() as client:
        result = client.system_one(
            state={"door": load("assets/in-door-person-with-package.png")},
            questions={
                "person_present": Noul(instructions="Is there a person in `door`?"),
                "carrying": Choice(
                    instructions="What is the person in the `door` carrying?",
                    criteria={
                        "package": "A box or parcel",
                        "bag": "A bag of groceries",
                        "nothing": "Empty hands",
                    },
                ),
            },
        )
        answers = result.answers
        is_person_present = cast(NoulAnswer, answers["person_present"])
        carrying = cast(ChoiceAnswer, answers["carrying"])

        if is_person_present.noul >= 0.8:
            print("Yes, there is a person in the door")
            match carrying.choice:
                case "package":
                    print("The person is carrying a package")
                case "bag":
                    print("The person is carrying a bag")
                case _:
                    print("The person is carrying nothing!")

        print(f"\n\n{is_person_present}")
        print(carrying)


if __name__ == "__main__":
    main()

After the call we read the answers. If the person is there with a probability of 0.8 or more, we print what they’re carrying. At the end we also print the full answers, so you can see the probabilities and the confidence.

Run it with the first picture, a person holding a package:

uv run src/jev/image.py

Jev should say there’s a person at the door, and the carrying question should pick the package.

Now change the picture to assets/in-door-person-with-bag.png, where the person is holding a bag of groceries, and run it again. This time carrying should pick the bag.

7. Example 2: compare two images

The state now holds two pictures, door and badge, and the question asks whether the person in door is the same as badge. As before, 0.8 or higher counts as a yes.

src/jev/compare_two_images.py:

from typing import cast

from typesafe_sdk import Noul, NoulAnswer

from jev.helpers import load, make_client


def main() -> None:
    with make_client() as client:
        result = client.system_one(
            state={
                "door": load("assets/person-in-door.png"),
                "badge": load("assets/badge.png"),
            },
            questions={
                "same_person": Noul(
                    instructions="Is the person in the `door` the same as `badge` ?"
                )
            },
        )
        is_same_person = cast(NoulAnswer, result.answers["same_person"])
        if is_same_person.noul >= 0.8:
            print("Yes, it is the same person")
        else:
            print("No, it is a different person")
        print(is_same_person)


if __name__ == "__main__":
    main()

uv run src/jev/compare_two_images.py

It prints whether it’s the same person, followed by the Noul answer itself, so you can see how sure Jev is.

8. Example 3: images as options

The options in a Choice question can be pictures too. Here the state is a picture of a hand. The options are a thumbs-up picture, an open-hand picture, and a text option for anything else. A match statement prints the gesture.

src/jev/images_as_choice.py:

from typing import cast

from typesafe_sdk import Choice, ChoiceAnswer

from jev.helpers import load, make_client


def main() -> None:
    with make_client() as client:
        result = client.system_one(
            state={
                "hand": load("assets/fist.png"),
            },
            questions={
                "gesture": Choice(
                    instructions="Which gesture is the `hand` doing?",
                    criteria={
                        "thumbs_up": load("assets/thumbs_up.png"),
                        "open_hand": load("assets/open_hand.png"),
                        "none": "No hand or a hand matching none of the above",
                    },
                )
            },
        )
        gesture = cast(ChoiceAnswer, result.answers["gesture"])
        match gesture.choice:
            case "thumbs_up":
                print("Hand is doing thumbs up!")
            case "open_hand":
                print("It's open hand")
            case _:
                print("This gesture is not in the list")
        print(f"\n{gesture}")


if __name__ == "__main__":
    main()

We run it twice. First with a fist, which matches neither option:

uv run src/jev/images_as_choice.py

The code should print that the gesture isn’t in the list, and the Choice answer below it shows the probabilities for each option.

Then swap assets/fist.png for a different thumbs-up photo, assets/thumbs_up_test.png, and run it again. This time Jev should pick thumbs_up and the code prints “Hand is doing thumbs up!”.

The state photo has to differ from the one in criteria. If both are the same file, the match is trivial.

9. The new Box question

Jev 1.14 adds a fourth question type, Box. It finds something in a picture and gives you a rectangle around it.

• It needs the images header (already set in make_client()).
• It needs the newer model, jev-preview or jev-1.14.0. The jev-latest model doesn’t support Box yet.

The answer is four edges: left, top, right and bottom. Each one is a fraction of the picture’s width or height, measured from the top-left corner. Because they’re fractions, the numbers hold up at any resolution.

Pair every Box with a Noul

A Box question always returns a box, even when the thing isn’t in the picture. So only trust it when a Noul says the object is there.

src/jev/box_mug.py:

from typing import cast

from PIL import Image as PILImage
from typesafe_sdk import Box, Noul, NoulAnswer

from jev.helpers import load, make_client


def main() -> None:
    with make_client() as client:
        result = client.system_one(
            state={"photo": load("assets/kitchen.png")},
            questions={
                "mug_visible": Noul(instructions="Is there a coffee mug in the photo?"),
                "mug": Box(instructions="the coffee mug in `photo`"),
            },
            model="jev-preview",
        )
        answers = result.answers
        is_mug_visible = cast(NoulAnswer, answers["mug_visible"])
        if is_mug_visible.noul > 0.5:
            answer = result.boxes["mug"]
            print(f"Box: {answer.box},\nConfidence: {answer.confidence}")

            print("Cropping image...")
            photo = PILImage.open("assets/kitchen.png")
            w, h = photo.size
            e = result.boxes["mug"].box
            photo.crop(
                (
                    round(e.left * w),
                    round(e.top * h),
                    round(e.right * w),
                    round(e.bottom * h),
                )
            ).save("assets/mug.png")
            print("Image cropped successfully!")


if __name__ == "__main__":
    main()

To crop, multiply each edge by the picture’s width or height and hand the result to Pillow.

uv run src/jev/box_mug.py

The terminal prints the box edges and the confidence, then crops the picture and saves just the mug as assets/mug.png, next to the original.

This example doesn’t handle a mug that’s cut off at the edge of the photo.

10. When things go wrong

When a request fails, look at the status code:

Video and audio still aren’t supported. Send video as a few frames, and audio as a transcript. If you’d rather send them directly, Cloudflare’s Clef-omni model accepts audio and video; I cover it in the Clef post.

Recap

• Turn on the images header.
• Run every picture through fit().
• Put shared images in the state once.
• Read a Noul as a probability, and pair every Box with one.
• Once you’ve tuned your thresholds, pin a versioned model like jev-1.14.0 so your answers don’t change when the alias moves.