Developers

Images

A memory can carry an image. The engine indexes the image, so a text query in any language matches it even when the record has no caption, title or alt text.

Supported on Tablet 2 and newer. An engine that does not implement images says so by name rather than answering a bare 404, so you can tell a missing feature from a missing memory. JPEG, PNG, GIF and WebP.

Storing one

Pass an image object to the ordinary add call. content may be empty; the image is then searchable on its own.

mem.add("at the beach", image={"data": b64})   # caption + image
mem.add("", image={"data": b64})               # the image IS the memory
// the image rides in the 4th argument; the 3rd is metadata
await mem.add("at the beach", undefined, {}, { image: { data: b64 } });
await mem.add("", undefined, {}, { image: { data: b64 } });   // the image IS the memory
let img = json!({"image": {"data": b64}});
mem.add_with("at the beach", None, json!({}), img.clone()).await?;
mem.add_with("", None, json!({}), img).await?;
curl -X POST https://api.wontopos.com/api/v1/memory/store \
  -H "X-API-Key: $WOS_KEY" -H "Content-Type: application/json" \
  -d '{"user_id":"alice","content":"","image":{"data":"<base64>"}}'

data is required. A data:image/jpeg;base64, prefix and the line wrapping added by base64 and openssl are both stripped for you.

FieldWhat it does
dataBase64 of the image. Required. The size ceiling is a server setting, not an SDK constant - /health reports it as memory.images.max_bytes.
referenceWhere your own copy of the original lives. Stored as a string and never fetched by us - and the place for it, since what we keep is downscaled.
taken_atRFC3339, usually from EXIF. Fills event_date when that is empty, so the memory sorts by when the image was taken rather than when it was uploaded.

Finding one

There is no separate image search. search and recall return images alongside text, ranked together.

Working with the ones you have

data, mime = mem.get_image(memory_id=mid)
page       = mem.list_images(limit=50)      # page["count"] = store total
mem.forget_image(memory_id=mid, preview=True)
const { bytes, contentType } = await mem.getImage(undefined, mid);
const page = await mem.listImages(undefined, { limit: 50 });
await mem.forgetImage(undefined, mid, { preview: true });
let (bytes, mime) = mem.get_image(None, mid).await?;
let page = mem.list_images(None, 50, None, None).await?;
mem.forget_image(None, mid, true).await?;
# the picture we hold — the one call on this plane that is not JSON
curl -X POST   .../api/v1/memory/image  -d '{"user_id":"alice","memory_id":"m_1"}'
curl -X POST   .../api/v1/memory/images -d '{"user_id":"alice","limit":50}'
curl -X DELETE .../api/v1/memory/image  -d '{"user_id":"alice","memory_id":"m_1","preview":true}'
CallWhat it does
get_imageThe picture we hold, as (bytes, content_type) - not necessarily your upload. An image whose long edge was over 1,568 px was stored downscaled, and re-encoded to WebP unless it was a JPEG, so a PNG comes back as image/webp. The type is sniffed from the bytes, not from whatever the upload was named, so name the file from content_type. A memory with no image raises rather than returning something empty.
list_imagesOne page, newest first, plus count - the store's total, not the page size. Paging is by cursor: hand next_before and next_skip_ids back. Both are needed because images can share a timestamp.
forget_imageRemoves the image and keeps the text. An image stored with no caption is the memory, so there it deletes the memory too.

Pass preview=True to forget_image to get memory_kept without changing anything. iter_images pages for you.

What an image costs

Images are billed in tokens, like text. Tokens = pixel area / 556.7. Above 1,568 px on the long edge the count is taken at 1,568 px, so a 2,500 px image and a 1,568 px image cost the same.

ImageCounted atTokens
700 × 700as sent881
1000 × 1000as sent1,797
1568 × 1568as sent4,417
1920 × 10801568 × 8822,485
2500 × 18751568 × 11763,313
2500 × 25001568 × 15684,417

Ceiling: 4,417 tokens per image. The ceiling is reserved against your balance before the call; the measured value is charged after and is never higher.

What we keep is the picture at the resolution of a memory, not your original. Above 1,568 px on the long edge it is downscaled to 1,568 on the way in, and that smaller picture is what gets indexed, stored and handed back. Downscaling means re-encoding, so lossless formats are written as WebP: a 2,500 px PNG comes back at 1,568 px as image/webp. JPEG stays JPEG, and below 1,568 px the bytes are untouched. (If re-encoding would make the file bigger, we keep your bytes as they were.) Both edges must still be ≥ 700 px and the long edge ≤ 2,500 px or the call is refused with a 400 - under 700 px a flat minimum applies, so a smaller image costs the same to store, and over 2,500 px we will not decode the file at all. Keep your own copy, or put its URL in reference, if you need the full-resolution file.

How many come back

Default 1, maximum 5 images per response. Five images is close to 20,000 tokens.

FieldWhat it does
max_images0 to 5. Images a single response may carry. Default 1. 0 returns text only. Out of range is rejected rather than clamped.
An English caption on an image improves queries in English and degrades queries in other languages - 11.4 points of recall@5 on average across fourteen languages. Store images without captions if your users search in more than one language.