Jev is a System One model developed by TypeSafe.ai. Unlike LLMs that generate text, a System One model makes fast, structured judgments that can be used directly in your code to make decisions. By evaluating typed questions against a provided state, it returns a structured result you can use immediately without having to parse conversational text first. Because it is incredibly fast and highly cost-effective, Jev is the perfect model to automate our workflows.
This is my second article about Jev. In my previous article I already wrote about practical examples for Jev on the ServiceNow AI Platform.
Recently, a new version of Jev dropped. Jev 1.14.0 is a release packed with innovative new capabilities. This article focuses on two of those new features and translates them into practical use cases you can implement on your ServiceNow instance. These two capabilities are:
Multimodality
Images anywhere in the request, in the state and in the questions alike.
“Box” question type
A question answered with the coordinates of a rectangle around an object.
01
Working with images
Turns out that Jev 1.13.0 already had multimodal capabilities behind the scenes, but TypeSafe wanted to improve it even further before making it available.
With Jev 1.14.0, image capabilities still don’t meet TypeSafe’s standards for production-ready features, but they made them available for builders and dreamers already. You can use them by activating the danger zone.
Danger zone
Image support and the box question type are both pre-production. Activate them by setting the header attribute to true:
TypeSafe-Danger-Zone-Images: true
What makes TypeSafe’s approach so interesting is the flexibility: you can use images anywhere in your request. You are not restricted to passing images only in the state. Your instructions and criteria can contain images too. This opens the door to incredible use cases. For example, if you want the AI to locate a highly specific, proprietary object that the model has never seen before, you can simply pass an image of that object directly in your question.
Images are passed inside an assets map within your payload. The key for each asset is the string you will use to reference that image elsewhere in the request. You can send these via a multipart request (with each image as a file part), or by passing a base64-encoded string directly in the JSON.
Request – an image in the state, an image in a question
Let’s look at a simple, hands-on example. I took some building bricks from my kids, sorted them by color, and placed them in different boxes.
The bricks, sorted by color into three boxes. This photo is the only thing the model gets to see.
I want Jev to estimate the fill level of each box. I don’t need a perfectly precise measurement. I just need to know if the boxes are completely full, empty, or somewhere in between.
For this task, the state contains only the reference to the image, while the image itself lives in the assets map as a base64-encoded string.
We use the Score question type to evaluate the fill level for each box. A Score question rates the image in the state against a set of ordered, descriptive levels provided in the criteria array. Here, those levels range from an empty box to a completely filled box.
Request – score question
"fill_blue": {
"type": "score",
"instructions": "How full is the box containing blue bricks in the picture photo?",
"criteria": [
"The box is empty.",
"Only a few bricks are left in the box.",
"The box is half filled.",
"There is only space left for a few bricks in the box.",
"The box is completely filled."
]
}
The criteria array is ordered, so the level the model picks is the fill level0
The box is empty.
1
Only a few bricks are left in the box.
2
The box is half filled.
3
There is only space left for a few bricks in the box.
4
The box is completely filled.
Because Score questions always return a result, we face a potential issue: what if the blue box isn’t even in the photo? To handle this, we pair every Score question with a Noul (yes/no probability) question. The Noul question acts as a gatekeeper, confirming whether the box is visible. If the Noul returns a high probability, we process the Score. If not, we ignore it.
Request – presence check
"present_blue": {
"type": "noul",
"instructions": "Is there a box containing blue bricks in the picture photo?"
}
The same questions back on the instance: one bar per color, carrying the level the model picked and how certain it was.
Complete request — 60 lines, scroll
{
"model": "jev-preview",
"state": {
"photo": "c57a76ba-photo"
},
"questions": {
"present_blue": {
"type": "noul",
"instructions": "Is there a box containing blue bricks in the picture `photo`?"
},
"fill_blue": {
"type": "score",
"instructions": "How full is the box containing blue bricks in the picture `photo`?",
"criteria": [
"The box is empty.",
"Only a few bricks are left in the box.",
"The box is half filled.",
"There is only space left for a few bricks in the box.",
"The box is completely filled."
]
},
"present_yellow": {
"type": "noul",
"instructions": "Is there a box containing yellow bricks in the picture `photo`?"
},
"fill_yellow": {
"type": "score",
"instructions": "How full is the box containing yellow bricks in the picture `photo`?",
"criteria": [
"The box is empty.",
"Only a few bricks are left in the box.",
"The box is half filled.",
"There is only space left for a few bricks in the box.",
"The box is completely filled."
]
},
"present_red": {
"type": "noul",
"instructions": "Is there a box containing red bricks in the picture `photo`?"
},
"fill_red": {
"type": "score",
"instructions": "How full is the box containing red bricks in the picture `photo`?",
"criteria": [
"The box is empty.",
"Only a few bricks are left in the box.",
"The box is half filled.",
"There is only space left for a few bricks in the box.",
"The box is completely filled."
]
}
},
"assets": {
"c57a76ba-photo": {
"type": "image",
"format": "jpeg",
"data_base64": "<34948 base64 chars>"
}
}
}
Complete response — 78 lines, scroll
{
"model": "jev-1.14.0",
"answers": {
"present_blue": {
"type": "noul",
"noul": 0.98
},
"fill_blue": {
"type": "score",
"score": 2.93,
"confidence": 0.63,
"legend": {
"0": "The box is empty.",
"1": "Only a few bricks are left in the box.",
"2": "The box is half filled.",
"3": "There is only space left for a few bricks in the box.",
"4": "The box is completely filled."
},
"probabilities": {
"0": 0,
"1": 0,
"2": 0.25,
"3": 0.56,
"4": 0.19
}
},
"present_yellow": {
"type": "noul",
"noul": 0.98
},
"fill_yellow": {
"type": "score",
"score": 2.13,
"confidence": 0.61,
"legend": {
"0": "The box is empty.",
"1": "Only a few bricks are left in the box.",
"2": "The box is half filled.",
"3": "There is only space left for a few bricks in the box.",
"4": "The box is completely filled."
},
"probabilities": {
"0": 0,
"1": 0.17,
"2": 0.54,
"3": 0.28,
"4": 0.01
}
},
"present_red": {
"type": "noul",
"noul": 0.97
},
"fill_red": {
"type": "score",
"score": 1.89,
"confidence": 0.76,
"legend": {
"0": "The box is empty.",
"1": "Only a few bricks are left in the box.",
"2": "The box is half filled.",
"3": "There is only space left for a few bricks in the box.",
"4": "The box is completely filled."
},
"probabilities": {
"0": 0,
"1": 0.2,
"2": 0.71,
"3": 0.09,
"4": 0
}
}
},
"usage": {
"input_tokens": 1451,
"output_tokens": 101
}
}
Use cases
Knowing how many bricks my kids left in a box doesn’t solve any enterprise problems, but the underlying architecture translates perfectly to high-value retail use cases.
Imagine a supermarket using cameras to detect when shelves need restocking. If the model detects that the apple bins are empty (or on the verge of becoming empty), it can automatically trigger a Store Case to dispatch a worker to restock that specific aisle.
The same question one aisle further: a crate per color becomes a crate per product, and the empty ones are what the store case is about.
03
Box question type
The Box question type is another powerful feature introduced in Jev 1.14.0 (also requiring the danger zone header).
With a Box question, you ask the model to locate an object, and it responds with the exact coordinates of a bounding box surrounding that object within the image.
Let’s extend the brick example. Suppose we want to find bricks that were sorted into the wrong box. If a red brick ends up in the yellow box, the model should return the coordinates of that rogue red brick. We also want it to locate any cracked or dirty bricks.
A Box question only requires an instruction telling the model what to look for.
Request – box questions
"box_wrong_blue": {
"type": "box",
"instructions": "the wrongly placed brick in the box containing blue bricks in photo - the brick in that box whose colour is not blue"
},
"box_defect_blue": {
"type": "box",
"instructions": "the cracked or dirty brick in the box containing blue bricks in photo"
}
Just like the Score primitive, a Box question will always return coordinates, even if the object isn’t actually in the image. Therefore, we again pair it with a Noul question to verify the object’s presence first.
Request – presence checks
"present_wrong_blue": {
"type": "noul",
"instructions": "Is there a wrongly placed brick in the box containing blue bricks in the picture photo? A wrongly placed brick is a brick in that box whose colour is not blue."
},
"present_defect_blue": {
"type": "noul",
"instructions": "Is there a cracked or dirty brick in the box containing blue bricks in the picture photo?"
}
Jev returns the bounding box coordinates as fractions of the image’s total width and height, measured from the top-left corner.
The returned coordinates drawn back onto the photo: a yellow brick in the blue box, a green one in the red box.
Complete request — 63 lines, scroll
{
"model": "jev-preview",
"state": {
"photo": "d8ab3aba-photo"
},
"questions": {
"present_wrong_blue": {
"type": "noul",
"instructions": "Is there a wrongly placed brick in the box containing blue bricks in the picture `photo`? A wrongly placed brick is a brick in that box whose colour is not blue."
},
"box_wrong_blue": {
"type": "box",
"instructions": "the wrongly placed brick in the box containing blue bricks in `photo` - the brick in that box whose colour is not blue"
},
"present_defect_blue": {
"type": "noul",
"instructions": "Is there a cracked or dirty brick in the box containing blue bricks in the picture `photo`?"
},
"box_defect_blue": {
"type": "box",
"instructions": "the cracked or dirty brick in the box containing blue bricks in `photo`"
},
"present_wrong_yellow": {
"type": "noul",
"instructions": "Is there a wrongly placed brick in the box containing yellow bricks in the picture `photo`? A wrongly placed brick is a brick in that box whose colour is not yellow."
},
"box_wrong_yellow": {
"type": "box",
"instructions": "the wrongly placed brick in the box containing yellow bricks in `photo` - the brick in that box whose colour is not yellow"
},
"present_defect_yellow": {
"type": "noul",
"instructions": "Is there a cracked or dirty brick in the box containing yellow bricks in the picture `photo`?"
},
"box_defect_yellow": {
"type": "box",
"instructions": "the cracked or dirty brick in the box containing yellow bricks in `photo`"
},
"present_wrong_red": {
"type": "noul",
"instructions": "Is there a wrongly placed brick in the box containing red bricks in the picture `photo`? A wrongly placed brick is a brick in that box whose colour is not red."
},
"box_wrong_red": {
"type": "box",
"instructions": "the wrongly placed brick in the box containing red bricks in `photo` - the brick in that box whose colour is not red"
},
"present_defect_red": {
"type": "noul",
"instructions": "Is there a cracked or dirty brick in the box containing red bricks in the picture `photo`?"
},
"box_defect_red": {
"type": "box",
"instructions": "the cracked or dirty brick in the box containing red bricks in `photo`"
}
},
"assets": {
"d8ab3aba-photo": {
"type": "image",
"format": "jpeg",
"data_base64": "<35168 base64 chars>"
}
}
}
This translates effortlessly back to our supermarket use case. Using Box questions, cameras could locate misplaced products or damaged goods on the shelves. A Store Case is then automatically generated for a store associate, and the exact coordinates can be overlaid onto the image attached to the case. The worker knows exactly where to go and exactly which item needs to be removed or replaced, completely eliminating search time.
The returned coordinates drawn onto the shelf photo: one rotten pear among the good ones, marked on the case the store worker picks up.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
