Someone says something bad and by the time a mod sees it, everyone in the chat already has.
Most AI moderation setups use an LLM for this. That works, but an LLM is usually doing way more than we need here and is way slower.
For moderation, I mostly want one thing: should I remove this message or not? So I tried Jev, a decision model from TypeSafe. I guess you're already familiar with the model, as its been over a week the model dropped.
It does not generate text. You give it a typed question, and it gives you a probability back.
I called Jev through Composio's Jev toolkit, so I didn't have to set up a separate HTTP client or keep the Jev API key inside the code.
If you want to play around with the project, or build on top of it, you can find the source code here:
Here's what we'll go through: What Jev is and how it is different from a normal LLM (in short) Setting up Jev through Composio Building a Discord bot that removes such messages Building the same moderation flow for Twitch with EventSub A quick YouTube version (I leave that upto you)😉
⚠️ NOTE: This is still an AI bot, so you might want to keep human mods around. But considering how cheap it is, at $0.042 per million input tokens with output basically free, it could be worth trying if you're a streamer or run a large live chat. What's Jev, Anyway?
You send Jev a state, which for us is the chat message, along with one or more questions. It can answer all of those questions in the same call.
Jev currently gives three question types: Noul - Ask whether something is true and get a probability between 0 and 1. Choice - Give it a few options and let it pick one, along with probabilities. Score - Ask it to place something somewhere on an ordered scale of 2 to 10 options.
Is this a hostile comment at a person? And instead of getting a paragraph back, you get something like:
To give you an approximate, moderating 1,000 messages (roughly counting ~400 input token per msg) comes out to less than 2 cents, almost free.
Only the part that receives and deletes messages changes. Discord: discord.js listens for messages and deletes them. Twitch: EventSub sends us messages over WebSocket, and the Helix API deletes them.
💁 I've never used Twitch, so I'm not sure how good this works in a long term compared to Discord. We also need a way to properly test the bots.
So I added a small flooder script that sends a bunch of normal/racist messages into the test chat.
💡 Why not use Composio for Discord too? Composio's Discord toolkit does not currently have a real-time new-message trigger. Polling would add extra delay, so I use the native SDK for incoming chat and Composio for Jev. Tech Stack
The project is pretty small. Jev (TypeSafe) - Decides whether the message is hostile and gives us a probability. Composio - Handles the Jev connection and gives us one tools.execute() call. discord.js - Listens for and deletes Discord messages. Twitch EventSub + Helix - Receives Twitch chat messages and deletes them when needed. Node.js 22 - Everything runs as plain .mjs files, so there is no framework or build step (perfect to test right away).
Node 22 already includes fetch and WebSocket, which also keeps the Twitch side pretty clean.