We ranked 527 posts with Jev and SubQ

A search for AI tools that reduce administrative work needs more than posts containing the word “AI”. It needs to distinguish an invoice-processing tool from a coding assistant, and an actual workflow from a promise about productivity.

Our Clairevue Signals search asks Jev to judge each candidate post against the reader's request. It sends groups of up to 64 posts, with one relevance question for each post and two requests running at once. The application then keeps scores of 0.85 or above.

We tested SubQ Snap as an alternative on the same 527 posts. Jev completed our batch-scoring workload faster. SubQ answered a single question over more than 100,000 input tokens in under a second.

Start with the request your application makes

Both products accept supplied information and return structured decisions. For search, we ask whether a particular post meaningfully addresses the request and get a number between zero and one. We don't need generated prose to rank the posts.

SubQ's launch announcement claims roughly one-second decisions at 100K tokens, up to 384K context and better decision quality than Jev on several benchmarks.

For search-related tasks, the announcement reports Snap ahead of Jev on Amazon ESCI product relevance (48.7 versus 43.8) and BRIGHT document ranking (44.3 versus 40.6). It reports Snap leading on 13 of the 16 enterprise benchmarks, with Jev ahead on the other three. Those are SubQ's reported scores; we haven't independently reviewed or reproduced the benchmarks.

We compared jev-1.13.0 and subq-snap-preview using the same relevance instructions and post content across twelve search requests. Some requests filtered the candidates first, such as engineering jobs in Canada. An unfiltered search needed nine batches to score all 527 posts.

Fewer calls didn't give us the fastest search

For the administrative-tools request, the matched batch configuration took 1.330 seconds with Jev and 22.024 seconds with SubQ. Both scored every candidate.

We also repeated the same 64-post request three times per provider:

64 posts and 64 relevance questions	Jev	SubQ
Median client time	0.238 seconds	2.065 seconds

These are client-observed API timings, including the network trip, rather than full production search times. We didn't control provider queues or caches, and three repeats can't estimate production tail latency.

SubQ's larger context window offered another possibility: send the entire 527-post set and all 527 relevance questions in one request.

That took 9.421 seconds. A fresh Jev run using nine batches took 1.742 seconds. SubQ's single request improved on its own 22.024-second batched result for this search, but it still didn't beat Jev's chunked run.

We kept the posts and questions unchanged; we changed how much of the workload one request carried. That compares two application designs, with different request sizes.

Ask one question over the same posts

Our search asks hundreds of separate questions. To check SubQ's large-input claim, we kept the supplied posts and asked about just one target post.

With 64 posts in the input, two probes took 0.203 and 0.272 seconds with Jev, versus 0.477 and 0.226 seconds with SubQ. The large gap from the 64-question test no longer appeared in those observations.

We then gave SubQ all 527 posts and asked one relevance question:

Target post in the input	Reported input tokens	SubQ client time
Beginning	106,738	0.540 seconds
Middle	106,738	0.554 seconds
End	106,738	0.412 seconds

SubQ returned a valid relevance probability within a second in all three probes. We had already sent related context, so this doesn't establish uncached performance. Judging one post also doesn't prove difficult reasoning across documents.

We didn't attempt the same large input with Jev. Its documented limits distinguish a 32K budget for state plus the longest question from 64K for state plus all questions.

Check which posts survive your cutoff

Speed alone can't tell us whether a search returns useful posts. We checked the selected results against AI-assisted relevance judgments that haven't been independently verified. These comparisons can guide further testing, but they don't establish an accuracy winner.

At our existing 0.85 cutoff, the two models selected:

Across comparisons with a clear reference judgment	Jev	SubQ
Accepted post/request pairs	35	41
Accepted pairs judged relevant	34	31
Accepted pairs judged irrelevant	1	10
Relevant share of accepted pairs	97.1%	75.6%
Relevant reference pairs missed	39	42

A post judged for two searches counts twice. Direct and partial matches count as relevant; uncertain judgments are excluded. The percentages describe the selected results, not overall accuracy, and depend on those unverified judgments.

Using the same cutoff with SubQ admitted more irrelevant matches and missed slightly more relevant ones according to that reference. SubQ might need a different threshold, but we haven't demonstrated which one would improve the trade-off. Raising the cutoff could also discard useful posts.

Before carrying a score threshold to another model, check both the posts it accepts and the useful posts it leaves out.

What the token counts cost at published rates

The input rates used for this test were $0.042 per million tokens for Jev and $0.05 for SubQ Snap; both documented free outputs. Different tokenizers mean the same text can produce different billed counts.

For the fresh 527-post comparison, Jev's batches cost about $0.00785 at those rates, versus $0.00883 for SubQ's single request. The simpler request arrangement didn't save money in that example. These are estimates from reported token counts, not verified invoice totals.

SubQ exposes a preview model name, so results may change after a provider update. Its documented retention policy keeps prompts indefinitely by default unless the organization changes its policy; check that policy before sending sensitive data.

Before replacing a decision API

Replay the requests your application makes, including its number of questions and its concurrency settings. Then test a different arrangement only if it could serve the same job.

For our search, this pilot gives us no measured reason to replace Jev for speed. If your workflow needs one judgment over a large input, test SubQ against reviewed examples from that workflow. Choose a cutoff on development examples and check it on unseen cases before changing production.