Queue and assessed set
The inclusion criteria say a product here is in one of five states — listed, queued, refused, delisted or discontinued — and that each state is published with the dates that decide it. This page is those lists. It exists so that the rules on the criteria page can be checked rather than taken on trust: the order of the queue against the rule that sets it, the assessed set against the launch cut, and each refusal against the sources it was decided on.
Page generated on . Every row carries its own dates.
Nothing on this page is a ranking. One thing here will be purchasable: a place in the queue of products not yet published can be expedited for a fee (paid placement policy); nothing can be bought until the terms of sale publish. That fee moves nothing else on this page and nothing anywhere else on this site — not eligibility, not whether a product is listed at all, not what a record says, and not the verdict.
Being in the queue is not a finding against a product; a refusal is a finding about the public evidence on a stated date. The criteria page says both of those things as rules, and this page holds the dates.
The queue
The queue is first in, first out: a product's place is set by the date it entered — the date we received the request, or, for a product we found ourselves, the date of the assessment sweep that found it — and an earlier date takes an earlier place, with alphabetical order of product name breaking a tie on the same date. The table below shows every waiting product with the date it entered and whether it has already been assessed, so the order can be checked against that rule.
| Product | Entered the queue | Where it stands |
|---|---|---|
| W&B Weave | Assessed on 22 August 2026 and passed every rule in sections 1 and 2; it publishes when the next publication slot is worked, with criteria 2.3 and 2.4 tested again on that date. |
Published records
Each record's publication date, in the order the comparison table lists them. Read with the queue above, these dates show the real rate at which the index grows rather than a rate we assert.
| Record | Published |
|---|---|
| Arize AX | |
| Braintrust | |
| Confident AI | |
| Galileo | |
| Helicone | |
| HoneyHive | |
| Laminar | |
| Langfuse | |
| LangSmith | |
| Opik | |
| Portkey | |
| Pydantic Logfire |
The assessed set
EvalIndex assessed 15 products on 22 August 2026, the sweep that produced the launch index. Thirteen passed every rule in sections 1 and 2 of the criteria: 12 of them were published when the site went live on 23 August 2026, and the thirteenth entered the queue. The twelve were chosen by alphabetical order of product name — section 3 of the criteria says so, and says why: it is the one cut no product can influence and every reader can check. It was a one-time cut and has governed nothing since. Two products were refused under a named rule. Every product assessed on that date is below with its outcome, including the ones that did not become records.
| Product | Assessed | Outcome |
|---|---|---|
| Arize AX | Listed | |
| Braintrust | Listed | |
| Confident AI | Listed | |
| Fiddler AI | Refused under criterion 2.1 | |
| Galileo | Listed | |
| Helicone | Listed | |
| HoneyHive | Listed | |
| Laminar | Listed | |
| Langfuse | Listed | |
| LangSmith | Listed | |
| Opik | Listed | |
| Portkey | Listed | |
| Pydantic Logfire | Listed | |
| Traceloop | Refused under criterion 2.3 | |
| W&B Weave | Queued |
Refused
A refusal is a finding about the public evidence on that date, not a
verdict on the product's quality. Each refusal below names the rule that
was not met, what we read to decide it and the date we read it, so
anyone can repeat the check. Either product is assessed again on request
or on new public evidence — write to hello@evalindex.dev — and a
refusal carries no fee, no waiting period and no prejudice to the
result of that later assessment.
Fiddler AI — refused under criterion 2.1, Self-serve obtainable
We read the vendor's pricing page on 22 August 2026 and found that every tier, the free one included, carries a "Request demo" call to action. That page is the one we cite below, and it is the page we looked at for a self-serve route to an account; we found none on it that day. Criterion 2.1 asks for a route a reader can take without speaking to anyone at the vendor, and that is the route we did not find on that page on that date.
What we read, and when:
Traceloop — refused under criterion 2.3, Active maintenance, dated
We read the vendor's blog on 22 August 2026 — the page we cite first below — looking for a dated release, changelog entry, product-update post or last-updated date for the hosted platform at app.traceloop.com, and found none that day; the newest post of any kind we saw there was dated March 2026. The dated changelog we did find is the second page we cite: OpenLLMetry's CHANGELOG.md, whose newest entry was v0.62.3, dated 10 August 2026. That is the changelog of a separately named open-source code base in its own repository, which section 1 of the criteria excludes as evidence of the hosted product's maintenance. Criterion 2.3 asks for a dated artefact about the product on the record, and that is what those two pages did not give us on that date.
What we read, and when:
Delisted
No record has been delisted. Delisting is what happens at the end of a 30-day mark: a listed record found to have stopped meeting a criterion is marked on its own face, keeps its place in the comparison table for up to 30 days, and leaves the table for this list only if it still fails when those 30 days are up (section 4 of the criteria). The first records here published on 23 August 2026, so no mark has yet been made and no 30-day period has yet run.
When a record is delisted it appears here with the criterion it stopped meeting, the date it was marked, the date it left the comparison table and a link to its record page, which stays at the same URL and states the same three facts. A delisted product is still alive and still sold, and it returns to the table at the next re-verification that finds every rule met again.
Discontinued
No product has been marked discontinued. That mark needs one of two things: a vendor publishing that the product is discontinued, end-of-life or no longer sold, or the product's site, documentation or domain failing on three checks made at least 7 days apart (section 4 of the criteria). The 15 assessments made on 22 August 2026 found every product's own site loading over HTTPS and serving its own content, and found no vendor statement of discontinuation.
When a product is marked discontinued it appears here with what we checked, the dates of those checks and the date the mark was made, and its record page stays at the same URL saying the same. Discontinued and delisted are different findings and we never publish one as the other: a product on this list is gone, not merely out of the table.