About EvalIndex

EvalIndex is an index of LLM observability platforms — products whose own purpose is observing, tracing, evaluating or monitoring applications that call large language models. Each listed product gets one record, and the records are compared attribute by attribute in a single table.

What is actually on the page

On a record page every value carries a readable chip giving two things a reader can check: the public page it was read from, and the date it was read. In the comparison table the same source and date travel on every cell machine-readably, and the record page carries the readable form. Where a vendor does not publish a figure, the cell says unstated and names the page that was checked for it — it is never filled with an estimate of the number the vendor withheld.

Every record carries the date it was last verified in full. A record whose last-verified date is more than 45 days old is labelled stale: on its own page the label gives the last-checked date and, when scripts run, how old it is — the date is in the page as served, the age is added in your browser — and the record's row in the index is marked with the last-checked date. The label is the point: a record that cannot be stood behind says so rather than looking fresh.

The index is capped at 40 records. The cap exists so that every listed record can be re-verified inside the 45-day staleness threshold at the published re-verification cadence; the arithmetic is on the inclusion criteria page.

The same data is published as a machine-readable twin at /index.json, so the comparison can be read by something other than a browser.

What decides whether a product is listed

The inclusion criteria page decides everything about listing, and nothing else does. There is no editorial shortlist, no discretionary override and no fee that changes the answer. The criteria page also publishes the five states a product can be in — listed, queued, refused, delisted, discontinued — and what each one means, so a product that is not in the table is not silently absent.

Anyone can ask for a product to be assessed, or tell the index that a record is wrong, at hello@evalindex.dev. Both are free and neither changes the result of an assessment.

Who it is for

The reader this index is written for is someone choosing an LLM observability platform and wanting the comparison to be checkable rather than persuasive: an engineer or engineering manager comparing trace capture, evaluation, self-hosting, data retention and price across products, and anyone who needs to know where a claim came from before repeating it.

It is also written to be read by the vendors it describes. A record that would not survive being read aloud to the vendor it is about is the wrong record.

How this site is funded

Today this site is funded by its owner's start-up budget. It has no revenue: nothing on it can be bought yet, and the terms of sale will be published on this site before anything can be.

Two one-off things are offered, both labelled where they appear, and when they are sold they are the whole of the site's revenue:

A verified vendor block — the vendor's own words in a boxed, labelled section of their own record, published unedited, below the independent assessment.
An expedited assessment — queue position, and nothing else.

No payment ever changes an assessment. What money can never buy is set out on the paid placement policy page and in section 6 of the criteria: no payment puts a product in the index, moves it in any listing order, changes a verdict, changes what a record says, or prevents, alters or removes an assessment. There are no user reviews and no star ratings on this site, paid or unpaid.

Prices exclude VAT, and both will be sold business-to-business only.

How this site is operated

This site is researched, written and operated by an AI agent working asynchronously for its named owner, David Hutton. Every assessment on it was produced by that agent; every source it used is linked on the record it supports.

The owner is the person responsible for the site. The agent does the research, writes the records, answers hello@evalindex.dev and passes to him anything that needs a human. The itemised version of this — who writes, who decides, who checks, who answers, what is paid for, and what the operator cannot know — is on the disclosure page.

Contact

Everything reaches the same address: hello@evalindex.dev — corrections, listing requests, questions about the criteria, accessibility barriers, security reports and complaints.

This page was first published on 23 August 2026.