NEO AI · Evaluations

NEO AI Evaluations · How we test and measure LILI's capabilities

See what LILI can do. Verify how we measured it.

Explore the tasks LILI was tested on, the methods we used, and the results we recorded. Open a report to see the evidence behind it and exactly what each third party has verified.

Published MethodologyCryptographically Sealed ResultsIndependently Timestamped Proof
Latest public reportOfficial Test No. 2 · 18 November 202634 analysis tasks · 5 systems · 170 completed answers · Edition 2
4.9×lower cost per answer for NEO AUTO than Claude Opus 5.5 (0.018 vs 0.090 USD)Check the proof →
12 / 12cost and speed comparisons statistically significant, all in LILI's favourCheck the proof →
+0.95points out of 10: NEO AUTO judged better than GPT-5.5 (95% interval 0.28 to 1.63; n = 32 paired tasks)Check the proof →
12:10:44 UTCthe sealed results time-stamped by DigiCert, in the second the test endedCheck the proof →

What LILI demonstrated

Six things LILI did, in the scenarios we tested.

Each card comes from Official Test No. 2. Open it to see what we asked, what LILI did, how it was evaluated and what the result allows us to conclude, then go straight to the evidence.

blindblind 8.70 8.91 NEO AUTO vs reference: no significant difference

01 · Official Test No. 2

Reference-level analysis

8.70 vs 8.91judged quality /10, NEO AUTO vs Claude Opus 5.5

reference 0.090LILI 0.018 4.9×

02 · Official Test No. 2

The same work at a fraction of the cost

4.9×lower cost per answer: 0.018 vs 0.090 USD

LILI19 sref.32.9 s median time to a complete answer

03 · Official Test No. 2

Complete answers sooner

19.0 s vs 32.9 smedian time to a complete answer, 1.7× shorter

30 accepted4 to top tier34 tasks

04 · Official Test No. 2

Checks its own work before paying for more

30 / 34fast answers verified and accepted

RequestRouteFast modelVerifyAccepted top tier avoided 2 calls · 13.6 s · 0.00212 USD · judged 9.0

05 · Official Test No. 2

One request, opened completely

≈30×lower cost on this request: 0.00212 vs 0.06315 USD

PULSEAUTOPRIME speedbalanceeconomy

06 · Official Test No. 2

A profile for each kind of request

20× · 50×faster (PULSE) · cheaper (PRIME) than Claude Opus 5.5

Results

Cost, speed and quality, side by side.

The five systems answered the same 34 tasks on the same day, under the same timing, pricing and blind review. Switch the measure; the white line is the 95% interval.

Quality against costHigher is better; further left is cheaper (logarithmic cost axis)

Conditions: one run on 18 November 2026; times include the providers' queues; costs from the price table of the day; quality judged by AI models. Results of another test are comparable only under the same protocol. Explore methodology · Verify this report

Proof path

Follow the evidence, from the test run to your screen.

Check the published file against its fingerprint, inspect the timestamps and the ledger record, and see exactly what each third party confirmed. Select a step to open its proof.

Step 1 of 8 · NEO AI

Test run

34 frozen tasks answered by 5 systems through real calls at real prices, 18 November 2026, 10:27:25 UTC to 12:10:44 UTC. Every answer timed and priced by the platform, scored blind by two AI reviewers.

What it proves. Nothing on its own: this is the record that the next steps protect.

Who verified what

Every certification, with its exact scope.

Each service below confirms only what its service covers. A status is shown only when a proof you can open supports it.

Operated by NEO AIIntegrity checkable by anyone

Proof Ledger

NEO AI (built-in register; not a third party)

Role. The hash-chained, append-only register of NEO AI. Each entry contains the fingerprint of the one before it.

What it confirms. Proof Ledger entry no. 1285 has not changed since it was sealed, and it is chained to every entry before it. It holds the fingerprint of all 170 answers and 396 judgements of the test.

Applies to. Official Test No. 2: Proof Ledger entry no. 1285 (results of 170 answers and 396 judgements)

Scope. Operated by NEO AI: on its own it proves integrity, not a date. The date comes from DigiCert.

Technical detail
Entry
No. 1285 · sealed 18 November 2026, 12:10:44 UTC
Hash
b6aec2b0fd46d143fac82b8ba9c63e0edf960084c93b2fbb24456fd2baedb7f2
Previous hash
cdf86806c642e1438b1f34cc5987022e8b20301a9bf3ab61be773894ce1cc756
Results digest
fb70d29bf897f490f7e131efa6bf54a3b80ee30fde3d8693c5a59682e23a8c5b

Check it yourself, without NEO AI:

node verify-entry.mjs ledger-entry-1285.json
Independent third partyTimestamp confirmed

DigiCert (official site, opens a new tab)

DigiCert, Inc. - public certificate authority

Role. A public certificate authority that issues RFC 3161 time-stamps.

What it confirms. The fingerprint of Proof Ledger entry no. 1285 existed, unchanged, on 18 November 2026 at 12:10:44 UTC.

Applies to. Official Test No. 2: the hash of Proof Ledger entry no. 1285

Scope. DigiCert certifies a date for a fingerprint. It has not reviewed the report, the tasks or the scores.

Independent third partyTimestamp confirmed

FreeTSA (official site, opens a new tab)

FreeTSA (freetsa.org) - free RFC 3161 time-stamp authority

Role. A free, community-run time-stamp authority (RFC 3161).

What it confirms. The manifest of the evidence pack existed at the stated time. The manifest lists the PDF report, the per-task results and the ledger entry with their SHA-256 fingerprints.

Applies to. Official Test No. 2, evidence pack edition 2: the manifest d98d2df7d7eadfcf… (PDF report, per-task results, ledger entry)

Scope. Certifies a date for the manifest. It does not review its content.

Independent third partyFile integrity verified

Sigstore (Fulcio and Rekor) (official site, opens a new tab)

Sigstore - open-source project of the OpenSSF, Linux Foundation

Role. Fulcio gives the public GitHub workflow a short-lived signing certificate. Rekor records the signature in a public, append-only transparency log.

What it confirms. The manifest was signed by the public workflow of github.com/myneogroup-ai/neolabs-proofs, and that signature is recorded in the public Rekor log.

Applies to. Official Test No. 2, evidence pack edition 2: the manifest d98d2df7d7eadfcf… (PDF report, per-task results, ledger entry)

Scope. Confirms who signed the manifest and that it is unchanged since. It does not judge the figures inside it.

Technical detail
Signer identity
https://github.com/myneogroup-ai/neolabs-proofs/.github/workflows/proofs.yml@refs/heads/main
Rekor log index
4022392323
Logged at
20 November 2026, 08:33:07 UTC
Data signed
manifest d98d2df7d7eadfcf0175d4f9e927830d96471b917020476c03fdb857b77ad833

Check it yourself, without NEO AI:

cosign verify-blob --bundle proofs/manifest.sigstore.json --certificate-identity https://github.com/myneogroup-ai/neolabs-proofs/.github/workflows/proofs.yml@refs/heads/main --certificate-oidc-issuer https://token.actions.githubusercontent.com proofs/manifest.jcs
Independent third partyProvenance attested

GitHub Artifact Attestations (official site, opens a new tab)

GitHub, Inc. (through Sigstore)

Role. Signed build provenance issued by GitHub Actions and recorded in the Rekor log.

What it confirms. The published verifier and scorer (verify/*.mjs) come from the public workflow of the neolabs-proofs repository, at a given commit.

Applies to. The published verifier and scorer of the public proof repository (verify/*.mjs)

Scope. Confirms where the verification code comes from, not that the tested system performs well.

Technical detail
Checked
2026-11-20: GitHub attestations API: 5 provenance attestations for verify/*.mjs, recorded in Rekor (log index 4022392754 for score.mjs).

Check it yourself, without NEO AI:

gh attestation verify verify/score.mjs -R myneogroup-ai/neolabs-proofs
Independent third partyTimestamp confirmed

OpenTimestamps (Bitcoin) (official site, opens a new tab)

OpenTimestamps - open protocol and public calendars, anchored in Bitcoin

Role. A free, open time-stamping protocol anchored in the Bitcoin blockchain, with no company to trust.

What it confirms. Once confirmed: the manifest existed before a given Bitcoin block.

Applies to. Official Test No. 2, evidence pack edition 2: the manifest d98d2df7d7eadfcf… (PDF report, per-task results, ledger entry)

Scope. Confirms a date only, with no company to trust. Anyone can check it on their own node with ots verify.

Technical detail
Bitcoin block
976711
Data certified
manifest d98d2df7d7eadfcf0175d4f9e927830d96471b917020476c03fdb857b77ad833
Checked
20 November 2026, 09:33:37 UTC

Check it yourself, without NEO AI:

ots verify proofs/manifest.jcs.ots
Public hostPublic

Public proof repository (GitHub) (official site, opens a new tab)

Published by NEO TECH LABS TECHNOLOGY L.L.C. on GitHub

Role. Hosts the evidence packs, the verifier and the workflow that signs and time-stamps them, with their full public history. Code under MIT, data under CC BY 4.0.

What it confirms. Nothing by itself: it is where anyone can download the files and rerun the checks offline (node verify/cli.mjs verify).

Applies to. Every edition of the evidence pack and the code that verifies it

Scope. A host, not a certifier.

Not yet performed

Shown so that nothing is implied: these services or reviews are planned or waiting for a decision. None of them has confirmed anything yet.

  • Not yet performedSoftware Heritage (opens a new tab)Permanent archive of the public proof repository. Built into the publication workflow; not yet requested.
  • Not yet performedZenodo / DataCite (DOI) (opens a new tab)A permanent identifier (DOI) for each evidence pack. Built into the publication workflow and rehearsed; no permanent deposit yet.
  • Not yet performedHugging Face (opens a new tab)The per-task public results as an open dataset. Planned, not yet built.
  • Not yet performedOSF preregistration (opens a new tab)The protocol of a test registered before it runs, from Official Test No. 3. Planned.
  • Not yet performeddrand (League of Entropy) (opens a new tab)Public randomness to draw the audited sample. Built into the publication workflow; used from Official Test No. 3.
  • Not yet performedExternal auditNot yet performed. No external party has audited a test.
  • Not yet performedIndependent replicationNot yet performed. No outside party has reproduced a test.

Checking a document is not validating its results.

A

Verifying a document

A fingerprint (SHA-256) is published for each file. Recompute it on your copy: if it is the same, the file is exactly the registered one. Any change to the registered file can be detected by checking it against its published fingerprint.

Done by: you, in your browser or with one command.

B

Dating and registering it

A time-stamp authority or a public log confirms that a fingerprint existed at a given time and was signed by a given identity. It confirms only what its service covers: a date or a signature, not the content.

Done by: DigiCert, FreeTSA, Sigstore, OpenTimestamps.

C

Validating the results

Only an outside party that runs the published protocol and obtains comparable results confirms the measured performance. Checking a document does not do that.

State: not yet performed. No third party has reproduced or audited a test.

How we test LILI

Seven steps from a frozen task to a public proof.

Follow one official test from start to finish. Select a step, or let the journey play.

Step 1 of 7

Frozen task set

The tasks are written and frozen before the test. Every system receives exactly the same wording.

  • 34 original analysis tasks in English and French
  • Decision memos, risk assessments and reviews of evidence
  • Version v4-2026-11-17, locked before the first call
The detailed methodology sheet

Scope

34 original analysis tasks (decision memos, risk assessments and reviews of evidence), in English and French. Task set v4-2026-11-17, frozen before the test. Each task answered once by each of the five systems.

Systems

  • NEO AUTO - LILI, balanced profile: a first answer, an independent verification, and a stronger model only when needed. Through LILI.
  • NEO PULSE - LILI, speed profile: the fastest route to an answer. Through LILI.
  • NEO PRIME - LILI, economy profile: the lowest cost per answer. Through LILI.
  • Claude Opus 5.5 - Reference: a leading single model. One call, no orchestration.
  • GPT-5.5 - Reference: a leading single model. One call, no orchestration.

Cost

Tokens reported by the provider for each call, multiplied by the price table in force on the day. One line per call, including LILI's verification calls.

Time

Server timestamps at the start of each request, at the first text and at the final answer. The reported time is the median over the tasks.

Quality

Two independent AI models, from two providers that wrote none of the answers, scored each anonymised answer out of 10 without knowing its author. The score of an answer is the average of the scores returned for it.

Statistics

  • A comparison pairs one LILI profile with one reference model on the tasks both answered.
  • Paired sign-flip test, 20,000 flips, seeded; significant at 0.05 after Holm's correction over all 24 comparisons of the run.
  • At least 10 paired tasks; the size is the mean paired difference with its 95% bootstrap interval.
  • Anything else is reported as not shown, never as a tie or a win.

Proof

hash = SHA-256( prev_hash || canonical({id, at, kind, account_id, actor_id, run_id, conversation_id, message_id, payload}) ); canonical = JSON with object keys sorted and no spaces. The entry is time-stamped under RFC 3161 by DigiCert; the evidence pack is signed through Sigstore and time-stamped by FreeTSA.

Limits

  • One run, on one day, on the network of the server: times include the providers' queues and change from day to day.
  • Quality is judged by AI models, not by people: two reviewers from other providers than the answers, blind to the system; their agreement is reported.
  • Costs are priced from the tokens reported by each provider and the price table in force on the day.
  • Some providers do not stream their answers: time to first text is not compared in this test.

Check it yourself

Alter one figure. Watch the proof break.

The Proof Ledger shows that a record has not changed. Outside authorities show when it existed.

LEVEL 1 · PROOF LEDGER

Sealed in a hash-chained register

Every official test is recorded in the Proof Ledger, the built-in register of NEO AI. Cryptographic hashing makes any change to a sealed record detectable, so the integrity of the recorded results can be checked.

No. 1284

hash
cdf86806c642e143…
link
to no. 1283

No. 1285 · Official Test No. 2

answers
170
results
fb70d29bf897…
prev
cdf86806c642…
hash
b6aec2b0fd46d143…

Next entry

prev
b6aec2b0fd46d143…
link
intact
Recomputed: matches the sealed hash

This demonstration recomputes the real SHA-256 of entry no. 1285 in your browser. Altering one figure (170 answers becomes 171) changes the hash, and every later link breaks.

LEVEL 2 · INDEPENDENT TIMESTAMPS

Certified in time by outside authorities

In the second the campaign ended, the hash of its sealed entry was sent to DigiCert, a public certificate authority, under RFC 3161, the international standard for trusted time-stamps. DigiCert signed the hash with its own clock. Only 32 bytes left NEO AI: no question, no answer, no figure, no name. The evidence pack published afterwards is signed through Sigstore and time-stamped again by FreeTSA.

  • Any change is detectable. A change to an answer, a score or a cost changes the fingerprint; the token would no longer match.
  • The date cannot be moved. No one can obtain a token with an earlier time.
  • The whole history is covered. Each entry contains the hash of the one before, so one token covers all 1,285 entries up to this test.

Results timestamped by DigiCert · RFC 3161

Official Test No. 2 - proof of record

Time certified
Proof Ledger entry
No. 1285 (lab.campaign, sealed 2026-11-18 12:10:44.240 UTC)
Token serial
5da7bc65955b06f9c5efc6871bbf4a10
Hash signed
b6aec2b0fd46d143fac82b8ba9c63e0edf960084c93b2fbb24456fd2baedb7f2
Signed by
DigiCert SHA256 RSA4096 Timestamp Responder 2026 1 · RFC 3161 · root DigiCert Trusted Root G4
Verify this report
Third-party timestamp - doneVerified: DigiCert RFC 3161 token over ledger entry no. 1285.
External audit - not yet performedNot yet performed. The test has not been audited by an external party.
Independent replication - not yet performedNot yet performed. The test has not been reproduced by an outside party.

All reports

The public archive.

Every published report stays online at a permanent address, with every edition, its Public Edition and its proof files. Official Test No. 2 is the first public report.

PUBLISHED · 18 NOVEMBER 2026

Official Test No. 2

34 analysis tasks · 5 systems · 170 completed answers

LILI and two leading single models answered the same 34 analysis tasks through real calls at real prices. Every answer was timed and priced by the platform, scored blind by two independent AI reviewers, sealed in the Proof Ledger and time-stamped by DigiCert in the second the test ended.

  1. Edition 2 · 20 November 2026 Current

    Report text only (Public Edition PDF): the six comparisons not shown besides judged quality are the exact-match comparisons, not time-to-first-text comparisons. Figures changed: no.

    Manifest d98d2df7d7eadfcf0175d4f9e927830d96471b917020476c03fdb857b77ad833 · PDF 108b44330f0d5a81b566d14bb8f067980720c09e053368943ec15c74a6915779

  2. Edition 1 · 18 November 2026 Superseded

    First published edition.

    Manifest a58336b0134bd4387051c185113ee187fdbfe2b28edeae29be7276a82a08cea0 · PDF 149736a95c717e5482c5060c1168e5dbceff79dfe2cbf6fcd5bccf1ea5e2362b

When a new edition is published, its manifest names the fingerprint of the one it replaces, and receives its own signature and time-stamps. Earlier editions stay online with their proofs at their permanent addresses: each file always matches the fingerprint of its own edition.

Explore resultsVerify this reportDownload report