NEO AI · NEOLABS

NEOLABS · Official LILI Capability Tests

Official Test No. 2 - 18 November 2026

34 analysis tasks · 5 systems · 170 completed answers

LILI and two leading single models answered the same 34 analysis tasks through real calls at real prices. Every answer was timed and priced by the platform, scored blind by two independent AI reviewers, sealed in the Proof Ledger and time-stamped by DigiCert in the second the test ended.

Published MethodologyCryptographically Sealed ResultsIndependently Timestamped Proof
4.9×lower cost per answer for NEO AUTO than Claude Opus 5.5 (0.018 vs 0.090 USD)
12 / 12cost and speed comparisons statistically significant, all in LILI's favour
+0.95points out of 10: NEO AUTO judged better than GPT-5.5 (95% interval 0.28 to 1.63; n = 32 paired tasks)
12:10:44 UTCthe sealed results time-stamped by DigiCert, in the second the test ended

Results

Reference-level quality at about one fifth of the cost.

NEO AUTO, LILI's balanced profile, against Claude Opus 5.5, the strongest reference on this test.

NEO AUTO against the strongest reference - 34 tasks; quality on 32 scored tasks per system
MeasureNEO AUTOClaude Opus 5.5Reading
Cost per answer0.018 USD0.090 USD4.9× lower cost
Time to a complete answer (median)19.0 s32.9 s1.7× shorter time
Judged quality (out of 10)8.708.91No statistically significant difference detected in judged quality on this test.

Judged quality: No statistically significant difference detected in judged quality on this test. Against GPT-5.5, NEO AUTO was judged better by +0.95 points out of 10 (95% interval 0.28 to 1.63; n = 32 paired tasks; significant after Holm's correction).

All five systems - quality: mean [95% interval] and scored tasks; cost: average per answer; time: median to a complete answer
SystemJudged quality /10Cost per answerMedian timeBest suited to
NEO AUTO8.70 [8.30-9.03] n=320.0182 USD19.0 sDemanding work where quality must hold
NEO PULSE6.26 [5.64-6.86] n=330.0116 USD1.6 sInstant replies, simple requests
NEO PRIME5.32 [4.73-5.94] n=330.0018 USD2.8 sVery high volume at minimal cost
Claude Opus 5.58.91 [8.56-9.20] n=320.0896 USD32.9 sReference
GPT-5.57.82 [7.26-8.35] n=330.0841 USD26.2 sReference

Comparator

Cost, speed and quality, side by side.

Switch the measure. Bars show the value; the white line, its 95% interval.

Quality against costHigher is better; further left is cheaper (logarithmic cost axis)
Ratios of the average cost and of the median time, rounded. All twelve underlying differences are statistically significant.
LILI profileCost vs Claude Opus 5.5Cost vs GPT-5.5Time vs Claude Opus 5.5Time vs GPT-5.5
NEO AUTO4.9×4.6×1.7×1.4×
NEO PULSE7.8×7.3×20×16×
NEO PRIME50×47×12×9.5×

Statistics

12 of 12 cost and speed comparisons significant, all in LILI's favour.

Every comparison is paired on the same task and corrected for the 24 comparisons made at once. 17 of 24 are significant, 13 of them in LILI's favour.

Dot: mean paired difference over 34 tasks. Bar: 95% interval. A bar that does not touch zero is significant.

The 12 significant cost and speed comparisons (mean paired difference, 95% interval, paired tasks)
ComparisonMeasureDifference95% intervaln
NEO AUTO vs Claude Opus 5.5Total time10.73 s faster7.63 to 14.0134
NEO AUTO vs Claude Opus 5.5Cost per answer0.0714 USD cheaper0.0506 to 0.096734
NEO AUTO vs GPT-5.5Total time8.77 s faster3.58 to 14.0034
NEO AUTO vs GPT-5.5Cost per answer0.0659 USD cheaper0.0488 to 0.083934
NEO PRIME vs Claude Opus 5.5Total time26.12 s faster22.60 to 29.6834
NEO PRIME vs Claude Opus 5.5Cost per answer0.0878 USD cheaper0.0716 to 0.110234
NEO PRIME vs GPT-5.5Total time24.17 s faster17.95 to 31.0234
NEO PRIME vs GPT-5.5Cost per answer0.0823 USD cheaper0.0664 to 0.100134
NEO PULSE vs Claude Opus 5.5Total time29.14 s faster25.48 to 32.9334
NEO PULSE vs Claude Opus 5.5Cost per answer0.0781 USD cheaper0.0624 to 0.097234
NEO PULSE vs GPT-5.5Total time27.18 s faster21.99 to 33.1934
NEO PULSE vs GPT-5.5Cost per answer0.0726 USD cheaper0.0576 to 0.089234
Judged quality: the 5 significant comparisons
ComparisonDifference /1095% intervaln
NEO AUTO vs GPT-5.5+0.95 points (LILI judged better)0.28 to 1.6332
NEO PRIME vs Claude Opus 5.5-3.64 points (reference judged better)-4.41 to -2.9132
NEO PRIME vs GPT-5.5-2.50 points (reference judged better)-3.12 to -1.8933
NEO PULSE vs Claude Opus 5.5-2.72 points (reference judged better)-3.44 to -1.9232
NEO PULSE vs GPT-5.5-1.56 points (reference judged better)-2.19 to -0.8932

Seven comparisons are not shown: NEO AUTO against Claude Opus 5.5 on judged quality (no statistically significant difference detected in judged quality on this test) and the six time-to-first-text comparisons, left for a coming test once every profile streams its answers. Under NEOLABS rules a result that is not significant is never reported as a tie or a win. NEO PULSE and NEO PRIME are built for instant and high-volume work; on these demanding analysis tasks the references keep the lead on judged quality, which is why LILI fits the profile to the request.

83%

of judgements returned: 281 of 340. 32 of 34 tasks have every system scored.

76.3%

of the 118 answers scored by both reviewers are within one point (mean difference 1.04; correlation 0.718).

78.1%

within one point on the position check (32 pairs), which guards against order effects.

The eight NEO layers

4 of 8 layers at work in this test.

LILI is a layer of intelligence above the best models of every provider: it decides how much compute each request deserves, verifies the result, remembers, and proves what it did.

LAYER 1

Dynamic Capability Routing

Reads each request and selects the tier, the tools and the data it needs, before any model is paid.

✓ At work: 34 requests routed: 32 to the fast tier first, 2 directly to the top tier
LAYER 2

Adaptive Model Cascade

A fast, low-cost model answers first. A top-tier model is called only when the answer needs it.

✓ At work: 32 first answers by the fast tier; 2 escalated to the top tier
LAYER 3

Verification-Driven Compute

An independent model from another provider scores the answer. Evidence decides whether more compute is spent.

✓ At work: 32 answers verified and accepted
LAYER 4

Memory and Context Optimization

LILI sends the facts that matter from its memory graph and a running summary, instead of the whole history.

✓ At work: 34 requests answered with graph facts
LAYER 5

Mixture of Agents

For expert work: up to three models from different providers answer in parallel, then one synthesis and one critic.

Built; not at work on this test
LAYER 6

Speculative Parallel Intelligence

Two approaches are sketched in parallel and scored blind; the stronger one is developed.

Built; not at work on this test
LAYER 7

Semantic Compute Reuse

A verified answer to an equivalent question is reused for the same person instead of being computed again.

Built; not at work on this test
LAYER 8

Hybrid Execution

Designed to place models on API, private or self-hosted capacity. Today every model runs through APIs.

Built; not at work on this test

Layers 5 to 8 are built and used by other profiles and request types; they were not at work on this test.

LILI decisions

The saving comes from the calls LILI did not make.

The saving comes from calls that were not made: on 30 of 34 tasks LILI established by verification that a fast-tier answer was good enough. Every escalation is counted in the averages: the 4.9× advantage is net of them.

What LILI decided, across the 34 tasks (NEO AUTO)
What LILI didTasksTypical costTotal
First answer by a fast-tier model, verified and accepted; top-tier call avoided300.0027 USD0.123 USD
First answer judged insufficient by LILI; escalated to the top-tier model20.141 USD0.281 USD
Request routed directly to the top-tier model, then verified20.107 USD0.215 USD
NEO AUTO, 34 answers340.018 average0.619 USD
Claude Opus 5.5, 34 answers340.090 average3.047 USD

One answer, opened completely: Task 19

A founder wants one assistant that remembers users across five different products. Give the strongest argument for a single shared memory, the strongest argument for keeping each product's memory separate, and the design you would recommend, with the consent and deletion rules it needs.
  1. 1Request arrives

    The question is received with its context. Nothing is sent to a model yet.

    0 USD
  2. 2Dynamic Capability Routing

    LILI classifies the request and selects a fast-tier model for the first answer.

    0 USD - no model call
  3. 3First answer

    The fast-tier model writes the full answer: 2,054 output tokens.

    0.00156 USD
  4. 4Verification

    An independent model from a different provider scores the answer: 1.0 out of 1.0, accepted.

    0.00056 USD
  5. 5Adaptive Model Cascade

    The cascade was armed. Because the answer was accepted, the top-tier model was not called.

    call avoided
About 30× lower cost than Claude Opus 5.5 on this request, for the same judged score (9.0). Recorded in Proof Ledger entry no. 1143.
SystemCallsTimeCostJudged /10
NEO AUTO213.6 s0.00212 USD9.0
Claude Opus 5.5130.8 s0.06315 USD9.0
GPT-5.5134.1 s0.08114 USD7.0

Integrity

Sealed, chained and time-stamped.

The fingerprint of all 170 answers and 396 judgements was sealed in one ledger entry, then certified in time by DigiCert.

LEVEL 1 · PROOF LEDGER

Sealed in a hash-chained register

Every official test is recorded in NEOLABS' built-in Proof Ledger. Cryptographic hashing makes changes to sealed records detectable, so the integrity of the recorded results can be checked.

No. 1284

hash
cdf86806c642e143…
link
to no. 1283

No. 1285 · Official Test No. 2

answers
170
results
fb70d29bf897…
prev
cdf86806c642…
hash
b6aec2b0fd46d143…

Next entry

prev
b6aec2b0fd46d143…
link
intact
Recomputed: matches the sealed hash

This demonstration recomputes the real SHA-256 of entry no. 1285 in your browser. Altering one figure (170 answers becomes 171) changes the hash, and every later link breaks.

LEVEL 2 · INDEPENDENT TIMESTAMP

Certified in time by DigiCert

In the second the campaign ended, the hash of its sealed entry was sent to DigiCert, a public certificate authority, under RFC 3161, the international standard for trusted time-stamps. DigiCert signed the hash with its own clock. Only 32 bytes left NEO AI: no question, no answer, no figure, no name.

  • The results cannot be changed afterwards. Any change to an answer, a score or a cost changes the fingerprint; the token would no longer match.
  • The date cannot be moved. No one can obtain a token with an earlier time.
  • The whole history is covered. Each entry contains the hash of the one before, so one token covers all 1,285 entries up to this test.

Results timestamped by DigiCert · RFC 3161

Official Test No. 2 - proof of record

Time certified
Proof Ledger entry
No. 1285 (lab.campaign, sealed 2026-11-18 12:10:44.240 UTC)
Token serial
5da7bc65955b06f9c5efc6871bbf4a10
Hash signed
b6aec2b0fd46d143fac82b8ba9c63e0edf960084c93b2fbb24456fd2baedb7f2
Signed by
DigiCert SHA256 RSA4096 Timestamp Responder 2026 1 · RFC 3161 · root DigiCert Trusted Root G4
Third-party timestamp - doneVerified: DigiCert RFC 3161 token over ledger entry no. 1285.
External audit - not yet performedNot yet performed. The test has not been audited by an external party.
Independent replication - not yet performedNot yet performed. The test has not been reproduced by an outside party.

Timeline

From a frozen task set to a certified record.

  1. Task set v4 frozen

    34 original analysis tasks version-locked before the test.

  2. Campaign starts

    The first of 170 live calls leaves the Model Lab.

  3. Campaign ends

    170 answers and 396 judgements stored; their fingerprint is computed.

  4. Sealed in the Proof Ledger

    Entry no. 1285, chained to the 1,284 entries before it.

  5. Time-stamped by DigiCert

    RFC 3161 token, serial 5da7bc65955b06f9c5efc6871bbf4a10. Only the 32-byte hash left NEO AI.

  6. Published by NEOLABS

    Methodology, results and proof files made public.

Method & limits

What this test shows, and what it does not.

How every figure was produced

No figure was typed by a person. Each one is computed from records written by the platform while the test ran: the frozen task set, live production calls, the tokens reported by each provider, the server's timing events, the blind review and the statistical engine.

  • A comparison pairs one LILI profile with one reference model on the tasks both answered.
  • Paired sign-flip test, 20,000 flips, seeded; significant at 0.05 after Holm's correction over all 24 comparisons of the run.
  • At least 10 paired tasks; the size is the mean paired difference with its 95% bootstrap interval.
  • Anything else is reported as not shown, never as a tie or a win.

Limits

  • One run, on one day, on the network of the server: times include the providers' queues and change from day to day.
  • Quality is judged by AI models, not by people: two reviewers from other providers than the answers, blind to the system; their agreement is reported.
  • Costs are priced from the tokens reported by each provider and the price table in force on the day.
  • Some providers do not stream their answers: time to first text is not compared in this test.
  • Each LILI profile is measured as configured on 18 November 2026; another configuration needs another run.
  • The time-stamp certifies that the recorded results existed, unchanged, at the stated time. DigiCert provides that public service and has not reviewed this report. The test has not been audited or reproduced by an external party, and results may differ on other tasks, dates or configurations.
  • Claude Opus 5.5 and GPT-5.5 are products of their respective owners, used as references through their public interfaces.

Next milestones

Breadth

More task families after 34 analytical tasks: reasoning, code, long documents and more languages.

Review

Two independent AI reviewers score blind today; human experts will be added on a sample.

Precision

Larger samples, to measure the gap with the strongest reference more finely.

Independence

Date and integrity are certified by a third party; next, the test reproduced by an outside party.