Cortrix v1.0.0-rc.2 is available as a prerelease for local evaluation and integration testing. This update brings better FiQA retrieval results in the published comparison, Apache 2.0 licensing for project code, and practical fixes across ingestion and agent integration.
Let's start with the numbers. In the published FiQA comparison, Recall@10 increased from 0.3635 to 0.4523, a relative gain of 24.4%. nDCG@10 increased from 0.2526 to 0.3125, or 23.7%. The compared configuration uses embeddings and a cross-encoder reranker, without calling an LLM.
Those results matter because a retrieval system has to get useful material in front of the user or agent before anything can be done with it. But the numbers need their context: these are observations from the July and August measurement rounds, not a controlled experiment that isolates one code change. They measure retrieval quality, not final answer accuracy or business outcomes.
The other changes are about making the project easier to evaluate and work with. Cortrix-authored RC2 material now uses Apache 2.0. We have also tightened the behavior of long ingestion jobs, request handling, and diagnostics. The Core release and its paired Benchmark Evidence release are both public.
FiQA: more relevant material in the first ten results
FiQA contains financial questions and answers. Its terminology and varied question phrasing make it a useful retrieval task to inspect when thinking about research assistants, financial support tools, or internal knowledge search. That connection describes the task; it does not establish that a benchmark covers every requirement of those applications.
Recall@10 asks how much of the judged relevant material appears in the first ten results. nDCG@10 also considers where relevant material appears in that ranking. A system can find a useful passage and still make the user work too hard to reach it, so both measures are worth keeping in view.
| Metric | July | August | Relative change |
|---|---|---|---|
| Recall@10 | 0.3635 | 0.4523 | +24.4% |
| nDCG@10 | 0.2526 | 0.3125 | +23.7% |
The July profile is no_llm, labeled Embedding + Reranking. The August comparison uses cross_encoder_rerank, also labeled Embedding + Reranking. It does not use the separate August cross_encoder_rerank_full arm or a configuration labeled Full Stack. The distinction matters because those configurations have different results.
Neither compared profile calls an LLM during ingestion or retrieval. There is still embedding and cross-encoder computation. The useful point is that this retrieval path does not depend on an external generative model or an LLM token bill for each query.
The published rounds retain the same corpus, public profile, and @10 cutoff definitions for this comparison, but the builds, configuration, and execution environments differ. Treat the gain as a comparison between those published results. It does not identify a single cause or establish the same gain for another workload.
What the four datasets tell us
The evidence package covers different retrieval tasks. We use that variety to see where a configuration helps and where it may be the wrong choice.
| Dataset | Documents | Queries | What to examine |
|---|---|---|---|
| SciFact | 5,183 | 300 of 300 | Scientific evidence retrieval. nDCG@10 +2.9%; Recall@10 +4.1%. |
| NFCorpus | 3,633 | 323 of 323 | Biomedical retrieval. nDCG@10 +4.6%; Recall@10 +5.9%. |
| FiQA | 57,638 | 648 of 648 | Financial questions. nDCG@10 +23.7%; Recall@10 +24.4%. |
| Quora | 522,931 | 2,000 of 10,000 | Duplicate questions. New in this package; no July comparison is presented. |
The percentage changes above compare the same two published cross-encoder profiles as the FiQA table. Across SciFact, NFCorpus, and FiQA, all six reported metrics moved upward. Taking the unweighted mean of the three dataset scores in each round, the mean nDCG@10 increased by 7.9% and the mean Recall@10 by 10.0%. These are changes in the mean scores, not averages of the three percentage gains.
The August package contains 16 measured configurations and five arm definitions overall. Not every corpus has every arm. The measurements accompany RC2, but the final release tag is distinct from the measured Core build; the evidence release records the exact identities. This article is not reporting a new benchmark run on the final release commit.
Quora: a relevant question is not always a duplicate
Quora tests whether questions express the same intent. That makes it useful for thinking about FAQ deduplication or finding repeated support requests. A passage can discuss the right topic while still answering a different question.
In this measurement, enabling the cross-encoder reranker on the same Quora namespaces lowered nDCG@10 by 59.4% and Recall@10 by 56.9% relative to the configuration without reranking. Adding DeepSeek listwise reranking then recovered 49.6% in nDCG@10 relative to the cross-encoder result, while Recall@10 stayed roughly flat. Both metrics remained below the result without reranking.
| Quora configuration | nDCG@10 | Recall@10 |
|---|---|---|
No reranking (dense_only) | 0.5003 | 0.7142 |
Cross-encoder (cross_encoder_rerank_full) | 0.2031 | 0.3078 |
Listwise (llm_listwise) | 0.3039 | 0.3092 |
The Quora configuration labeled Full Stack covers listwise reranking during the query on namespaces without LLM enrichment at ingestion. It should not be read as the same setup as every other Full Stack result.
The lesson for us is practical: test a ranking strategy against the task it is supposed to help. RC2 exposes and verifies the per-request rerank=false path so callers can opt out when the reranker is unsuitable. The namespace-level reranker_config.enabled setting is not wired into the live query path; the per-request flag is the verified option. See the upgrade notes before changing an existing setup.
Quora uses the full corpus with a deterministic subset of 2,000 judged queries, not the complete set of 10,000. These measurements concern retrieval at top_k=10.
Apache 2.0: a more familiar basis for adoption
Cortrix-authored material in RC2 is now available under Apache License 2.0. Current Benchmark runner, test, and tool code also uses Apache 2.0.
We want to reduce friction for teams evaluating the project, making internal changes, or integrating it into a product. That does not make the license irrelevant: redistribution still has conditions, including keeping applicable license and notice information. The license text is the place to check those terms.
- The historical
v1.0.0-rc.1release, tags, and artifacts retain their original AGPL-3.0-only terms. - The RC2 change applies to Cortrix-authored material. Third-party material retains its identified license.
- Benchmark documentation, curated results, schemas, and provenance remain under CC-BY-4.0. The code change does not relicense the datasets.
The public Benchmark license map lists the exact paths and exceptions.
The practical fixes behind RC2
Beyond the retrieval comparison and license change, RC2 brings together work on the parts of a system that are easy to overlook until something fails. Here are the changes I would look at first when evaluating it.
Long ingestion jobs and task cleanup
Deep queue dispatch and managed input release no longer do the same amount of unnecessary work in the task manager's critical path. Task identity, terminal state transitions, cancellation races, cleanup scheduling, and duplicate submission handling have also been tightened. These changes address concrete problems in keeping a large ingestion job and its temporary data under control. The public implementation records include queue dispatch, managed input release, and enrichment deduplication.
Retrieval choices and source identity
The verified rerank opt-out lets a query choose a different path when a relevance reranker is a poor fit. HyPE-derived matches also preserve the originating document identity in public query results, helping callers get back to the source. See reranking guidance and the source identity fix.
MCP calls that are easier to trace
The local MCP server supports modern SDK v2 stdio discovery while retaining legacy initialization compatibility. Session, trace, and agent identity are passed through to Cortrix Server, making the request path easier to inspect when an integration behaves unexpectedly. This is the documented local stdio scope, not a claim that every transport or client is supported. See identity propagation and discovery compatibility.
Errors that surface more clearly
Parser bridges reserve stdout for the JSON response so dependency output cannot corrupt the protocol. Oversized sparse vectors are rejected, and memory scope validation rejects unknown values instead of silently taking another path. Configuration, operation log, and GC CLI checks also received fixes. These changes help surface bad inputs and failed operations. The release notes link each change to its public record.
Deployment defaults and tests
Docker Compose remains bound to loopback by default. An explicit address override supports isolated test networks, while production Web dependencies received security updates. That option does not create a complete authenticated public deployment path. The test work also replaced 26 assertions that could not meaningfully catch the intended defects and improved the CI path for documentation changes. The assertion audit and Web dependency update are part of that work.
What I would try next
For a first look at RC2, start with one retrieval question that matters to your application. Inspect what comes back, follow the source, and compare the behavior with reranking enabled and disabled where the task calls for it.
The FiQA results show progress in the published comparison. Quora makes the case for testing task assumptions. Apache 2.0 changes the adoption conversation, while the fixes improve specific parts of ingestion and integration. Together, they give us a more useful point from which to evaluate semantic storage for real agent workflows.
RC2 remains a prerelease. Authentication, authorization, tenant isolation, parts of memory extraction, and other readiness boundaries still need attention; consult the compatibility matrix. The published retrieval measurements do not establish production readiness or an independent reproduction of every result.
You can start the local Docker evaluation or inspect the retrieval evidence overview. If you try it, bring the question you are working on to GitHub Discussions, or open an issue with a concrete result we can investigate.
Release and evidence sources
- Cortrix v1.0.0-rc.2 prerelease and its linked change records.
- Paired Benchmark Evidence prerelease.
- July result summaries and August result summaries.
- RC2 upgrade notes and Benchmark license mapping.