The article suggests Common Crawl as a replacement which probably doesn’t make sense on its own due to low update frequency (monthly) and somewhat limited crawl scope.
(Looks like the sentence following the suggestion addresses this somewhat.)
Their API doesn't provide the mixed Kagi metasearch results though. Only results from their smallweb Teclis index, Curlie, and I think Marginalia results are included. Probably also other sources I'm not aware of.
Marginalia has a free-ish API for non-commercial purposes if you go direct:
An LLM does not continuously index the Internet and is therefore stale the second it starts training. You need to use something like RAG. And the R in RAG needs an index.
This was announced in January.
https://programmablesearchengine.googleblog.com/2026/01/upda...
The article suggests Common Crawl as a replacement which probably doesn’t make sense on its own due to low update frequency (monthly) and somewhat limited crawl scope.
(Looks like the sentence following the suggestion addresses this somewhat.)
Is this good for Kagi?
Does Kagi have a free developer api?
Not free, 2.5c per search:
https://help.kagi.com/kagi/api/search.html
Their API doesn't provide the mixed Kagi metasearch results though. Only results from their smallweb Teclis index, Curlie, and I think Marginalia results are included. Probably also other sources I'm not aware of.
Marginalia has a free-ish API for non-commercial purposes if you go direct:
https://about.marginalia-search.com/article/api/
Good. Duck google and use other indexing.
ps. Modern AI is in reality talking web search engine imo.
What other index is there that is both free, reliable, and accurate at web scale?
An LLM does not continuously index the Internet and is therefore stale the second it starts training. You need to use something like RAG. And the R in RAG needs an index.
I just see another small competitor becoming big by taking over