Minghan Yu, Youran Sun, Chugang Yi, Yixin Wen, Haizhao Yang
An agentic search system is proposed to find relevant datasets and tools from NASA's Earth Observation Knowledge Graph using natural language queries, along with a new benchmark and performance validation.
Large data centers like NASA hold thousands of geoscience datasets and tools, making it difficult even for domain experts to find the right ones. Existing keyword-based search struggles to fully understand the meaning of natural language research queries.
The authors construct NASA-EO-Bench, a benchmark of 47k query-dataset pairs derived from the NASA Earth Observation Knowledge Graph (NASA EO-KG). They train a neural scorer that outperforms cosine similarity and BM25 baselines, and further improve performance by fusing its scores with BM25. Finally, a zero-shot agentic reranking stage using LLM reasoning is added to refine search results without additional training.
The proposed supervised pipeline combined with BM25 score fusion improves Recall@10 and MRR by over 5x. The zero-shot agentic reranking further boosts MRR by 28%, demonstrating that LLM reasoning is complementary to supervised retrieval. The release of the NASA-EO-Bench benchmark contributes to advancing geoscience data discovery.