USFGeoAI: Applying LLMs and Foundation Models to Assist in Image Retrieval

Abstract. Geospatial analysis is often hampered by two major obstacles: a scarcity of high-quality, annotated satellite imagery, and the complexity of interacting with machine learning tools for image analysis. To address this, we introduce USFGeoAI, a proof-of-concept system that allows users to query, retrieve, and analyze high-resolution imagery using natural language interaction and direct processing of images. The system incorporates IBM-NASA’s Prithvi Foundation Model for supervised detection of environmental features and the Clay Foundation Model for unsupervised similarity search when detectors are unavailable. An interactive interface allows users to search for features (such as swimming pools, vegetation changes, and burn scars), apply detectors to TIFF images, and explore new regions for model training.