EARTH DATA COLLECTOR

A repository from the Earth Data Discovery Project

PROJECT

This repository is part of a project created by the Earth Data Discovery Team for the 2020 Space Apps Challenge.

The Challenge

Our team decided to address the Data Discovery for Earth Science Challenge":

This challenge will ask you to devise a tool or technique to guide users to relevant datasets to study specific events.

The goal of our project is to make Earth Science Data easily accessible for everyone. We think that finding a database for any project should be a simple, fast and straight forward process, that doesn't require lots of expertise or previous knowledge. In order to reach this goal, we provide a search and visualization interface, providing information about publicly available databases from the main Space Agencies over the world.

Our main goals:

• Provide an instant, relevant and typo tolerant search engine for Earth Science datasets.
• Provide an intuitive, simple yet rich interface for finding the dataset that fit your needs.

ABOUT THIS REPOSITORY

The goal of this repository is to fetch data from the CMR Earth Data and index it into a MeiliSearch search Engine.

This is a necesary step to create an accessible and open interface for finding Earth Science datasets.

HOW TO USE IT

Preparing your environment

Clone the repository and install dependencies. You need python 3 and pip

git clone https://github.com/earthdatadiscovery/earth-data-collector.git
cd earth-data-collector
pip install -r requirements.txt

RUN A MEILISEARCH INSTANCE

Here we use a docker image, but there are other ways to install here

docker run -it -p 7700:7700 --name MeiliEarthData getmeili/meilisearch:latest ./meilisearch

ADD YOUR MEILISEARCH SETTINGS

You need to set up your MeiliSearch URL (and eventually the API Key if you want to use one) in services/meilisearch/meilisearch.py

RUN IT! GO GO GO!

python collector.py