Data exploration and access
Introduction
The WorldCereal Reference Data Module (RDM) offers multiple tools to explore, filter, visualize and download reference data and associated metadata, either through a web interface or API service.
Both services can be used without authentication, initially granting users access only to publicly available reference data.
Upon login with their Copernicus Data Space Ecosystem (CDSE) credentials, users can additionally upload, consult, filter, retrieve and share their own private datasets as well. Read more on data upload and publication.
Data use and citation
Each published dataset in the RDM is properly documented using a standardized metadata form, including details on the data owner, applicable data license, required citation and harmonization procedure.
When using reference data obtained through RDM in your own applications, please make sure to use the appropriate citation information as captured in the metadata of the associated datasets, as well as to include proper reference to the RDM application:
Karanam, S., Laso Bayas, J. C., Fritz, S., Boogaard, H., Pratihast, A. K., Degerickx, J., Butsko, C., Dries, J., & Van Tricht, K. (2024). WorldCereal Reference Data Module (RDM). International Institute for Applied Systems Analysis (IIASA). https://doi.org/10.60566/80p50-6z433
Explore data through our User Interface
The interactive RDM user interface for data exploration can be accessed here.
The map (image below) provides a quick overview of distribution of reference datasets. Pins are colored based on data privacy levels: - blue for public datasets (accessible to anyone) - yellow for fully private datasets (accessible to owner) - green for restricted datasets (accessible to owner and WorldCereal consortium)
The filter options in the upper right corner allow you to quickly filter reference datasets based on observation year and land cover/crop type labels. Clicking an individual pin shows the link to the dataset, along with the total number of observations.
Upon clicking the link to a dataset, you end up on a dedicated page showing all metadata and direct download links to the different components of the dataset, including: - full harmonized dataset - subsampled harmonized dataset - metadata excel file - explanation of harmonization procedure - mapping of original crop labels to WorldCereal legend
Alternatively, you can manually browse through the list of public collections or (once authenticated) your private collections.
Click the following image to watch our full video tutorial on data exploration:
Retrieve data through API
The RDM provides REST APIs to access data.
The WorldCereal processing module makes use of these REST APIs to request reference data prior to training crop classification algorithms.
RDM demo notebook
In this Jupyter notebook we have implemented some user-friendly functionalities in Python allowing anyone to find, query, inspect and download entire datasets, metadata and individual observations. The notebook teaches you the practical skills needed to interact with reference data, representing the first step towards building your own crop type models.
The notebook includes instructions on how to run the notebook without the need for installing anything to your local system.
Other resources
In case you would like to learn more about the underlying API functionalities, here are some additional resources for you to consider:
- Jupyter notebook on how to access public datasets
More detailed insights into how API requests are constructed to access public datasets.
- Jupyter notebook on how to access private datasets
To access user uploaded private datasets through APIs, credentials are required. These credentials are the same as those that were used during upload of the datasets. The following Python notebook demonstrates how to get user private datasets:
- Swagger API Documentation
Full documentation on the different API requests can be found on our dedicated Swagger page: