Self-Assessment Framework for Earth Observation Platforms from User Experience

This work proposes a framework for Earth Observation (EO) cloud platforms to carry out a self-assessment from a user experience perspective. The study outlines the methodological approach used to develop the framework, including the classification of existing EO platforms through a literature review of EU-funded projects. Additionally, a survey on user experience was conducted to further inform the design of the framework. We consider as target users the developers of policy-relevant use cases. The final phase of the methodology identifies user requirements across ten dimensions within the EO value chain and suggests a range of indicators to facilitate self-assessment. The paper emphasises the need to address the identified challenges and limitations to improve the usability and user satisfaction of existing EO cloud platforms. It also highlights the importance of principles such as FAIR (findability, accessibility, interoperability, and reusability) and TRUST (transparency, responsibility, user focus, sustainability and technology), open source components, and Open Geospatial Consortium (OGC) standards in shaping the future of EO data platforms and infrastructures.


Introduction
The European Strategy for Data (European Commission, 2020) aims at creating a single market for data sharing and exchange to increase the European Union's (EU) global competitiveness and data sovereignty.Furthermore,it places a strong focus on prioritizing people's needs in technological advancement and upholding EU values and rights.The EU has significantly invested in making data accessible.This is exemplified by initiatives such as the Copernicus Programme, the Group on Earth Observation (GEO) intergovernmental partnership, and the Horizon 2020 and Horizon Europe funding programmes.Within the purview of these programs, several Earth Observation (EO) cloud platforms have been developed, offering access to data, tools, and services catering to a diverse range of users, including assistance for policymakers in creating evidence-based, datadriven policies.To amplify the coordination of EO activities and data exploitation across Europe, the EU has established Euro-GEO, a regional contribution to the GEO initiative (Kona et al., 2023).
Typically, these EO cloud platforms are the expression of different communities of researchers, practitioners and policy makers with varying sizes and scopes, sometimes focusing on specialised areas, which may not always align with the broader demands of users, as opposed to more mainstream platforms with wider user uptake.The proliferation of infrastructures, while indicative of a commitment to data accessibility, does not automatically translate into user satisfaction.Indeed, users may find it challenging to navigate the extensive array of EO services and platforms to find those that meet their specific demands.This abundance can lead to fragmentation, which is partly due to the diversity and local characteristics of data and services within the EO research and decision-making sphere.This situation can be compounded by a top-down approach to the design of infrastructures.Additionally, the multitude of services and initiatives might not have established a strong user base, raising questions about their long-term sustainability (Di Leo et al., 2023).Digital platforms cater to a diverse user base that in-cludes scientists, policy makers, and analysts, each with their unique needs and expectations.Their workflows can vary significantly, encompassing a range of processes from the initial handling of raw data to the generation of actionable insights.These differences are reflected in the distinct steps they usually undertake to produce products and services tailored to their specific objectives.Our key interest is in use cases with a prominent policy priority, which typically require heterogeneous data to be shared and used at scale (Kilsedar et al., 2023).In this study, we identify needs in terms of products, services, and functionalities.We aim to highlight the most common requirements of use cases, commonalities, needs and challenges faced in the implementation phase, used infrastructures and their scope, datasets, intended user categories, and other relevant aspects.It is crucial to supplement the analysis of user needs with an understanding of emerging technological trends relevant to the EO domain, such as the widespread incorporation of Artificial Intelligence (AI) technologies, data-driven algorithms, IoT data streams, as well as edge and fog computing (Granell et al., 2022).These trends are reshaping user requirements due to their escalating popularity fueled by growing demand.Key technologies, together with their most relevant hardware (HD) and software (SW) solutions, are presented in Table 1 because of their relevance to the scope of the study, having the potential to drive users into choosing one service provider over the other because of the need for given setups.
Focusing on the development phase from a user viewpoint, the overall process is not free from inefficiencies and bottlenecks, also due to the typical complexity of solutions and fragmented landscape of tools and infrastructures that EO application developers use and orchestrate.One of the main objectives of EO platforms is to support the streamlining of this end-to-end process from raw data to insights for decision-makers.To accomplish this, it is necessary to understand where the main bottlenecks and frictions currently are.This is in itself a challenging task because of the many options available to users (developers) and the tailoring taking place according to specific use cases and needs.Along with this, we aim to understand whether the  1).We started our research by classifying existing EO infrastructures, identifying good practices, and highlighting technological enablers to identify and leverage the building blocks that may be needed to improve the usability of EO platforms Di Leo et al. (2023).In this follow-up study, we seek to provide a usercentred perspective, aiming at identifying limitations in the current offer of EO cloud platforms by conducting a research study on user experience.Ultimately, we aim to propose good practices to improve both the platform design and functionalities by considering the user viewpoint.
After this introduction, the remainder of the paper is structured as follows.Section 2 presents our methodology, which leverages the inventory and classification of existing EO platforms and derives additional insights from both a review of a prominent project and a survye on user experience.This is followed by Section 3, where we present the dimensions and indicators for a self-assessment and self-evaluation of EO platforms.Section 4 concludes the paper by reflecting on the main outcomes and anticipating future research.

User experience analysis: methodology
In this section, we present the methodology as composed of the following steps: 1. Inventory and classification of existing platforms; 2. Systematic review of the e-shape project documentation; 3. Survey on user experience in use-cases.

Inventory and Classification of existing platforms and their characteristics
The state of EO cloud platforms and initiatives was mapped in our previous research (Di Leo et al., 2023).To provide insights from it, we developed a classification scheme for digital EO infrastructures and initiatives according to the services they offer.
The classification scheme entails the following categories: 1.Data providers: they make EO datasets available within infrastructures; 2. Cloud-based geoprocessing platforms: computational capacity offered by data providers according to the paradigm "bring the user to the data"; 3. Brokers and catalogues: they offer discovery services by harvesting data from existing catalogues; 4. Thematic Hubs and Research Infrastructures: they incorporate EO data relevant to specific thematic domains, such as agriculture and atmosphere; 5. Data cubes: they implement a multidimensional array structure, on which one can load several bands and perform operations; 6. Virtual infrastructures: they place additional layers on top of existing platforms to facilitate data access and increase the discoverability of and interoperability among platforms; 7. Initiatives and programmes: EO-related, publicly funded programmes.

Systematic review of e-shape documentation
We analysed the use cases and pilots from the e-shape project (Ranchin et al., 2021).e-shape is an EU Horizon 2020-funded project (H2020 GA 820852) entailing 37 pilots, grouped into 7 fields of application: Agriculture, Health, Energy, Ecosystem, Water, Disaster Management, Climate and Climate Change.A questionnaire, presented in Section 2.3, was prepared to collect details about the use cases.Furthermore, we built a vast corpus of information to be queried, aiming to identify the digital platforms used by the pilots.Such corpus is composed of: the information we collected from the ID cards of the e-shape pilots1 ; reports by e-shape2 ; scientific papers published by the e-shape consortium; documents, press releases, etc. available on the Internet.Using text mining techniques, we located mentions of platforms.Then we identified -by assumption -the platforms and the services of interest they offered, and finally we interviewed members of the e-shape consortium to verify our assumptions.The core interest was in discovering which EO digital platforms were used by the pilots, the services used from such platforms and, if the pilots published any software or services, how these were deployed, with which license and on which platform.

Survey on user experience in external use cases
We also designed a survey3 for the users of the aforementioned platforms.The questionnaire is organised into 6 sections: 1. Sector of application, type of use case, and involved partners; 2. Identification of clients of the product/service; 3. Identification of the nature of source datasets and technological challenges associated with their discovery, access and integration; 4. Identification of the nature of product or service delivered and technological challenges associated with the deployment; 5. Technologies and challenges associated with data processing; 6. Evaluation of the platforms currently used and reasons that drove the choice.
We asked the use case leaders of some EU flagship initiatives -including, in addition to e-shape, Open-Earth-Monitor4 and GEOSS Platform Plus5 -to answer it.The 26 responses to the survey highlighted use cases in a wide range of sectors, including agriculture, energy, health, ecosystem, disaster management, water, climate and climate change, forestry and oceans.Target clients of the use cases ranged from business owners to analysts, developers, data scientists and policymakers, as well as citizens.

User needs: dimensions and indicators
In this section, we propose a set of dimensions and indicators conceived for self-assessment and self-evaluation of EO platforms.Our aim is to determine, through qualitative and quantitative means, whether the user needs identified from the review of e-shape documentation and the user experience survey are met.Target users are developers of policy-relevant use cases, who use EU-funded EO platforms.The main concerns we identified are summarised below: 1.The difficulty to discover the services offered by platforms; 2. The reduced accessibility of data and services, timeliness and coverage of data provision and quality; 3. The limited transparency of the price; 4. The difficulty to integrate heterogeneous datasets and tools from different providers; 5.The limited quality of learning material and documentation, and frequency of their updates; 6.The lack of effectiveness of support services such as helpdesks and forums; 7. The limited possibility of exchanging code, good practices, and support with other users, and the liveliness of the communities around the platforms; 8.The lack of customisation of tools and services; 9.The lack of strategies for the sustainability of platforms after the end of the project; 10.The lack of facilities for storage and for functionalities such as ML, DL, parallel computing, etc.Out of such concerns, we identified a set of user requirements grouped into ten dimensions: 1  2, along with a short description.

Discussion and conclusions
The European Strategy for Data aims to create a single market for data sharing and exchange to enhance the EU's global competitiveness in all societal sectors and in full respect of EU values and rights (European Commission, 2020).As a result of several initiatives and funding programmes aimed to enhance the availability and accessibility of EO data, several EO cloud platforms have emerged which provide access to data, tools, and services for a wide range of users, including support for evidence-based policymaking.
The current trends in the landscape of digital platforms are influenced by the need to handle and interpret the massive amounts of data being produced, particularly within the domains of EO and environmental monitoring.Notable trends include the increasing utilisation of AI and ML to analyse huge datasets, as well as automated processes offering predictive insights that enhance the process of decision-making.Cloud computing and storage solutions are increasingly used, delivering scalable storage and computational capabilities, which allow users to access and process extensive amounts of datasets without the need for a significant infrastructure of their own.However, for such platforms to grow, we believe that it is crucial to jointly consider technological developments and user needs.Along these lines, we stress the importance of considering and fulfilling the indicators that emerged from the user requirements analysis (see Table 2).Digital platforms that propose services and future developments in agreement with user needs and expectations, ranging from data accessibility and discoverability to analytical capabilities, are more likely to be adopted by their target audience, leading to broader usage and greater impact.Platforms and users benefit from features that promote collaboration, especially when different users such as scientists, policymakers and analysts are considered as target audiences.Conversely, platforms failing to keep pace with user requirements may become obsolete in a rapidly evolving digital landscape.Finally, we also stress environmental requirements, translating into the need for strongly decreasing emissions, an aspect that is rapidly gaining importance for users (Rosário and Dias, 2023).
As an exemplary requirement among others, and considering the diverse and fragmented world of digital platforms, systems failing to meet the interoperability challenge may risk a decline in their use.In addition to interoperability, discoverability, accessibility, usability, preservation, and curation of shared data resources are all mentioned in the GEO Data Management Principles Implementation Guidelines, a document recently compiled by the GEO Data Working Group (2022).These guidelines are also grounded on open data principles, highlighting their complementarity with the FAIR (Wilkinson et al., 2016) and the TRUST Principles (Lin et al., 2020).The guidelines apply to all EO data products, including remote sensing and in-situ data, and are intended to cover raw data, high-level products, and ondemand data services.It is crucial to note that these principles focus on the data rather than on specific platforms.The emphasis is on the principles that should govern the management and sharing of data, and less on the specific tools or systems used to store or access those data.We complement such vision with our self-assessment framework for EO platforms, thus considering also models, services, and other aspects.
Another key ingredient for platforms to succeed is federation, i.e. adhering to standards within groups (federations) to facilitate the achievement of common goals.The evolution towards such paradigm has already started in the GEO world (Baek, 2022;Santoro et al., 2023;Mazzetti and Nativi, 2024), and we expect that the advent of data spaces (Farrell et al., 2023) will further boost it, with clear benefits for all types of users.
A specific aspect we would like to highlight is the role that privacy-enhancing technologies (PETs) may play in the presence of valuable data that are typically hard to access because of privacy or confidentiality constraints.The joint use of a data space approach -in a nutshell, an ecosystem of data models, datasets, ontologies, data sharing contracts, and specialised management services, together with soft competencies around it (i.e.governance, social interactions, business processes)together with PETs holds the potential to increase the possibil-ity for users to access such data while preserving data features that data owners/data producers do not want to disclose.The same principle may apply to the case of sensitive data covered by the General Data Protection Regulation (GDPR) in the EU European Parliament and Council (2016).
As a valuable reference when it comes to EO platforms and federations, we point the reader to Santoro et al. (2023) and Mazzetti and Nativi (2024).The authors define the Geoscience Digital Ecosystem as a 'system of systems that applies the digital ecosystem paradigm to model the complex collaborative and competitive social domain dealing with the generation of knowledge on the Earth plane'.The proposed Virtual Earth Cloud infrastructure, composed of the Virtual Earth Laboratory (VLab) among other elements, allows a holistic view of diverse and autonomous entities, in line with the conceptual we push forward in this work, aiming at a single entry point for EO cloud platforms towards a one-stop shop approach.
To conclude, we outline the future steps we foresee in our work, starting from the development and deployment of exemplary and policy-relevant use cases.The aim will be to show how the proposed dimensions can be used, identifying the most relevant ones on a case-by-case basis and showing how the framework can be adapted to different scenarios.We remark that the framework presented in this work is designed for a self-evaluation by platforms, since it is unlikely for users -the role we are currently playing -to access to the information required for a complete assessment.Therefore, only a few dimensions will probably apply to each use case we plan to develop.The scope will be to demonstrate the importance that the indicators have in the development phase, rather than to carry out any kind of assessment.Presently, we would like to collect feedback from both digital platforms and users about the proposed dimensions to make sure they capture what is relevant for both, and subsequently identify platforms willing to act as pilots to test the self-assessment procedure.The proposal we make in this work aims to improve usability and user satisfaction, with the ultimate goal to foster discussions towards the definition and subsequent refinements of a self-assessment (as opposed to an assessment) framework for EO cloud platforms.

Disclaimer
The views expressed are purely those of the authors and may not in any circumstances be regarded as stating an official position of the European Commission.

Figure 1 .
Figure 1.Stages of the development lifecycle of an EO use case for policy-informed decision making.

Table 1 .
Exemplary technologies and most relevant hardware/software solutions whose availability may influence the choice of EO platforms by users.

Table 2 .
Dimensions and indicators to support the evaluation of digital platforms from a user viewpoint.