SCORE

The APIAPI component

This open source data management component provides cities with a user friendly, visual interface to explore, select, merge, combine and remodel data from different data sources (API's) to a single expected data model.

The challenge 

Most smart city frontend applications such as SCORE's IOT Registry expect the data that they consume to come in a specific structure or data model. Sometimes this data model is tailored, sometimes it's following a standard, sometimes it follows agreements in line with the city's digital architecture. In practice, when replicating a tool from city A to city B, in most of the cases we experienced that the data model expected by a solution does not match perfectly with the way the data is presented to it.

Even when open standards are followed, these standards apply mainly to the way in which the data is exchanged on a technical level. In the actual fields and values that form a data stream, standards could still be interpreted differently. Nuanced differences pop up, causing compatibility problems. Without access to expert API developers who can solve these issues on a case by case basis, such issues can make it hard for a city's innovation team to reuse and combine different data sources in their applications.

We have seen this challenge especially present in the case of data streams (API's) that are provided by different IT service suppliers, IoT companies, other service providers ...

The solution

The APIAPI component addresses this problem. The idea is that rather than a standalone application it is a generic component that can be included as part of a new solution or prototype. The idea is that in a no-code fashion, a data operator can manage the API's where the solution gets its data from. In the tool, the data operator or a person with access to data sources can configure different links to different kinds of incoming data streams (eg. public open API's, API's that require authentication, static files that are frequently updated, historic data in static files ...).

After setting up a Collection of these incoming data streams (through an intuitive interface, using a couple of simple, built in tools) the data can then be:

  • Explored → eg. Which specific data fields and values is a sensor or any data stream exactly providing us? What is the meaning of each field? What is behind the values that the data stream produces, how are they captured or calculated?

  • Selected → Which fields from the data stream do we need in our specific application? In many cases not all data that an API provides is useful for the purpose that we want to use it. To prevent information overload effects on both people and machines, it's important to limit the amount of data that is sent through to the minimum needed fields;

  • Combined → Multiple similar data sources can be combined in a merged collection, forming a new and merged data stream according to specific needs. Being able to merge data sources into a single, harmonized data collection is a crucial ability for more data driven public service;

  • Remodeled → Which named fields and other specific data structures does our application expect? How should the combined incoming data values be mapped, combined, reshaped or even recalculated in the required data field model.

The APIAPI application can thus be seen as an API toolkit that helps cities to provide structured and configurable data collections to other applications and prototypes. Working with API data can become complex, especially if one needs to combine and remodel data. Traditionally it requires specific programming skills and expert tools. Therefore we created this tool with user friendly workflows and minimal, "step by step" visual interfaces in mind.

Example use

A good use case to explain the role of the APIAPI solution is bike sharing services in a city. Typically the city government agrees with multiple suppliers to start providing shared bikes and services in the city. From the data driven perspective of the smart city, these companies are also asked to provide their data as open or shared API's to either the general public or the city administration. However, on the technical side, cities are not always able to enforce a single required data model to the individual data sharing systems per bike supplier. As different companies each with their unique history, the bike sharing companies' backend platforms have been developed separately from each other. The service providers can also have small differences in their business model. 

Consequently, the used data model and terminology is fragmented: it may be the same for some of the data, somewhat similar for other data and even completely different in some cases. More realistically, suppliers would be asked to be as compliant/compatible as possible to the data model the city requires.

In this case the APIAPI can function as the "missing link" to help the city remodel and harmonize the variant data structures from these different sources to the single "bike sharing" data model they have in mind for a specific tool.

Our current status

The first version of the APIAPI component was developed by District09 Ghent. You can see a screenshot here.

 

Ambitions after SCORE

The long-term ambition of our work on the APIAPI is that any European city can use it. That is why we are co-developing this solution in an open source repository, where it will remain available to any city after the project ends.

Join us
For more information about the APIAPI please contact these SCORE partners:

The application's source code is available on Github, so its development can be followed closely and improvements can be suggested.

SCORE is a collaboration oriented, open source development project. You're most welcome to co-develop this application or follow its development more closely, if so don't hesitate to contribute to its Github repository.

undefined