BLOG: To a Higher Level of Open Data (Re-)Use in Ghent - Launching the New Open Data Portal

06 October 2020 - Published by SCORE comms
Ghent has a long lasting history with open data. What first started in 2010 with some rudimentary excel files on an open server has now evolved to a mature data portal with over 100 high-quality open data sets. Not only the quantity has increased, the quality has also improved over the years. Better metadata, higher update frequencies and regular revisions of the available data ensures that anyone who wants to use the Open Data is able to use the correct data in a machine-readable format of their choosing.

It is clear that in the past 10 years there has been a small ‘revolution’ within Ghent in the field of open data. The last step in this revolution was on the 5th of March 2020. On a rainy Thursday evening, in the depot where all the lost and stolen bikes are stored, the Data & Information unit of the city of Ghent launched its new Open Data Portal. 

It is certainly not an established habit of the city to launch new websites or portals on thursday evenings in a rather cold bicycle depot. The portal was launched at the end of a so-called datadive. By the term 'data dive' we mean an evening in which various partners and organizations come along to present their data to the interested residents of Ghent, in a rather informal way. Ghent has built up a tradition of holding such events a few weeks before its annual hackathon, Apps For Ghent. The theme of the hackathon this year was / is mobility, hence the extremely suitable location, the bicycle depot.

The Belgian railway-infrastructure company, Infrabel, presenting their open data sets.

But why was there a launch of a new portal in the first place? What was available before this portal and what were the reasons for the city to switch to a new portal?

Long story short, Ghent's old data portal could no longer cope with the current new forms of open data. We are evolving towards more and more real-time open data, API keys are less and less an exception and some datasets are getting bigger and bigger. The city’s old portal collected its data from 3 different sources:

  • Their Geoserver, in which all our GIS-data is located;

  • Virtuoso, our Linked Open Data database;

  • Last but not least, the Datatank. In this database, all the ‘other’ data was stored. Real-time API’s, CSV files, .xml files… 

The Virtuoso database and the Geoserver did not cause any, so to speak, ‘problems’. These systems can still handle the current forms of (open) data perfectly and therefore did not need to be replaced immediately. The biggest issues were situated in our Datatank database. With a maximum limit of 50 MB per dataset, the inability to offer multiple file formats and the absence of an extensive API module for the end users; it was clear for the city that the Datatank was in urgent need of replacement.

The landing page of the old Open Data Portal in 2019.

In September 2019, the search for a replacement of the Datatank started. Digipolis, the IT partner for the city of Ghent, and the Data & Information unit were very clear about what they wanted for their new portal: 

  • A more efficient system that can handle higher and heavier volumes of data;

  • A system that supports the most common (open) data formats;

  • A simpler backend with more sophisticated user right management;

  • A user-friendly landing page on which you could have a visual preview of the data.

After a thorough investigation of a few months, the French company OpenDataSoft came up with the best solution to meet the above mentioned demands of the city. 

The new Open Data Portal now supports almost every data format that is out there. It also gives the city the opportunity to perfectly integrate the above-mentioned Geoserver and the Virtuoso, resulting in better usability of these 2 sources.

In addition to supporting many data formats and the perfect integration of existing services, there are many new advantages and features:

  • A better filtering system for users to find the right dataset;

  • For each open dataset, a preview mode is available. This way, a user can get an idea what the data entails. For geographic data, this results in a handy map with the data displayed on it. The more analytical data will have standard graphs available to present the data more comprehensively;

  • As a user, you can combine multiple datasets in one graph and/or map. You can also save your map and share/embed the map on a different website;

  • A new API module for users to make proper API calls for every available dataset; 

 

  • The backend of the new portal is more user-friendly, has a better user-right-management and has an extensive analytical part, which gives the city better insights in how (much) the open data is used.

Thanks to the participation in the SCORE project, Ghent paid more attention to the reusability of the available open data when choosing this new portal. As mentioned earlier, the preview mode (maps and graphs) as well as the API module ensures that users can easily reuse the available data.

The landing page of the city’s new Open Data Portal - https://data.stad.gent/

The launch of the new data portal was just the starting point for Ghent. At the end of September, the city rolled out user registration on the new portal. Again, this is something that the city has learned from the SCORE project. Citizens who want to use the open data a little more actively (e.g. more API calls then a regular user) and want to link their self-made applications and put them in the spotlight, can register on the portal (without obligation) from the beginning of September 2020. This way, we want to promote the re-use of our open data. 

Furthermore, there are also plans to fully translate the metadata per dataset into English, set up an open data community and organize more online and offline open data events. The SCORE project has proven that a data community can be very valuable to think and work together about all kinds of problems and solutions.