For the first time, hackdays were held as part of the Swiss Statistics Meeting during the last weekend in Zurich. This pre-event called TWIST 2018 was not about dancing as one could assume, but all about data, statistics and the truth – it’s all in the name: Truth WIthin STatistics. In what follows I will give you a few insights into this fabulous event jointly organised by OpenData Stadt Zürich, Statistisches Amt Kanton Zürich, University of Zurich, Open Network Infrastructure Association and the R User-Group.
I joined all the other data and statistics enthusiasts who gathered this Saturday morning at the University of Zurich (my cherished alma mater) for a coffee before we were all taken on a data trip by the various providers (sponsors) of interesting data sets:
The challenges
All in all, 17 challenges were presented to us. Each data sponsor had a couple of minutes to present their challenge. The range of topics was wide and very interesting: it included financial information, data on real estate transactions, bicycle traffic counts, data on popular votes, Linked Data, air traffic data, passenger flows, and many more. Fittingly, during the event, the representatives from the municipal Statistics Office announced the launch of Linked Open Statistical Data (LOSD) for the City of Zurich.
Some of the presented data was already open data, some was made available specifically for TWIST 2018. The choice which topic one wanted to dive into was not an easy one! I am sure you will agree when you have a look at the complete list of challenges:
After the short presentations, the participants gathered at the spot assigned to their topic and we all had the chance to get to know our team members and the data sponsor. First ideas were exchanged and more detailed questions about the data could be asked.
Be excellent to each other!
By 10:30am (just as planned – living up to the cliché of Swiss punctuality), the teams spread out to find their favourite spot to work and started “hacking”. There were two rooms generously offered by the University with coffee and snacks in the hallway between them.
We formed a group of five (plus the issuer of the challenge, who ended up being a full team member during the whole weekend) to work with data about air traffic provided by the Zurich Airport. Our good-spirited team made up of a biologist, two biostatisticians, a political scientist and two geographers fully lived up to the mission statement of “being excellent to each other”.
We all enjoyed exploring the data and exchanging ideas about what could be done with it: We did some explorative data analysis (EDA), flight delay predictions using various statistical models, some geogiffery and a bit of analysis related to night flights (have a look at our project results here). We used mainly R for our work. The hacking went on until 9pm, including a quick status update of all the teams and a Pizza break. During the status update, teams reported in a couple of minutes on what they had achieved so far, what they planned to do next, where they were struggling and what help they were looking for from other participants.
The second day
12 hours later, we were all back for coffee and croissants and another day of hacking! After some more data analysis and visualisation, the groups were getting everything ready for the wrap-up session. At 4pm sharp, all 9 project teams presented their results: It was amazing to see what was achieved during such a short time period, despite various challenges resulting e.g. from incomplete data sets or difficulties in accessing the data at the start.
Have a look at everything that was accomplished during these two days on the slides of the TWIST 2018 final presentation. The results were subsequently also presented at the Swiss Statistics Meeting. During these two days, the teams worked with and learned about different technologies such as Python, R, D3, SPARQL, Leaflet, Shiny, and many more.
As far as I know, it is not clear yet if there will be another TWIST event, since the Swiss Statistics Meeting is always organised in a different city. The organising team of TWIST 2018 sure did a great job at making this event a success and inspiration for the participants, and I would not be surprised if they are already thinking about future events. As for me, I found the TWIST 2018 weekend very inspiring and motivating to continue working in this area.