Hong Kong Government Data API

O, our government finally built up the API for data provided in data.gov.hk. Let's see how we can make use of it.

I picked a simple example which could potentially be useful to many industries - the population. You will find the details of the API in:

https://data.gov.hk/en-data/dataset/hk-censtatd-census-2011-population-census-main-tables/resource/b375291d-088e-45a9-8dba-994d6f2ce61f#filter-api

However, the documentation is not really helpful because there is no sample url call but a small piece of code for "real" developers. So I look into the "Build a Query". Great! I can get the result. The url does not look good though.

After several minutes test and research, I converted the url into something readable. Therefore, I can see the query structure and it would be easier to perform future tasks.

Before:

https://api.data.gov.hk/v1/filter?q=%7B%22resource%22%3A%22http%3A%2F%2Fidds.census2011.gov.hk%2FMain_tables%2FBatch_04%2FD302.XLSX%22%2C%22section%22%3A2%2C%22format%22%3A%22json%22%7D

After:

https://api.data.gov.hk/v1/filter?q={"resource":"http://idds.census2011.gov.hk/Main_tables/Batch_04/D302.XLSX","section":2,"format":"json"}

Oops, if I put the converted url to a browser, I got a bad request. Probably there is decoding issues. I also put the original url, it directed me to download an unknown file which can be opened with notepad and containing the json format data. Anyway, I would rather say the API is not sophisticated enough since user cannot even query with a browser.

When it goes to Alteryx, things become simpler. Just put the url as an input for download tool (remember to check to box "Encode URL text"), then you will get the result from the server.

Afterwards, it's only some tricks to transform the data into rows and columns.

Overall, I found three points interesting:

1. The API is not typical. I have built several workflows to access the data through API with Alteryx, usually I will test the query with browser before moving on so I have an idea on what would I get. In this case, seems the query with browser is not available and even took time to do investigation. Apparently the API is not user-friendly.

2. The data available are not useful, diverse, not to mention "real time". If you check carefully the data available in different categories, first of all, most of them are not up-to-date and the update frequency is very low; second of all, the API basically allow the access to the ready-made excel but not a database server...why would I need to do extra thing to access an excel while I can download it directly.

3. One good thing is that the API does not apply authentication and query limitation so far (probably they do not think many access will appear!) which makes the whole process simpler. This policy should maintain until more useful and exciting improvement has been done and the server is almost overwhelmed.

Happy Alteryx!