12 Big Data Predictions for 2014

Big Data had been one of the buzzwords of 2013. Thor Olavsrud from CIO.com looks at 12 ways the technology and the market will evolve in the coming year. My favorite one is: Analytics based on the R programming language, geared specifically for data scientists performing statistical analysis, will go mainstream in 2014, according to Alteryx, edging … Continue Reading »

Discover Your Neighborhood with Census Explorer

On http://blogs.census.gov/ Michael Ratcliffe describes the new Census Explorer, an interactive mapping tool for the governmental census data. Our customers often want to explore neighborhood-level statistics and see how their communities have changed over time. Our new Census Explorer interactive mapping tool makes this easier than ever. It provides statistics on a variety of topics, such as percent … Continue Reading »

London in Motion

This animation, produced by Jay Gordon, shows the daily flows of London’s commuters. It combines the 16 million or so daily transactions made with London’s Oyster cards with vehicle-location data from the city’s 8,500 buses to infer journeys of approximately 3.1 million Oyster users. The work behind this formed part of Jay’s masters thesis.

Michael Batty about Big Data and Smart Cities

This video is about the use of (big) data in urban systems. Batty talks about how they analyze the public transportation system in London. There are two things that are quite interesting. The first is his definition of big data: “big data is everything that doesn’t go into an Excel spreadsheet”. The other thing, that … Continue Reading »

Civic Tech Landscape

Nathan Yau pointed out that Fathom Information Design, in collaboration with the Knight Foundation and Quid, visualized the growth of the civic tech landscape. They based their findings on an analysis of terms used to describe civic tech organizations and investments in them. A new report released today by Knight titled “The Emergence of Civic … Continue Reading »

Some fun: ASCII fluid simulator

Here’s a neat fluid simulation program that runs entirely in ASCII: If you want to try it out yourself you can download the souce code for the program here. Just feed it a text file with an ASCII art drawing of the scenario you want to simulate (more details here). What you may have missed is that in … Continue Reading »

Data Visualization of Climate Change: the State of Science

This video is a great data visualization of the state of science in global climate change. It was funded by the UN Foundation and produced by the International Geosphere-Biosphere Programme and Globaia. It combines a range of impressive data visualizations depicting the human impacts on our environment with a clear commentary. As almost all scientists … Continue Reading »

Bokeh, a Python library for interactive visualization

Bokeh, a Python library by Continuum Analytics, helps you visualize your data on the web. Bokeh is a Python interactive visualization library for large datasets that natively uses the latest web technologies. Its goal is to provide elegant, concise construction of novel graphics in the style of Protovis/D3, while delivering high-performance interactivity over large data to … Continue Reading »

Databases for lazy people in Python

Friedrich Lindenberg and Gregor Aisch recently released a Python library called “dataset” that enables developers to use a SQL database like a JSON file or NoSQL store. Although managing data in a relational database has plenty of benefits, they’re rarely used in day-to-day work with small to medium scale datasets. Why is that? Why do we see … Continue Reading »

How to share data with a data scientist

Jeff Leek posted a very interesting guide for anyone who needs to share data with a statistician. The goals of this guide are to provide some instruction on the best way to share data to avoid the most common pitfalls and sources of delay in the transition from data collection to data analysis. The Leek group works … Continue Reading »