The movement and sleep patterns of a city tell an amazing story about its culture and people. How active is a city? When do they go to bed on average, and how much do they sleep? How stark are the differences between weekends and weekdays? What events brought people together and significantly impacted the health of a city? Each pattern forms a distinct “thumbprint” for the city, the unique way its citizens live their lives. To an untrained eye, these images may just look like the abstract brush strokes of a Rothko painting. To a data scientist, however, these graphics richly detail the routines — and occasional abnormalities — of city denizens. Some quick stats:
- Least Sleep: Tokyo, Japan – 5hr 44min
- Most Sleep: Melbourne, Australia – 6hr 58min
- Earliest to Bed: Brisbane, Australia – 10:57pm
- Latest to Bed: Moscow, Russia – 12:46am
- Earliest to Rise: Brisbane, Australia – 6:29am
- Latest to Rise: Moscow, Russia – 8:08am
- Most Steps: Stockholm, Sweden – 8,876 steps
- Least Steps: Sao Paulo, Brazil – 6,254 steps
But that only tells part of the story. Let’s explore some examples more deeply:
City Steps and Sleep (Interactive)
Sleep on the Average Day (Interactive)
Here are the sleep patterns of UP wearers for seven cities from Monday, March 31, 2014, a typical night. New Yorkers work hard and play hard, and they’re the first to bed and among the first to rise. Users in Tokyo are among some of the last to go to bed and the first to wake up, since they only average 5 hours and 46 min of night sleep. Dubai has the most leisurely sleep schedule, with 10% of users still asleep by 11am. In Beijing, you can see workers taking their afternoon workplace naps. We can also see people in Madrid taking their afternoon sleep (although it’s much more common on weekends, with greater than 10% of UP wearers logging a siesta). Only a maximum of 95% of a city is asleep at any given time, since the early risers are awake before the last to sleep are in bed.
What city do you love? Let me know at @brianwilt.
Statistical Significance. All international cities include a sample of > 5000 users. All US cities include samples of tens of thousands of UP wearers.
Sleep. UP by Jawbone sleep tracking records not only bedtime and waketime, but also time to fall asleep, awakenings, and sleep quality. The average hours of sleep cited above do not include time awake in bed. Thus, even though Moscow average bedtime and waketime are 12:46am and 8:08am respectively, a difference of 7hr 22min, they only average 6hr 42min of sleep a night.
Tools. The thumbprint visualizer is powered by d3.js. The storytelling elements use qtip2 tooltips. For the graph, I used Highcharts.js. I wrote a short function to inject storytelling tooltips into Highcharts.js available here. My colleague Emi Nomura led development of the geo pipeline for aggregating data by city and my colleague Eugene Mandel helped backfill hourly steps data using EMR.
Optimization. Originally, the data files took a long time to download, making the visualization slow. Each .csv line had a lot of data, some of which was redundant —
Atlanta, GA, United States|2013-06-01|12pm|0.513942|121.4819 — 61 bytes per line with 365 days x 24 hours x 15 min intervals = 35,040 lines = roughly 2 MB per city. To save space we encoded only the last two columns (representing percentage of people asleep and average number of steps by hour) in hex, requiring just 4 bytes, representing the red and blue colors in hex. So the example above simply becomes
Data Visualization. Our VP of data, Monica Rogati, says her favorite data science algorithm is division. Simple calculations like sums and percentages beat complex analysis with enough data, particularly in terms of ROI. My favorite data science algorithm is the Excel conditional formatting button (“Color Scales” for the win!). It augments one of the most powerful pattern detection tools in our arsenal: human eyesight. A dear colleague from my time as a neuroscientist, James Fitzgerald, researches the visual system. He studies the algorithms hardwired into the retina that process visual signals before they reach the brain. In short, our eyes are programmed to directly detect spatial and temporal patterns and anomalies, before our brain gets to “think” about it. Ramón y Cajal shared a Nobel prize for developing tools to directly visualize the brain under the microscope. His observation that the brain is composed of discrete cells (neurons) fundamentally changed neuroscience. Data visualization is powerful means of interrogating and understanding data. It’s hardwired into our eyes and brain and has a storied history of discovery.