Finding patterns within Fitbit's sleep data
On May 28, 2019, I left my old life behind, boarded a train, and started my world traveling adventure.
Being the data person I am, I thought that besides documenting my travels through beautiful and colorful pictures, I could also record it and relive my experiences through data.
In this article, I will share and analyze the data I gathered in my second week.
On last week's' recap of Wander Data, I presented data from Fitbit, Spotify, and an Android sensor app I wrote, and concluded the following points:
- I should get more active and move more because my Fitbit's' "very active minutes" numbers were quite low.
- Tyler, the Creator, was the only artist I listened to during my first days.
- The days were sunny, really bright, and the atmospheric pressure of the mountains of Austria is lower than that of my old place.
In this second update of Wander Data – written from the mountains of Tyrol, Austria – I'll once again, use and analyze the same kind of data (Fitbit, Spotify, and sensors) to see how it has evolved since last week. Furthermore, I'll also introduce my Fitbit sleeping data, and add more granularity to the Spotify numbers to know the time of the day in which I feel more musical.
As I said last week – I'll repeat some sentences for those who are reading for the first time – my favorite thing about owning an activity tracker such as the Fitbit is not the device itself, but the data it provides. Having such a small and easy-to-use tool stuck in your arm, collecting data 24/7, grants us the opportunity of performing longitudinal studies with the information it provides.
For the visualizations I'll be presenting in this section, I collected Fitbit data about my activity (active and non-active minutes), movement (number of steps, kilometers walked, and meters climbed), and sleeping patterns. To get the data, I used Fitbit's API, and the Python library "python-fitbit" to interface with it. Now, time to see the graphs.
The first plot I'll introduce shows my steps taken, and distance traveled – in log scale – from June 6, 2019, until June 15, 2019.
The blue line displays my steps taken, and it shows that only on two occasions, I walked over 10000 steps. One of these days was June 9, a Sunday, in which I hiked a 1,495 meters mountain named Gschwandtkopf; since that day was an outlier (I'm not hiking mountains every day), I'll keep referencing it throughout this article. The other day with over 10000 steps was June 12, and instead of walking another mountain, that day I was in the town of Seefeld playing Pokemon Go, buying yogurt, and trying to photograph the train station (typical traveler life). In general, the average steps per day was 7269, with a standard deviation of 2755.53.
Analogous to the steps metric, the graph also presents the distance metric (in km.) Sadly, it shows that unlike my first week, this time, I didn't manage to walk 10 km. on any single day. Instead, the maximum value is 7.63 km. while the average is 5.11 km. Does this mean that I became a lazy guy? Not necessarily. As we'll shortly see, my activity level was a bit higher than last time – still, I should walk more :). Additionally, this time's correlation between steps and distance is 0.97 , 0.01 less than the one reported in the first article. What this number suggests is that my walking stride length rarely varies (and it shouldn't, unless tomorrow I wake up being 10 cm. taller).
The second graph I want to share shows Fitbit's stat named elevation, a metric that describes the distance I have walked vertically. As previously stated, at the time of writing, I am in Tyrol, a mountainous region in the north of Austria, so it is unavoidable to find some literal up and downs.
The peak on June 9, refers to my hike up to the top of Gschwandtkopf, the mountain I previously mentioned. This mountain's altitude is 1,495 meters, but, the trek to conquer its summit starts on Seefeld, a town with an elevation of 1,180 meters, so by doing the math, we can see why Fitbit's registered altitude is 295 meters.
On June 13, the second highest point in the graph, I did the opposite of hiking a mountain. On this sunny day, I felt a bit adventurous and decided to go all the way down to some valley on a bike. And while the whole trip down the hill was fun and full of bliss, the same cannot be said about the way up, because according to Fitbit (and how sweaty I was), the way back home involved a 173 meters walk.
So far, I have talked about hiking, bikes, sweating, and such, terms that evoke feelings of exercise and activity. But what does the data say about this? Was I active? If you recall, in the previous article I wrote that during my first nine days, my good pal Fitbit registered a total of 6510 sedentary minutes, an average of 723.3 minutes (around 12 hours) per day, and 183 very active minutes. How have these numbers evolved? The next graph answers this.
Again, the "minutesSedentary" stat seems to dominate my days, which makes sense because I spend around 8 hours sleeping (or do I?). In total, I managed to amass 7091 sedentary minutes. But! This time it was over ten days, meaning that the average value is 709.1 minutes per day. Thus, I was less lazy than last week, :). On a more active note, I spent 1910 lightly active minutes (average of 191.5), 327 fairly active minutes (average of 32.7), and 218 very active minutes (average of 21.8). As a bonus, I'll present a boxplot with the same data, so we can appreciate which observation is an outlier.
Of course, the outlier had to be my most active day (the hiking day).
The last piece of information I want to present here is about my sleeping times. One might think that since I am living that hostel and vagabond life, I would sleep the whole day. Wrong! As you'll see, my sleeping patterns are not that out of the ordinary.
This graph above shows what Fitbit has recorded as my sleeping start time and end time. The red line indicates the start time, and it clearly states that I never went to bed before midnight; the average time was 1:15 am. On the other hand, the blue line says that – except for June 14 – I never wake up after 10.
How Fitbit calculates these times is a bit fuzzy for me – the only piece of documentation I found says that it determines the starting time when the person has not moved for around an hour. This rule implies that the data presented here do not show the actual time I spent sleeping, but luckily for me, Fitbit provides this magic number, and I'll present it on the following chart.
Atmospheric Pressure and Light Intensity values
Storytime: A few months ago, I had a problem. The issue was that I wanted to log the atmospheric pressure (for one of my experiments) of my current location using my mobile device (Android), and since I was aware that the phone (and yours too probably) had a barometer sensor, I went to Google Play Store looking for an app that does this. And sure, I found many apps that read sensor data and displays it on the screen. However, unfortunately, none of the apps offered the option to export the sensor data to an external file, while the app itself is running in the background. So, I wrote (and published) an app, and thanks to it, now I can continuously log and export the data from the various sensors my phone has.
In this section, I will present the atmospheric pressure data, measured in hectopascal (hPa) of the locations I have been, and the light intensity value, measured in lx or lux as reported by my app.
Now you might be asking: "why is this important?" or "Why are you calculating this?" Good question. First, I am a data person, so I like to collect data (pretty sure you already deducted this). Second, and most importantly, I believe that this data can tell a bit more about the current conditions of the places I have been, adding a bit of "spice" to the story I'm saying. Good? Cool. Now, back at it.
I'll open this segment with a scatterplot that visualizes the atmospheric pressure registered by the app.
What we see here is that the pressure of the places I have visited is relatively the same – an average value of 882.76 with a standard deviation of 26.67. But there's another way of interpreting this graph. A not-so-changing pressure implies that I have been at almost the same elevation all this time, which would be 1147.2 meters, if we convert the average pressure value to altitude. Nonetheless, there's a glaring exception on June 9. In case you already forgot, that was the day I went hiking up a mountain, and since the highest you are, the lowest the pressure is, then it is expected to have this value somehow different than the others.
The second kind of sensor data I have been calculating is light intensity, a metric that essentially says how bright an area is. What prompted me to use this obscure, and honestly, kind of non-important, piece of data is to get an idea of how dark my days are, and to try to deduct the time I spent outdoors (outside is brighter than indoors).
Getting the correct light sensor data from the phone is a bit tricky because most of the times the phone is resting inside my pocket (a very dark place), and as a result, most of the readings are zero, or values close to zero. Nonetheless, during those moments in which I am walking playing Pokemon Go, or trying to find my way out of the woods with Google Maps, the phone should be able to get a couple of readings, which in most cases is more than enough for creating the two charts I'll present here.
The plot above explains the average light intensity sensor readings per hour. Its first peak was registered on June 7, a really sunny day. Followed by it, there's another one on June 8. On that day there was a Pokemon Go event, and I spent a good couple of hours outside chasing the best of the best Pocket Monsters. Then, there are two noticeable-but-not-so-strong spots on June 9 (hiking day) and June 12 (Seefeld day), followed by another two entries on June 14, and June 15, two very sunny days ideally made for making a back-to-back BBQ marathon.
I'll be frank with you. I'm not a massive fan of this graph; it is just messy and somehow hard to read. Thus, I promise that for next week, I'll change the way I am presenting these values, and even better, I'll find a way to correlate it with another source of information, such as UV index.
For the last plot of this session, I grouped by hour the same light data and calculated the average of each group. The result is a curve that reflects the daytime and nighttime period of the region I currently am. For instance, you can see that the values start to increase after ten in the morning, and commence to wind down around 20. A curious detail the plot points out is that the average intensity seems to decrease at 17, followed by a sudden increase that might explain my tendency of going for an afternoon walk to catch some Pokemon.
To double check and corroborate my findings, here's today's sun graph of my actual location (taken from timeanddate.com)
While Spotify is not directly linked to anything "wander-ish" or "outdoor-ish," its data can tell a couple of things about my daily activities and pattern. To give a bit of background, for more than two years I have been heavily analyzing (and collecting through a service I wrote) my Spotify data and have discovered information such as my preferred style of music, favorite artists and even the times in which I don't listen to music. Therefore, I'm well aware of my musical patterns, and for this reason, I wanted to keep studying this data and see how it evolves during my journey.
I'll start this Spotify recap by showing the number of songs played by days and by the hour to understand what's my most musically active day and time of the day. But before that, I need to mention that my Spotify data gathering service was down for some days so unfortunately there are some missing days in the dataset.
Since the last recap, I have listened to 90 songs on Spotify over seven days, an average of 12.85 songs per day. Of these days, June 10 was the most active one while June 7 and 8 were the quiet days. What exactly was I listening? The next tables show presents my top songs and artists of this period.
|1||Tyler, The Creator||41||45.6|
|2||A BOY IS A GUN*||4||4.44|
|5||NEW MAGIC WAND||4||4.44|
|7||RUNNING OUT OF TIME||4||4.44|
|8||U Say (feat. Tyler, The Creator & Jay Prince)||4||4.44|
|9||ARE WE STILL FRIENDS?||3||3.33|
|10||GONE, GONE / THANK YOU||3||3.33|
Once again, Tyler, The Creator takes the first place with 41 (45.6% of the complete dataset) songs played, a significant decline if we compare it to last week's reported value of 103. Besides being in the mood for Tyler, these days I have been thinking about Thanos, and the fate of the Avengers, so I blasted a couple of songs – in particular, the one titled "Porch" – from the Infinity War and Endgame soundtrack, earning Alan Silvestri the second post of my prestigious top artists list. Lastly, we have Goldlink, and his song "U Say," featuring guess who? Of course, my boy Tyler.
Ever since I started traveling, I'm dedicating a couple of hours of my mornings and evenings to writing and coding. To acquire focus and concentration, I open up the windows to admire the mountains surrounding me and plug in my headphones to block any external noise (and to listen to Tyler). To confirm this assumption, I grouped all my played songs by the hour, plotted, and proved my hypothesis.
On this article and recap number two of my series Wander Data, I told and relived through Fitbit, Spotify and sensor data, some of my travel stories and events that occurred between June 6 and June 15, 2019. The analysis presented here, has reveal me that I've been more active than last the beginning of my adventure, that my sleeping patterns are better than I thought, and lastly, that even though Tyler's new album IGOR has been out for more than a month, I'm still listening to it as I did on the first day.
Thanks for reading and see you soon. If you have any questions, doubts, or just want to chat, leave me a comment, and I'll be happy to answer.
The code I used for this report is available at the wanderdata-scripts repo, available here: wanderdata-scripts.
The app Background Sensors is available at the following link: https://play.google.com/store/apps/details?id=com.juandes.sensors