OK, bear with my meandering rambles today. It’s something I’ve had in my head for some time, trying to wrestle some sort of sensible conclusion about our beloved fitness devices.

I love numbers. I love data.

It’s an opportunity to understand more about what is going on beyond the hand waving and pointing.

It’s not just running, my first career out of university involved a lot of figures. And as my degree was Chemistry, there were a lot of numbers in that too.

The devices we carry around with us all the time all collect data. Whether this be when we’re exercising, or even when we’re sleeping, we have devices that record all this information.

This isn’t a post about data harvesting and sale, that’s another issue that has been well covered by others. Instead I’m looking at the personal value of this data.

Fitness devices

Fitness watches have rapidly gained popularity in recent years, offering users the ability to track various health metrics and monitor their daily activities effortlessly. With companies like Garmin, Polar, Fitbit and Apple leading the charge, these devices have become a staple in the world of health and fitness enthusiasts. However, while they appear to be the perfect companion for an active lifestyle, it’s essential to understand the limitations that come with relying solely on the data derived from these wearable gadgets.

I use my Garmin for all my “fitness” data. So it’s recording every run and every ride; routes, paces, effort, heart rate, balance between feet, power, VO2 Max. And a whole slew of other things that aren’t jumping out to me at the moment. Mostly because they’re metrics I don’t know what to do with.

WTF?

This is the problem that is across all platforms of fitness tech. Our watches give us all this data and the vast majority of it is “interesting” but pretty well useless to most of us. And it is only a number which, in isolation, may not be significant of anything. And there’s the question of accuracy as unless you’ve got something to compare that number with, you’re left with either to trust it as accurate. Or not!

In recent years, and perhaps with the above in mind, companies such as Garmin have started to aggregate all the numbers into a ‘fitness score’. I’ve made that phrase up, but I think you’ll understand what I mean rather than quoting some marketeer’s catchphrase that might conjure up something completely different.

It’s well understood that your activity performance is directly affected by your overall state of fitness. In addition, how well rested you are impacts this too; indeed when it comes to structured training, one main (and often overlooked) aspect is recovery time. If you’re not well recovered your performance is likely to be adversely affected and there’s an associated increased risk of injury as a result.

But is it meaningful?

The theory is great. But what about when it’s shoehorned into a watch and shoved on the wrist of someone who isn’t an expert?

It all started out with sleep tracking, then companies started to add HR variability (a measurement of the inter-heart-beat gaps) and along with other measurements use this to ‘predict’ how recovered and how ready you are to do sport/life things.

Interestingly it wasn’t more than a couple of years ago that companies were advising that HRV couldn’t be done accurately on wrist-based devices. So either that’s changed for the better or we’re in fantasy numbers land…….

There is some science behind it, but in order to provide something user-friendly and measurements outside of a lab, what we increasingly risk is making decisions based upon numbers we don’t fully understand. And just because a marketing department shout’s “IT’S SCIENCE” into your consumer channel of choice doesn’t necessarily make it scientific!

Feel the reviews

I’ve always been a fan of sports tech reviewers such as Ray Maker (check out his DC Rainmaker blog) as I feel that he’s worked with enough products independently over many years to at the very least sort the wheat from the chaff. It’s certainly helped me zone in on choosing devices out of the plethora that are out there in the market. And that’s just looking at Garmin’s own confusing arsenal of similar but different devices, let alone those devices from every other manufacturer!

I mention this because he does do a deep dive into some of these numbers and talks about their reliability or otherwise. Whilst this is useful (based on his industry experience) it does highlight that much of the fanfare behind these numbers is coming from a place of credibility we just can’t measure as consumers; it’s just from the marketing department of the companies themselves.

But what about this number I’ve just got?

I actually first covered this last year when I was trying to make sense of why my Apple Watch seemed to be making up the distance when compared to the several Garmin’s on my friends’ wrists. I wrote about my own experiments here, if you fancy a dive into the why’s and WTFs in that rabbit hole.

But that has nothing to do with the health metrics. So let’s get into those.

I did what last night?

The aspect that brought this into focus for me recently has been my sleep data.

Very often my perception of my sleep data seems to completely contradict what Garmin told me.

I’d wake up feeling refreshed yet Garmin would tell me it was a poor night’s sleep. Conversely on the nights where I’ve had a terrible sleep with me staring at the ceiling for hours and thus felt proper rubbish in the morning, Garmin has apparently had the best night ever.

Clutching at straws

It really doesn’t make much sense.

I am aware that my sleep actually hasn’t been that great recently. I came off my bike in early June and smashed both arms up, so there’s been a lot of discomfort as wounds have a lot of healing to do which has undoubtably impacted the quality of my sleep. And of course any repair work takes energy to do.

I believe there have been control studies where participants have been measured in lab conditions but have been (intentionally) given the wrong results to see if that had an impact e.g. someone who slept well was told they had a rough night. And the impact on the participant reflected more of this verbal cue than the actual sleep. And it was found that participants seemed to “believe” the fake result and feel like they’ve had a bad night, when actually this was not the case.

Anecdotal evidence admittedly, but it does potentially highlight the impact of biases over our own experiences.

So it has left me in an awkward place with my data. I love the idea of recording all these metrics with a view to help me get the best out of my training and activities. Yet I find myself increasingly questioning the validity of the data and thus whether it is reliable or not.

Indeed as an experiment I’m now tracking the sleep with my Apple Watch to see how Tim Cook’s reality corresponds to my own when it comes to sleeping. But I’m not doing a direct comparison i.e. a watch on either wrist because, honestly I don’t see it answering the question over “what is accurate?”

Too much detail?

When proper sleep studies are undertaken under lab conditions, the electrical activity of participants is measured directly with a set of wires attached to their heads. Researchers have been able to work out the different stages of sleep e.g. awake, light sleep, deep sleep, REM sleep based upon this electrical activity (as well as visual clues by filming the participants to see what their bodies / muscles are doing.

The consumer-market fitness watches don’t come with all these wires and external reviewing of a given night’s sleep. Instead, in order to estimate the sleep states they base things on the wearer’s movement during sleep and extrapolate from there.

As such, whilst it’s an educated guess at what is going on, that’s the top and bottom of it. And you’d have to question why the labs would go to so much trouble with the wires and clever machines if the analysis could just be done on the participant’s movements.

So whilst it’s possible (ish) to draw some conclusions about the sleep state with the activity watches, it’s important to understand the limitations of this method. Sure, they can reasonably assess whether you’re awake or asleep, but beyond that we’re running into a lot of guessing, educated or otherwise.

So what does that mean?

And this is where I find the whole thing gets tricky.  When the particular watch is working out how ready you are (or not) for more training (or whether you should be resting) based upon ‘metrics’ which are really no more than educated guesses, it leaves questions about the validity of its recommendations based on these ‘metrics’.

And this gets more confusing when you, as the user, disagree with its interpretation of your night’s sleep; does this invalidate the recommendation, or does one disregard their own feeling about how they slept?

Interestingly in the last few years Garmin’s own software now includes an option for Perceived Effort following workouts, which is where the user can input their own qualitative feeling about the session. I presume this is incorporated into the fitness algorithm (otherwise why bother). So shouldn’t this also be carried through into your own perception of how rested you feel?

Your thoughts?

Do you take any notice of the numbers that come out of your smart watch / fitness watch? Do you obsess over them at all or do you completely ignore them? Do they correspond with your own experiences or does it feel like a random number generator (plus colourful graph)?