Author Topic: data and decisions  (Read 185 times)

0 Members and 1 Guest are viewing this topic.

Offline screwtape

  • The Great Red Dragon
  • Administrator
  • *******
  • Posts: 12340
  • Darwins +677/-28
  • Gender: Male
  • Karma mooch
data and decisions
« on: May 07, 2014, 01:22:08 PM »
Very important piece on data vs intuition. 
http://www.nytimes.com/2014/05/07/upshot/universal-mammogram-screening-shows-we-dont-understand-risk.html?rref=upshot&_r=0

It puts it in terms of healthcare, but the principle is the same.  We are not highly rational animals.  We take shortcuts in thinking and making decisions.  We must be smart enough to realize this and let the data drive our decisions.  This is hard because it requires effort.  The last paragraph really crushes it.

Quote
We can’t make these kinds of changes, though, while we allow beliefs to trump facts. Study after study, and our reaction to them, show that our perceptions don’t line up with reality. If we want to live longer, healthier, and, most importantly, better lives, it’s worth fighting some of those assumptions.

Links:
Rules
Guides & Tutorials

What's true is already so. Owning up to it does not make it worse.

Offline penfold

  • Postgraduate
  • *****
  • Posts: 681
  • Darwins +63/-4
  • Gender: Male
  • ...buzz buzz buzz...
Re: data and decisions
« Reply #1 on: May 08, 2014, 10:05:50 AM »
Have you read Thinking Fast and Slow[1] by Daniel Kahnemen?

He's very big on the side of data over intuition. Not sure I agree in all cases though; data, especially big data, can really lack resolution. I teach and am constantly bombarded with data; and interesting one came up recently - a study showing that "what" and "how" questions work better on students than "why" questions (less aggressive). I don't doubt the data, but I still trust my intuition to tell me when a student is ready for a "why" question, and when a more aggressive style is called for in order to push a student out of their comfort zone.

Data is great for policy, but freedom should be left in the hands of the practitioners to use their intuition or else we only serve those who sit in the middle of the bell curve and leave behind the outliers.
 1. http://www.amazon.co.uk/Thinking-Fast-Slow-Daniel-Kahneman/dp/0141033576
"Reality is that which, when you stop believing in it, doesn’t go away." - P.K.D.

Offline jdawg70

  • Reader
  • ******
  • Posts: 2102
  • Darwins +375/-8
  • Ex-rosary squad
Re: data and decisions
« Reply #2 on: May 08, 2014, 10:28:45 AM »
Have you read Thinking Fast and Slow[1] by Daniel Kahnemen?

He's very big on the side of data over intuition. Not sure I agree in all cases though; data, especially big data, can really lack resolution. I teach and am constantly bombarded with data; and interesting one came up recently - a study showing that "what" and "how" questions work better on students than "why" questions (less aggressive). I don't doubt the data, but I still trust my intuition to tell me when a student is ready for a "why" question, and when a more aggressive style is called for in order to push a student out of their comfort zone.

Data is great for policy, but freedom should be left in the hands of the practitioners to use their intuition or else we only serve those who sit in the middle of the bell curve and leave behind the outliers.
 1. http://www.amazon.co.uk/Thinking-Fast-Slow-Daniel-Kahneman/dp/0141033576
I would think that practitioners primarily using their intuition rather than data would be the ones in danger of sitting just in the middle of the bell curve, ignoring the outliers.  It is unexpected data that allows us to see the outliers, not our intuition.  Intuition is, in a sense, the result of large amounts of potentially correlated data being processed for long periods of time that 'wire in' shortcut algorithms for our decision making process.  Intuition has the effect of elevating the weight of data that meets our expectations over the weight of data that do not meet our expectations (in terms of how that data is incorporated into our decision-making process).

Intuition is great - it's the process of creating cognitive shortcuts for us so that we don't have to spend so much time rationally processing every little bit of data.  It allows us to make connections (that are actually correlated) that we otherwise wouldn't make if subject just to rationality.  However, it also allows us to make connections that we otherwise wouldn't make if subject to just rationality that are not actually correlated.

So how best to take advantage of both worlds?  I propose that one should accept their intuition, but if possible, check it against the data, and if the data disagrees with your intuition, go with the data.  If your intuition is strong enough to make the data less compelling than you rationally think it should be (i.e. the data says it's a straight line, but it really, really, really feels like a logarithmic function), bring other intuitions (i.e. other people) into the decision making process.  Do what we can to eliminate intuitive gut feel that comes from correlating data that isn't actually correlated.  It's not perfect, and will still lead to some missing of outliers, but this seems, to me, to be the most practical consideration.

But I think I totally need to read that book.
"When we landed on the moon, that was the point where god should have come up and said 'hello'. Because if you invent some creatures, put them on the blue one and they make it to the grey one, you f**king turn up and say 'well done'."

- Eddie Izzard

http://deepaksducttape.wordpress.com/

Offline screwtape

  • The Great Red Dragon
  • Administrator
  • *******
  • Posts: 12340
  • Darwins +677/-28
  • Gender: Male
  • Karma mooch
Re: data and decisions
« Reply #3 on: May 08, 2014, 11:51:06 AM »
Have you read Thinking Fast and Slow[1] by Daniel Kahnemen?
 1. http://www.amazon.co.uk/Thinking-Fast-Slow-Daniel-Kahneman/dp/0141033576

Yes!  I've been picking at it on my kindle for a couple of months.  I'm about 1/4 of the way through.  It is amazing stuff, but I just cannot get any momentum with the reading.  I keep getting distracted by stupid stuff.  I planned to do a book report here when I finished. 

Data is great for policy, but freedom should be left in the hands of the practitioners to use their intuition...

eehh, I disagree.  Having data driven policy is pointless if it is applied intuitively.  The whole concept is undermined.

If I am Hal Steinbrenner, I don't want my manager going off range and putting in a pinch hitter based on his gut.  Follow the data.  What is the on deck batter's OBA against this pitcher as compared to the PH's?  Lefty-righty?  Go with the averages.  The gut PH might get a hit, but that does not mean his gut knew anything.   

Intuition, like faith, is so wrongly celebrated, vaunted and valued.  It scares the crap out of me.
All these people got it so, so wrong:
http://www.ideachampions.com/weblogs/archives/2011/02/the_only_real_v.shtml
Especially Oprah.  Fuckin Oprah.

This one too. This guy is going the completely wrong way.  And he's supposed to be a business leadership guru.
http://www.goodreads.com/quotes/19853-leaders-trust-their-guts-intuition-is-one-of-those-good
Maybe he forgot our last president was gut-driven and where that landed us?
 
Links:
Rules
Guides & Tutorials

What's true is already so. Owning up to it does not make it worse.

Offline penfold

  • Postgraduate
  • *****
  • Posts: 681
  • Darwins +63/-4
  • Gender: Male
  • ...buzz buzz buzz...
Re: data and decisions
« Reply #4 on: May 09, 2014, 01:24:46 AM »
I would think that practitioners primarily using their intuition rather than data would be the ones in danger of sitting just in the middle of the bell curve, ignoring the outliers.  It is unexpected data that allows us to see the outliers, not our intuition.  Intuition is, in a sense, the result of large amounts of potentially correlated data being processed for long periods of time that 'wire in' shortcut algorithms for our decision making process.  Intuition has the effect of elevating the weight of data that meets our expectations over the weight of data that do not meet our expectations (in terms of how that data is incorporated into our decision-making process).

Intuition is great - it's the process of creating cognitive shortcuts for us so that we don't have to spend so much time rationally processing every little bit of data.  It allows us to make connections (that are actually correlated) that we otherwise wouldn't make if subject just to rationality.  However, it also allows us to make connections that we otherwise wouldn't make if subject to just rationality that are not actually correlated.

So how best to take advantage of both worlds?  I propose that one should accept their intuition, but if possible, check it against the data, and if the data disagrees with your intuition, go with the data.  If your intuition is strong enough to make the data less compelling than you rationally think it should be (i.e. the data says it's a straight line, but it really, really, really feels like a logarithmic function), bring other intuitions (i.e. other people) into the decision making process.  Do what we can to eliminate intuitive gut feel that comes from correlating data that isn't actually correlated.  It's not perfect, and will still lead to some missing of outliers, but this seems, to me, to be the most practical consideration.

But I think I totally need to read that book.

Well put, perhaps I am on the wrong side of the fence on this one... Will think on it  :)
"Reality is that which, when you stop believing in it, doesn’t go away." - P.K.D.

Offline penfold

  • Postgraduate
  • *****
  • Posts: 681
  • Darwins +63/-4
  • Gender: Male
  • ...buzz buzz buzz...
Re: data and decisions
« Reply #5 on: May 09, 2014, 01:42:40 AM »
Yes!  I've been picking at it on my kindle for a couple of months.  I'm about 1/4 of the way through.  It is amazing stuff, but I just cannot get any momentum with the reading.  I keep getting distracted by stupid stuff.  I planned to do a book report here when I finished. 

It is well worth finishing up - one of those books that really makes you see everything just a little bit differently.

A friend of mine who is doing his phd on philosophy and cog. sci. and very involved in this kind of area did warn me that a number of Kahneman's experiments have been found to have very low repeatability - but don't know where he gets that from, if I remember will send him an email and ask.

Anyhow enjoy  :)
"Reality is that which, when you stop believing in it, doesn’t go away." - P.K.D.

Offline Jag

  • Reader
  • ******
  • Posts: 1793
  • Darwins +191/-7
  • Gender: Female
  • Official WWGHA Harpy, Ex-rosary squad
Re: data and decisions
« Reply #6 on: May 09, 2014, 12:39:48 PM »
Someone I used to work with had a good way of framing this. He said that knowledge is a combination of data and information, and that good decision-making requires both.

It's not particularly profound but it gave me the leverage to use on my then-boss to change the approach on a project I was struggling to get off the ground. I had piles of data, but little information from the people who did the work represented by the data points. It made a significant difference in the implementation process.
"It's hard to, but I'm starting to believe some of you actually believe these things.  That is completely beyond my ability to understand if that is really the case, but things never cease to amaze me."

Offline screwtape

  • The Great Red Dragon
  • Administrator
  • *******
  • Posts: 12340
  • Darwins +677/-28
  • Gender: Male
  • Karma mooch
Re: data and decisions
« Reply #7 on: May 09, 2014, 01:30:05 PM »
He said that knowledge is a combination of data and information,

?  what's the difference?
Links:
Rules
Guides & Tutorials

What's true is already so. Owning up to it does not make it worse.

Offline Jag

  • Reader
  • ******
  • Posts: 1793
  • Darwins +191/-7
  • Gender: Female
  • Official WWGHA Harpy, Ex-rosary squad
Re: data and decisions
« Reply #8 on: May 09, 2014, 02:58:59 PM »
I suppose it would vary depending on what is under discussion.

In this case, we had literally tens of thousands of computer records (data) of various shipments paid for by the highly decentralized company. What the data couldn't provide was the rationale behind the decisions (information) to ship in any particular way, or what the shipment actually contained (actually a stand-alone problem). This was in the medical field, and a shipment could be anything from tiny screws to surgical room equipment, with the occasional body donated to science thrown in for good measure.

The data told me what was done, but in order to implement effective change, I also needed to know why it was done that specific way. THAT was only known to the buyers, not to our department. Sometimes the answer was as simple as knowing what a shipment contained (or what a particular vendor provides routinely), other times it was because the person who decided didn't know any better, or was under contract to a carrier, or under time-constraints (temperature sensitive or viability issues), or any of a hundred other reasons. Context mattered a lot.

Data is the facts, information is "the rest of the story"? I'm not sure that's quite right... data is the objective part, and information is the subjective, or human factor? Information is whatever relevant to the matter that the data alone can't tell you, is probably closest. I'm sure it can't be applied to everything, but it was a really useful distinction at the time.
"It's hard to, but I'm starting to believe some of you actually believe these things.  That is completely beyond my ability to understand if that is really the case, but things never cease to amaze me."