weather-cartoon6 (1)

Perspective on the accuracy of Meteorologists

I know what you are thinking. “Here we go another Meteorologist telling me how great they are”. That is not the purpose of this blog post. I am simply attempting to put what we do in perspective. Plus I’m fascinated by how we perceive certain things about weather forecasting versus other professions we are all familiar with.

The joke starts like this……

Or the always popular,

“I wish I could be wrong ____% of the time and keep my job”

I get it we aren’t always right, even though we are accurate more times than we get credit for. What I always found fascinating is that when people give forecasters a tough time over a busted forecast. I wonder why they never apply this same logic to other professions?

Medical Field:

For instance did you know that according to the Journal of Clinical Oncology that 44% of some types of cancer are misdiagnosed? The Mayo Clinic in 2012 found that there was about a 26% misdiagnosis rate amount all illnesses. Doctors have very tough jobs and medicine has advanced  way more than meteorology. Not to mention this one gigantic caveat is that doctors are not trying to predict the future.

Now I’m not picking on doctors but we would never get angry at a doctor for being overly cautious if he or she thought we might have cancer.  Common sense says if you have a lump, dis-colored mole or other symptoms that might be cancer you get it check out ASAP. We have done a great job educating people on early diagnosis and being safe not sorry when it comes to cancer.

Yet people get mad that we issued so many tornado warnings even though on average about 75% don’t produce a tornado.

Tornado Accuracy Rates & Lead Time:

So what about tornadoes how good or bad are we?

Warning accuracy

 

(1996 – 2007 Tornado Warnings used County-Based Methodology. 2008 – Present Tornado Warnings use new Storm-Based Methodology.Weather Enterprise: Protecting Lives, Livelihoods, and Your Way of Life.)

The above chart shows the percentage of times a tornado touched down with a warning issued. That’s an average of 69% of the time a tornado touches down a warning is in place over this time period.

average lead time

The average lead time over this time period 11.6 minutes.

What about the forecast in general?

As I have blogged before we take getting the forecast correct very seriously more so than anyone can imagine. http://wxbrad.com/charlottes-most-accurate-forecast-its-not-just-marketing-for-us/

Over the past 4 years our 24 hour forecast for the next day has been right 94% of the time. I bet you are thinking, NO WAY!!! Well it’s true but that still means that 6% of the time we blow it just 24 hours out.  Over one calendar year that means 26 days we didn’t get it right.

Problem for me is that people remember those 26 days really well and really forget about the other 339 days I did get it right.

I get it…. because if we are getting it right then no one notices and that’s what we are supposed to be doing.  I will remind you again though we are trying to predict the future.

Sports; what’s good for them sucks for us!

You saw the tornado warning accuracy stats around 70% and our 24 hour forecast accuracy of 94%. What if we compared that to All-Star and Hall a Fame athletes, who are doing incredible things yet still not trying to predict the future?

NBA:

Amir Johnson let the league in field goal percentage this last season at 65%.  Which means he missed 35% of the time. Kevin Durant was the MVP and he only shot 50.3% from the field. Meaning he missed half his shots. He did make 87% of his free throws though.

Again not predicting the future.

NFL:

Philip Rivers led the league in competition percentage last season at 69.5%. That was awesome but he still missed 30.5% of his passes. Again a very hard thing to do is to play NFL quarterback but still not predicting the future.

MLB:

Always a stats field day here. Miguel Cabrera led the league in hitting last season at .348 which basically means he got a hit 35% of the time. That means he didn’t get a hit the other 65% of the time. Ty Cobb is considered the best hitter of all time at .366 but that’s only a 37% success rate. Again hitting a baseball maybe the hardest thing to do in sports, yet still not predicting the future. Though hitters always try to guess what type of pitch is coming next.

Conclusion:

Breaking News….we aren’t always right. The beauty is I know this and I work within the inherent uncertainty of weather forecasting. I know I can’t get it right all the time my goal is to reduce the error to as small as possible.  I also learn more from a blown forecast just as much or more than a correct one. Like any walk of life you have to fall down to learn to get up. Mistakes are only bad if you don’t learn from them.

Just remember to keep things in perspective. Whatever job or career you have. Think about the times you messed up. Some small, some large, but usually always correctable. Think about the mistakes that happen in everyone’s line of work that have nothing to do with predicting the future. Then realizing how far we have come and how awesome it is that in 2015 we can get the 24 hour forecast correct 94% of the time and give you on average 12 minutes warning before a tornado hits. It amazes me and I work as a Meteorologist.

  • Jesse Ferrell

    Excellent post.

    ALSO: Investment (stock) gurus (who ARE predicting the future) have an accuracy rate of 22%-68%, avg. 47% http://ow.ly/x1g5U

  • Josh Herman

    I see it differently. When does the weather forecast really matter? When weather will actually impact the area. Take precipitation for example, that seems to carry an impact. Charlotte averages 69 days (00-13) with pcpn > 0.10″ a year. Lets assume of the 26 days that are remembered as busts per year, there was actual weather (pcpn) affecting the area. Instead of the gaudy 6% bust ratio, it’s a more realistic 37% bust ratio. LOL.

    • wxbrad

      Couple of wrong assumptions there Josh, the busted forecast number accounts for temperature and precipitations busts. Not all 6% of our busts were predication busts almost 60% were temperature busts. Which especially in the winter and the growing season have even larger impacts than ;precipitation. With freezing temperature forecasts being very impactful to travel and crops.

      • Josh Herman

        How many days a year is there impacting weather in your area. Those are the days that should be calculated with your bust ratio, not an entire calendar year. Your numbers are inflated. :)

        • wxbrad

          Impactfull is just too subjective, virtually every type of weather is impactful to certain segments of the population. The numbers speak for themselves IMO. We get blamed for a wrong forecast even when they are right because if it rains on someones event. then they think its wrong, even if we forecast it. People perceive that as wrong. We sometimes bust even when people don’t notice an think we got it right. Like when it rains in the middle of the night when no one notices and we didn’t forecast it. Sometimes we forecast 85° and it only hits 81° people don’t notice, but it still goes down as a bust. In the end we actually do really well on the extreme events because they are easier to forecast. The marginal events are the tough ones, wedge events during the cool season and isolated thunderstorms in the summer.

          • Josh Herman
          • wxbrad

            Thank you, yeah most of those forecasts are model derived only which is something I wanted to add to my post. Just to show how quality control of a human forecaster can improve the forecast from just sticky model driven forecast.

          • Josh Herman

            Another reason to stick with local Mets when an accurate forecast is desired.

  • Pingback: The Deterministic Dilemma | The Show()

  • Pingback: The Deterministic Dilemma - Dan's Wild Wild Science Journal - AGU Blogosphere()

  • Jim

    “Over the past 4 years our 24 hour forecast for the next day has been right 94% of the time.” Just curious, who is “our” in this statement? Also, I would be interested in hearing your thoughts about the accuracy (and usefulness) of detailed 30 – 45 day forecasts. Thank you.

  • Dave

    Brad, thank you for all the extra things you do. You should not have to explain yourself to anyone. People should remember this, a meteorologist does not control the weather, they predict what the weather. A meteorologist cannot predict to the exact mile where a low pressure system will track, but predict a general vicinity. Although I was disappointed by the amount of snow we got, or the lack thereof, I cannot blame that on someone who is going out of their way to keep us all informed; and who has zero control on the weather itself. Thank you again for keeping us all informed on the weather.

  • Scott

    Well said!

  • Carrie

    Some more perspective for you, or a “you’re not alone” comment. The vast majority of police officers are not racist or given to excessive brutality, but what has dominated the media for the past 6 months? The vast majority of homeschoolers are effectively, compassionately and responsibly educating their children, but who ever talks about that? And meteorologists forecast the weather accurately 90% of the time, but no one comments on those forecasts. Bottom line, failures of the minority make more waves. But those of us in the majority just have to keep on faithfully working and know the truth about what we do. You’re not alone.

  • James Urban

    I know of a local TV Meteorologist that precedes each of his weather segments with the words “Accurate” & “Trusted”. By the strict definition of the the word, ACCURATE means “without error”. He is rarely correct when it comes to the high or low temperature in the next 24 hours.. When I confronted him online regarding his claim and results, he indicates that what is meant is that he is the “most accurate” of the local TV meteorologists. When I asked him to show independent proof of that claim, he conveniently ignores the question. His exact words to me is, “I know in my heart of hearts that I am the most accurate.” I told him you are just trying to gain viewership through a false claim. Now another local TV meteorologist/forecaster is claiming to have the most accurate weather forecast. Both can not be the most accurate. It bothers me because both of these people are supposed to be professionals and they lack any semblance of credibility.

    • Rob Haswell

      Actually James…

      Merriam Webster defines accurate in three ways. You are accurate about one.

      However…
      2: conforming exactly to truth or to a standard
      3: able to give an accurate result

      Both would be appropriate when talking about meteorology. All meteorologist (TV or otherwise) conform to a standard of practice.. thus accurate. All good meteorologist are able to give an accurate result with the proper tools and data… thus accurate.

      So it seems to me that by only using one of the three definitions, you comment itself is only 33% accurate.

      http://www.merriam-webster.com/dictionary/accurate

    • rick

      If when you predict a tornado your only right 25% of the time and when a tornado happens its only predicted 70% of the time. You are actually only right about 14% of the time. You can twist numbers any way you like but a quarter with a 14% completion percentage isn’t good and he doesn’t get to keep his job