Skip to content

Perspective on the accuracy of Meteorologists

weather-cartoon6 (1)

I know what you are thinking. “Here we go another Meteorologist telling me how great they are”. That is not the purpose of this blog post. I am simply attempting to put what we do in perspective. Plus I’m fascinated by how we perceive certain things about weather forecasting versus other professions we are all familiar with.

The joke starts like this……

Or the always popular,

“I wish I could be wrong ____% of the time and keep my job”

I get it we aren’t always right, even though we are accurate more times than we get credit for. What I always found fascinating is that when people give forecasters a tough time over a busted forecast. I wonder why they never apply this same logic to other professions?

Medical Field:

For instance did you know that according to the Journal of Clinical Oncology that 44% of some types of cancer are misdiagnosed? The Mayo Clinic in 2012 found that there was about a 26% misdiagnosis rate amount all illnesses. Doctors have very tough jobs and medicine has advanced  way more than meteorology. Not to mention this one gigantic caveat is that doctors are not trying to predict the future.

Now I’m not picking on doctors but we would never get angry at a doctor for being overly cautious if he or she thought we might have cancer.  Common sense says if you have a lump, dis-colored mole or other symptoms that might be cancer you get it check out ASAP. We have done a great job educating people on early diagnosis and being safe not sorry when it comes to cancer.

Yet people get mad that we issued so many tornado warnings even though on average about 75% don’t produce a tornado.

Tornado Accuracy Rates & Lead Time:

So what about tornadoes how good or bad are we?

Warning accuracy

 

(1996 – 2007 Tornado Warnings used County-Based Methodology. 2008 – Present Tornado Warnings use new Storm-Based Methodology.Weather Enterprise: Protecting Lives, Livelihoods, and Your Way of Life.)

The above chart shows the percentage of times a tornado touched down with a warning issued. That’s an average of 69% of the time a tornado touches down a warning is in place over this time period.

average lead time

The average lead time over this time period 11.6 minutes.

What about the forecast in general?

As I have blogged before we take getting the forecast correct very seriously more so than anyone can imagine. http://wxbrad.com/charlottes-most-accurate-forecast-its-not-just-marketing-for-us/

Over the past 4 years our 24 hour forecast for the next day has been right 94% of the time. I bet you are thinking, NO WAY!!! Well it’s true but that still means that 6% of the time we blow it just 24 hours out.  Over one calendar year that means 26 days we didn’t get it right.

Problem for me is that people remember those 26 days really well and really forget about the other 339 days I did get it right.

I get it…. because if we are getting it right then no one notices and that’s what we are supposed to be doing.  I will remind you again though we are trying to predict the future.

Sports; what’s good for them sucks for us!

You saw the tornado warning accuracy stats around 70% and our 24 hour forecast accuracy of 94%. What if we compared that to All-Star and Hall a Fame athletes, who are doing incredible things yet still not trying to predict the future?

NBA:

Amir Johnson let the league in field goal percentage this last season at 65%.  Which means he missed 35% of the time. Kevin Durant was the MVP and he only shot 50.3% from the field. Meaning he missed half his shots. He did make 87% of his free throws though.

Again not predicting the future.

NFL:

Philip Rivers led the league in competition percentage last season at 69.5%. That was awesome but he still missed 30.5% of his passes. Again a very hard thing to do is play NFL quarterback but still not predicting the future.

MLB:

Always a stats field day here. Miguel Cabrera led the league in hitting last season at .348 which basically means he got a hit 35% of the time. That means he didn’t get a hit the other 65% of the time. Ty Cobb is considered the best hitter of all time at .366 but that’s only a 37% success rate. Again hitting a baseball maybe the hardest thing to do in sports, yet still not predicting the future. Though hitters always try to guess what type of pitch is coming next.

Conclusion:

Breaking News….we aren’t always right. The beauty is I know this and I work within the inherent uncertainty of weather forecasting. I know I can’t get it right all the time my goal is to reduce the error to as small as possible.  I also learn more from a blown forecast just as much or more than a correct one. Like any walk of life you have to fall down to learn to get up. Mistakes are only bad if you don’t learn from them.

Just remember to keep things in perspective. Whatever job or career you have. Think about the times you messed up. Some small, some large, but usually always correctable. Think about the mistakes that happen in everyone’s line of work that have nothing to do with predicting the future. Then realizing how far we have come and how awesome it is that in 2014 we can get the 24 hour forecast correct 94% of the time and give you on average 12 minutes warning before a tornado hits. It amazes me and I work as a Meteorologist.

  • Jesse Ferrell

    Excellent post.

    ALSO: Investment (stock) gurus (who ARE predicting the future) have an accuracy rate of 22%-68%, avg. 47% http://ow.ly/x1g5U

  • Josh Herman

    I see it differently. When does the weather forecast really matter? When weather will actually impact the area. Take precipitation for example, that seems to carry an impact. Charlotte averages 69 days (00-13) with pcpn > 0.10″ a year. Lets assume of the 26 days that are remembered as busts per year, there was actual weather (pcpn) affecting the area. Instead of the gaudy 6% bust ratio, it’s a more realistic 37% bust ratio. LOL.

    • wxbrad

      Couple of wrong assumptions there Josh, the busted forecast number accounts for temperature and precipitations busts. Not all 6% of our busts were predication busts almost 60% were temperature busts. Which especially in the winter and the growing season have even larger impacts than ;precipitation. With freezing temperature forecasts being very impactful to travel and crops.

      • Josh Herman

        How many days a year is there impacting weather in your area. Those are the days that should be calculated with your bust ratio, not an entire calendar year. Your numbers are inflated. :)

        • wxbrad

          Impactfull is just too subjective, virtually every type of weather is impactful to certain segments of the population. The numbers speak for themselves IMO. We get blamed for a wrong forecast even when they are right because if it rains on someones event. then they think its wrong, even if we forecast it. People perceive that as wrong. We sometimes bust even when people don’t notice an think we got it right. Like when it rains in the middle of the night when no one notices and we didn’t forecast it. Sometimes we forecast 85° and it only hits 81° people don’t notice, but it still goes down as a bust. In the end we actually do really well on the extreme events because they are easier to forecast. The marginal events are the tough ones, wedge events during the cool season and isolated thunderstorms in the summer.

          • Josh Herman
          • wxbrad

            Thank you, yeah most of those forecasts are model derived only which is something I wanted to add to my post. Just to show how quality control of a human forecaster can improve the forecast from just sticky model driven forecast.

          • Josh Herman

            Another reason to stick with local Mets when an accurate forecast is desired.

  • Pingback: The Deterministic Dilemma | The Show

  • Pingback: The Deterministic Dilemma - Dan's Wild Wild Science Journal - AGU Blogosphere