INFORMS Open Forum

My new article with Scott Armstrong presents evidence that complexity increases forecast error. Have we missed any evidence that might challenge that conclusion?

The article, "Simple Forecasting: Avoid Tears Before Bedtime" proposes that simplicity in forecasting requires (1) method, (2) representation of cumulative knowledge, (3) relationships in models, and (4) relationships among models, forecasts, and decisions are all sufficiently uncomplicated as to be easily understood by decision makers. Our review of studies comparing simple and complex methods has found 93 comparisons in 28 papers. Complexity beyond the sophisticatedly simple failed to improve accuracy in all of the studies and increased forecast error by an average of 32 percent in 21 studies with quantitative comparisons.

The effects are so consistent and substantial that we are concerned that we might have overlooked disconfirming evidence. If you know this area, please look at the references in the paper to see if we have overlooked any key studies, and send me your suggestions. The paper is available on the simple-forecasting.com page.

------------------------------
Kesten Green
University of South Australia
Adelaide
------------------------------

I get security warning on the site http://www.kestencgreen.com/whiich I was led to when the site simple-forecasting.com was not recognizable.
(Sorry, We could not find simple-forecasting.com)

I do not know the source of the security alerts I received on the above web site. It may or may not be related to the controversy of advocacy politics surrounding global warming, where Armstrong and Green's work is used to question the forecasting accuracy of the surrounding science. They are quoted as saying, "Based on our analyses, we concluded that the global warming alarm is an anti-scientific political movement. " This was quoted in a report before a US House subcommittee as captured at http://science.house.gov/sites/republicans.science.house.gov/files/documents/hearings/Report%20for%20Congressional%20hearing-R14%20%282%29%20armstrong%20update.pdf.

I also do not know if this problem is on-going -- it may be momentary, but I thought it worth mentioning.


------------------------------
Roger Shepherd
Business Engineering
Crook and Fleece
Laurel MD
------------------------------


Original Message:
Sent: 11-07-2014 18:26
From: Kesten Green
Subject: Last call for evidence that complex forecasting methods are better

My new article with Scott Armstrong presents evidence that complexity increases forecast error. Have we missed any evidence that might challenge that conclusion?

The article, "Simple Forecasting: Avoid Tears Before Bedtime" proposes that simplicity in forecasting requires (1) method, (2) representation of cumulative knowledge, (3) relationships in models, and (4) relationships among models, forecasts, and decisions are all sufficiently uncomplicated as to be easily understood by decision makers. Our review of studies comparing simple and complex methods has found 93 comparisons in 28 papers. Complexity beyond the sophisticatedly simple failed to improve accuracy in all of the studies and increased forecast error by an average of 32 percent in 21 studies with quantitative comparisons.

The effects are so consistent and substantial that we are concerned that we might have overlooked disconfirming evidence. If you know this area, please look at the references in the paper to see if we have overlooked any key studies, and send me your suggestions. The paper is available on the simple-forecasting.com page.

------------------------------
Kesten Green
University of South Australia
Adelaide
------------------------------

I'm getting this as a dead link:
"The paper is available on the simple-forecasting.com page."
The link was also dead when I tried to access it from here:
http://www.forecastingprinciples.com/index.php?option=com_content&view=article&id=610:last-call-for-evidence-that-complex-forecasting-methods-are-better&catid=34&Itemid=201 


------------------------------
Michael Kruger
Glenview IL
------------------------------


Original Message:
Sent: 11-07-2014 18:26
From: Kesten Green
Subject: Last call for evidence that complex forecasting methods are better

My new article with Scott Armstrong presents evidence that complexity increases forecast error. Have we missed any evidence that might challenge that conclusion?

The article, "Simple Forecasting: Avoid Tears Before Bedtime" proposes that simplicity in forecasting requires (1) method, (2) representation of cumulative knowledge, (3) relationships in models, and (4) relationships among models, forecasts, and decisions are all sufficiently uncomplicated as to be easily understood by decision makers. Our review of studies comparing simple and complex methods has found 93 comparisons in 28 papers. Complexity beyond the sophisticatedly simple failed to improve accuracy in all of the studies and increased forecast error by an average of 32 percent in 21 studies with quantitative comparisons.

The effects are so consistent and substantial that we are concerned that we might have overlooked disconfirming evidence. If you know this area, please look at the references in the paper to see if we have overlooked any key studies, and send me your suggestions. The paper is available on the simple-forecasting.com page.

------------------------------
Kesten Green
University of South Australia
Adelaide
------------------------------

Kesten,

Very interesting article. (Your link simple-forcasting doesn't work)

What does your research say about 'big data', where complexity is guaranteed, causality is unknown, and understanding is impossible?

------------------------------
Robert Rose
President
Optimal Decisions, LLC
Franklin Park NJ
------------------------------


Original Message:
Sent: 11-07-2014 18:26
From: Kesten Green
Subject: Last call for evidence that complex forecasting methods are better

My new article with Scott Armstrong presents evidence that complexity increases forecast error. Have we missed any evidence that might challenge that conclusion?

The article, "Simple Forecasting: Avoid Tears Before Bedtime" proposes that simplicity in forecasting requires (1) method, (2) representation of cumulative knowledge, (3) relationships in models, and (4) relationships among models, forecasts, and decisions are all sufficiently uncomplicated as to be easily understood by decision makers. Our review of studies comparing simple and complex methods has found 93 comparisons in 28 papers. Complexity beyond the sophisticatedly simple failed to improve accuracy in all of the studies and increased forecast error by an average of 32 percent in 21 studies with quantitative comparisons.

The effects are so consistent and substantial that we are concerned that we might have overlooked disconfirming evidence. If you know this area, please look at the references in the paper to see if we have overlooked any key studies, and send me your suggestions. The paper is available on the simple-forecasting.com page.

------------------------------
Kesten Green
University of South Australia
Adelaide
------------------------------