INFORMS Open Forum

Over the last few years, we've seen rapid advances in machine learning and analytics models, yet many initiatives struggle to deliver sustained business value.

In applied settings, the main bottlenecks often appear after model development: orchestration, data governance, monitoring, and alignment with decision processes.

From your experience, which operational or organizational factors most often limit the long-term impact of analytics solutions?

I'd be interested in hearing perspectives from both academic and industry settings.

The INFORMS Analytics Framework^TM addresses these issues, but from the perspective of issues to consider prior to embarking on any analytics initiative. See our recent article in OR/MS Today here:  https://pubsonline.informs.org/do/10.1287/orms.2025.04.07/full/ .  You can download the details here:  https://www.informs.org/Professional-Development/Professional-Development-Classes/INFORMS-Analytics-Framework

To more directly answer your question, I would list the following issues:

• Creating models to solve some problem without first creating a business problem statement, as opposed to starting with an analytics problem statement
• Not understanding the issues of deploying a potential solution prior to creating the solution.  If you're going to come up with some solution, you need to understand how that solution will be implemented in the business context, including what change management principles will need to be applied to implement the solution in that context.
• Assuming that the right data exists prior to coming up with a model to solve the problem.  People assume that all the necessary data is available, but often it isn't.
• Not getting stakeholder agreement on the business problem statement, the analytics problem statement, and the vision of how the solution will change business operations.
• Not planning for continuous monitoring of the solution in the context of the business environment.  Changes will happen due to changes in business operations, or even changes in the design of some back-end database that is feeding the solution.

I'm sure there is more I could add, but those are the ones that more immediately come to mind.

Over the last few years, we've seen rapid advances in machine learning and analytics models, yet many initiatives struggle to deliver sustained business value.

In applied settings, the main bottlenecks often appear after model development: orchestration, data governance, monitoring, and alignment with decision processes.

From your experience, which operational or organizational factors most often limit the long-term impact of analytics solutions?

I'd be interested in hearing perspectives from both academic and industry settings.

Irv and Richard, another issue I've seen might be called 'model creep'. After a model is deployed, the users very quickly figure out that something more, or something different, could be done. The model and the associated procedures and operating practice might not support that very well.  In my experience this usually takes about 6 months from deployment, though it can occur much more quickly.  This affects both perception of the model among the business participants and the ability of the support team to respond to make the enhancements.

I think that to respond to this the analytics team must be constantly be thinking during design, construction, training, and deployment of simplicity and resilience of the entire modeling complex. Don't program in anything that will be hard to change in the future.

The INFORMS Analytics Framework^TM addresses these issues, but from the perspective of issues to consider prior to embarking on any analytics initiative. See our recent article in OR/MS Today here:  https://pubsonline.informs.org/do/10.1287/orms.2025.04.07/full/ .  You can download the details here:  https://www.informs.org/Professional-Development/Professional-Development-Classes/INFORMS-Analytics-Framework

To more directly answer your question, I would list the following issues:

• Creating models to solve some problem without first creating a business problem statement, as opposed to starting with an analytics problem statement
• Not understanding the issues of deploying a potential solution prior to creating the solution.  If you're going to come up with some solution, you need to understand how that solution will be implemented in the business context, including what change management principles will need to be applied to implement the solution in that context.
• Assuming that the right data exists prior to coming up with a model to solve the problem.  People assume that all the necessary data is available, but often it isn't.
• Not getting stakeholder agreement on the business problem statement, the analytics problem statement, and the vision of how the solution will change business operations.
• Not planning for continuous monitoring of the solution in the context of the business environment.  Changes will happen due to changes in business operations, or even changes in the design of some back-end database that is feeding the solution.

I'm sure there is more I could add, but those are the ones that more immediately come to mind.

Over the last few years, we've seen rapid advances in machine learning and analytics models, yet many initiatives struggle to deliver sustained business value.

In applied settings, the main bottlenecks often appear after model development: orchestration, data governance, monitoring, and alignment with decision processes.

From your experience, which operational or organizational factors most often limit the long-term impact of analytics solutions?

I'd be interested in hearing perspectives from both academic and industry settings.

The INFORMS Analytics Framework^TM addresses these issues, but from the perspective of issues to consider prior to embarking on any analytics initiative. See our recent article in OR/MS Today here:  https://pubsonline.informs.org/do/10.1287/orms.2025.04.07/full/ .  You can download the details here:  https://www.informs.org/Professional-Development/Professional-Development-Classes/INFORMS-Analytics-Framework

To more directly answer your question, I would list the following issues:

• Creating models to solve some problem without first creating a business problem statement, as opposed to starting with an analytics problem statement
• Not understanding the issues of deploying a potential solution prior to creating the solution.  If you're going to come up with some solution, you need to understand how that solution will be implemented in the business context, including what change management principles will need to be applied to implement the solution in that context.
• Assuming that the right data exists prior to coming up with a model to solve the problem.  People assume that all the necessary data is available, but often it isn't.
• Not getting stakeholder agreement on the business problem statement, the analytics problem statement, and the vision of how the solution will change business operations.
• Not planning for continuous monitoring of the solution in the context of the business environment.  Changes will happen due to changes in business operations, or even changes in the design of some back-end database that is feeding the solution.

I'm sure there is more I could add, but those are the ones that more immediately come to mind.

Over the last few years, we've seen rapid advances in machine learning and analytics models, yet many initiatives struggle to deliver sustained business value.

In applied settings, the main bottlenecks often appear after model development: orchestration, data governance, monitoring, and alignment with decision processes.

From your experience, which operational or organizational factors most often limit the long-term impact of analytics solutions?

I'd be interested in hearing perspectives from both academic and industry settings.

Over the last few years, we've seen rapid advances in machine learning and analytics models, yet many initiatives struggle to deliver sustained business value.

In applied settings, the main bottlenecks often appear after model development: orchestration, data governance, monitoring, and alignment with decision processes.

From your experience, which operational or organizational factors most often limit the long-term impact of analytics solutions?

I'd be interested in hearing perspectives from both academic and industry settings.