An independent study reference written by Dr Phuc V. Nguyen. It is not official subject material — for assessment requirements always follow your subject outline and vUWS.
Why analytics work stalls
Most analytics that fails is not wrong. It is unused. The recurring causes are structural. No decision was named, or nobody has authority to act, so the output has nowhere to land. The result arrives after the decision window closed. The data was fit for billing but not for the question. The model never left a laptop because nobody owned the integration. The finding threatens a target somebody is measured on. Nobody checked whether acting on it helped, so at the next budget round the work cannot defend itself. The last stretch between a correct answer and a changed decision is where most value is lost.
Why it matters
Imagine a very good weather forecast delivered on Friday afternoon to a farmer who decides on Thursday. The forecast is not wrong. It is useless. Almost every stalled analytics project is a version of that. Something about timing, ownership, format or trust means the answer never reaches the moment where a person actually chooses. Fixing that is less glamorous than the modelling and usually worth far more.
A model predicting equipment failure is accurate, but maintenance crews ignore it. Which investigation is most likely to find the cause?
Worked examples
A grocery wholesaler improves demand forecast accuracy noticeably, but buyers keep ordering from their own spreadsheets. Diagnose the stall.
Trace the decision rather than the model. Buyers place orders on Thursday morning. The new forecast arrives as a Friday email attachment, in a format that has to be retyped, produced by a team the buyers have never met. Three separate failures sit in that sentence: timing, format and trust. None of them is fixed by a better model. The repair is to deliver the forecast into the ordering screen before Thursday, agree with the buyers how they can override it, and show them the first month of results.
An analysis shows a promotion loses money once cannibalisation is counted. The category manager whose bonus depends on volume disputes it and the report is shelved.
This is an incentive stall, and better evidence will not resolve it. The workable responses are procedural. Agree the measurement rules before the next promotion runs, so the outcome is not negotiated afterwards. Present the result to whoever owns the margin, not only to whoever owns the volume. Where possible, run the next promotion in some regions and not others, so the comparison is designed in and does not depend on anybody conceding a point.
Common mistakes
- ✗Analytics projects fail mainly because of bad models. Far more fail on framing, timing, data fitness and adoption. A mediocre model wired into the decision beats an excellent one that arrives late.
- ✗Better visualisation would have saved it. Presentation matters, but a beautiful chart delivered after the decision, or to somebody with no authority to change anything, still changes nothing.
- ✗If the analysis is correct, resistance to it is irrational. Resistance is often rational given how people are measured. Look at incentives and at who carries the cost of being wrong before assuming the objection is about method.
- ✗Once a model is deployed the work is finished. Deployment is where a different kind of failure begins, because inputs change, behaviour shifts, and an unmonitored model degrades quietly while still returning confident numbers.
Revision bullets
- •Most failed analytics is unused rather than incorrect
- •Common stalls: no decision owner, wrong timing, unfit data, no integration
- •Incentives can block a correct finding; check who is measured on what
- •Without measuring the action taken, the work cannot defend itself later
- •A deployed model degrades quietly unless somebody monitors it
Quick check
A model predicting equipment failure is accurate, but maintenance crews ignore it. Which investigation is most likely to find the cause?
A team wants to show its recommendation engine created value. Which approach gives the strongest argument at the next budget round?
Connected topics
More in What Business Analytics Is
Sources
- Sculley et al. (2015)Sculley, D., Holt, G., Golovin, D., Davydov, E., Phillips, T., Ebner, D., Chaudhary, V., Young, M., Crespo, J.-F., & Dennison, D. "Hidden Technical Debt in Machine Learning Systems." Advances in Neural Information Processing Systems 28, 2015.Shows the model as a small component inside a much larger system of data plumbing, configuration and monitoring, and documents the maintenance debt that surrounding system accumulates. It is evidence about production and maintenance cost rather than about the adoption, timing and incentive stalls described here.
- Davenport & Harris (2007)Davenport, T. H., & Harris, J. G. Competing on Analytics: The New Science of Winning. Harvard Business School Press, 2007.Argues that sponsorship, process integration and skills, rather than technique, separate organisations that act on analysis from those that do not.