The quality of market research reports seems to have been a persistent meme throughout my life in market research. In the 1990s, the Advertising Research Foundation (ARF) conducted research with CEOs that showed that they valued information from market research lower than any other source (e.g., marketing, human resources, etc).
Although there are many anecdotes from clients and agencies, how big an issue is this? And given the improvements in presentation and visualization software, and the development of the insights industry, is this still true or have things improved?
In March 2019, NewMR were kind enough to help Mike Sherman and myself conduct an online survey to update our knowledge of report quality. A sample of more than 500 participants included clients, agencies and other service providers in the market research community, including both those who create and write reports and those who receive and use them.
The results show a clear divide between those who use reports and those who create them, with a 35% gap in perceptions between those who receive reports (users) versus those who create reports (writers). Report creators think that 65% of their reports are ‘Very good’ or ‘Excellent’ compared with 30% of report receivers.
We were surprised at the size of this gap. One of the report creators who took part in the survey summarized one of the main issues very pithily, saying their most recent report was “readable” but not “actionable”.
The most common complaints about reports were that many are too long, with too much data and not enough insights. This may be a bit unfair, as the insights are often in the report, buried in a sea of less relevant information. As one report receiver from the supply side of market research put it, “the report contained the survey results” but “tried to cram in too much info, which made it very confusing to pull out the insights and actions”.
This is a long-standing problem in market research, and the solution is to focus on writing reports that are end result focused and concise. This is something that both creators and receivers of reports agree on, with ‘Insightful’ chosen by almost all survey participants as the most important attribute for an ‘Excellent’ report. ‘Concise’ and ‘Practical’ were both chosen by more than half of participants. Other attributes such as ‘Objective’, ‘Rigorous’ and ‘Detailed’ were considered less important.
So how to achieve this? End results focus comes from synthesizing findings and identifying what they mean for the client and their business objectives. Put another way, the first thing a report should do is answer the business question. Business objectives should not be confused with research objectives in writing a report.
Making a report concise is best achieved by focusing on the end result before writing a report. The best way to do this is to work out an ‘Elevator speech’ before you start, representing the 30 seconds (or less) you might spend in the lift with a client without your PowerPoint deck. How would you summarize your findings to give a clear answer to the business question?
With an elevator speech prepared it is then quite easy to write a one-page (and one page only) summary of findings in which you expand on the two or three main points in an elevator speech to provide more detail. These main points then become the backbone of a concise 15-25 page report which adds the research evidence to support the key points. This is all you really need, along with an extensive appendix which includes everything else from the research that is not relevant to the answer (i.e. all the other tables, questionnaire, proposal, etc).
Most importantly, this is what most clients want based on my experience and on their feedback in the survey. They want nice visualizations of data too. However, a clear succinct answer is the key to making them happier. While the survey provided evidence of some improvements in report quality, it makes it clear that there is still much for all of us to do to improve.
[This is a written version of a presentation with Mike Sherman for NewMR on 30 April 2019. You can access our presentation along with a more detailed report of the survey findings here. You can learn more about my co-presenter Mike Sherman here.]