Can we instill trust in the ratings industry?

The recent fiasco around manipulation of target rating points (TRPs) once again brings to attention the importance of ratings and the agencies who collect, process, and provide that information in today’s business ecosystem. Details are limited but what we know from media coverage is that a few media houses hired people to provide ratings that would have boosted their TRPs. TRPs are important for channels as significant revenue (advertising) is tied to TRPs. Currently, in India, Broadcast and Audience Research Council (BARC) is the only firm responsible for collecting and sharing this information.

BARC collects this data by employing a panel of respondents across different cities who regularly input their preference for different channels (most of this preference is collected automatically and electronically through a customised set-top box with regular audits done by BARC). While it is possible that the data collected by BARC has margins of error, we are reasonably confident that if BARC uses the latest industry standards to investigate its TRP measurements, it will reveal little or no effect on the overall ratings.

BARC, for its part, has rightfully stopped its TRPs in certain areas and is conducting an investigation on its end. Due to the ongoing investigation, we will not comment anymore on this issue. It is also important to also clarify that our statements have nothing to bear on issues pertaining to legality; just that the ratings system might well be robust enough to handle a few corrupt actors.

While, we don’t want to comment on the current issue, we do wish to point out that such controversies are bound to occur regularly given the large monetary gains tied to TRPs. Also noteworthy is that such events are not restricted to the media industry. During the financial crisis, credit rating agencies were under increased scrutiny for the ratings they provided and the role of these ratings on the financial collapse. Similarly, Amazon, Flipkart, and other online platforms routinely deal with issues of fake reviews and ratings. While each isolated event may be pretty short-lived and local in its impact, the quotidian nature of these events shake our trust in these rating systems.

So, how do we improve trust and reputation of these institutions and rating systems? We could bring in competition and hope for market forces and competition to solve some of these issues. However, a reasonable argument can be made for a regulated monopoly, especially in this instance, as there could be large upfront investments, standardisation etc. that may be required to build the infrastructure needed to deliver the desired performance. How then do we resolve the issue with a regulated monopoly? This is where we think we can bring some novel ideas to the fore.

We propose setting up an academy-industry collaboration through which BARC is required to share historical data (after anonymising the data and taking all necessary safeguards on privacy) with select academic institutions and scholars for scrutiny and research. Note here that we are only asking for historical data (maybe data from a year ago which has less industry relevance and lower economic value). The institution or centre that is responsible can require that access can be limited and impose other legal protections, but the primary objective driving this approach should be to ensure access of the data to a variety of academic scholars for academic research.

The advantages of this approach are the following: First, it improves credibility in the system as making the data accessible to inquisitive researchers ensures that any flaws in the system will be identified. Many innovative and progressive companies do some version of this where the wisdom of experts is leveraged. Second, it improves innovation and development of new strategies and measures. If we want to develop “Research in India” and be ready for the next technology leap, we must enable our scientists, researchers, and students to work on data sources. There is no reason why an inquisitive student from Indian Statistical Institute (ISI) cannot come up with better ways of sampling and measurement when provided with access to this data.

Similarly, PhD students at the Indian School of Business (ISB), Indian Institutes of Management (IIM), and other premier institutes may come up with a robust strategy to use this data and provide novel insights to advertising and media companies. If data is the future, we must enable our students to get better at using data and developing models that can lead the way on thinking. Finally, the act of disclosure in itself puts pressure on firms to be more efficient with their systems and process.

While what we are suggesting may sound radical in the Indian context, it is not entirely new. AC Nielsen, which is well regarded for its retail consumer panels in the FMCG industry, in association with KILTS Center for Marketing at the University of Chicago, runs a robust program where retail and panel data is routinely shared with select academic researchers. The research impact has been phenomenal. The number of papers based on this initiative is currently around 200 and has more than 43,205 downloads as per SSRN. Much closer to home and more recently, the NSE is providing access to trading data as per their data policies to researchers at ISB (and other recipients as well) to enable research in Indian financial markets.

In sum, our proposal for rating agencies and marketing research firms to collaborate with academics and academic institutions for research by sharing data is the way forward to improve trust and transparency in our institutions and to develop a strong research talent pool in India. We hope that this article can make it happen.

The authors are faculty at the Indian School of Business (ISB).