The Dividends of Best Practices for Social Media Studies: ARF Interview with Jacqueline Anderson, Miriam Eckert, and Gina Pingitore of J. D. Power & Associates

Here at the ARF, we are very concerned with research quality, and social media research is no different. Conscientious researchers need to make sure their data is valid and reliable, but since social media is relatively new, we don't yet have a body of best practices to guide us. Enter

,

, and

of

We spoke with them about data quality in social media research and their efforts to establish best practices.

ARF: In your opinion, why is it crucial that we standardize practices for social media studies? What are some of the specific issues that you see affecting credibility of social media studies? How can they be addressed?

Anderson, Eckert, Pingitore:Right now social media research feels a bit like the "Wild West." With so many tools of differing caliber available to the public anyone can sit down and run a social media "query." In this environment, there is no standardization to the approach and the results vary greatly. Skeptics use this variation as a way to call the practice of social media into question. If anyone can get different results at any time why should companies rely on the results? By creating a set of best practices we can bring social media research to the next level, offering results that companies can really trust. Simple guidelines around how to build the queries and how to process the data can make the results more valid and reliable. These best practices will bring credence to this methodology, establishing it as a great addition to any researcher's toolkit.

ARF: Why are queries so important for developing social media studies? What are the challenges in developing queries and interpreting their results?

Anderson, Eckert, Pingitore:Queries are the basis of all social media research. The old adage "garbage in, garbage out" is a perfectly applicable. Queries tell social media engines what to look for among the billions of data points that exist in the social sphere. The queries you build will greatly impact the data that you pull back to analyze. Simple brand level queries can result in a lot of spam, irrelevant comments, or non-user generated comments. In these instances the post-query cleaning process becomes much more burdensome and the results become less reliable and valid. There is an art and science to developing a perfect query. You want to confine the search enough so that your results are accurate, but you also don't want to restrict your search too much so as to exclude potentially relevant data.

ARF:How can your findings help clients use social media more effectively?

Anderson, Eckert, Pingitore:Our findings will provide clients with some basic best practices that will help improve their social media research. Our examples will also give them information they can use to convince internal parties about the importance of taking the time to select the right tools and train properly when engaging in social media research. We often hear that clients aren't given the resources internally (both time and funding wise) to select the best tools and really learn how to garner quality data from social media. Then, when the data comes back and is all over the place, those same people who wouldn't dedicate resources to the process initially quickly dismiss social media research as a valid research methodology.

Our findings can also serve as guidelines clients can use when researching vendor partners. Using our findings they will be able to determine which service providers have actually taken the time to determine the science behind social media research.

Want to hear more? Jacqueline Anderson, Miriam Eckert, Gina Pingitore will be speaking at the ARF Re:think 2012 Convention on Tuesday, March 27. Register now!