Some background

Research interests

• How do people navigate political information in digital media environments?

Problems

• Social media use is hard to measure
• People are no good at recalling exposure to political media (Prior, 2010; Prior, 2015)

New data sources

• Real-time and digital-sources of data

Overview

• Current issues in quantitative research
• Rise of big data and data science
• Competing approaches to quantitative research

*quick note about organization

Issues in COMS research

• Generalizability of student samples
• Attrition in longitudinal [panel] studies
• Real-time/tracking equipment is expensive
• Limitations in self-report/questionnaire data
• Looming questions about replicability

"Replication crisis"

• Recent discovery that most attempts to reproduce previous scientific research fail.
• Originated in psychology, but applies to all social science.
• There are a lot of potential causes, but people like to agree on 2 in particular...

1. p-hacking

• p-hacking refers to systematic biases favoring significant (e.g., p < .05) results in analysis and peer review stages of research
• Result of p-hacking is literature with:
  • many reported false positives
  • few reported null [debunking] findings

2. Open data

• Replication crisis says more about documenting the scientific process than the validity of previous findings
• Leading cause of replication failure has been poor documentation
  • unpublished data
  • missing or misnamed variables
  • ommitted procedures

New challenges

• Fragmented media audiences and probability sampling
• Respondent fatigue/information overload
• Increasingly salient privacy concerns

Big data

• Big data refers to "datasets that are too large for traditional data-processing systems and that therefore require new echnologies" (Provost & Fawcett, 2013, p. 54)
• "Big data" isn't really that different from "small data"
  • It has rows, more columns, or more rows and more columns

Dealing with big data

• The phrase "big data" is useful as a heuristic for new digital sources of data
• New sources of data tend to be plentiful and less structured
• To deal with new digital sources of data, scholars have begun turning to data science

Data science

• Data science is "a set of fundamental principles that support and guide the principled extraction of information and knowledge from data" (Provost & Fawcett, 2013, p. 52)
• More practically, data science combines both quantitative statistics and computer science methods

Quantitative research

Data science

Better samples

• Convenience samples of students tend to be homogenous and limited in size
• Digital sources are not restricted by geography and can be massive
• In a few hours, one could, for example, stream a million tweets from the U.S.

Better samples

• Digital [online] sources of data can be gathered across space and time
• Scholars often criticize these samples because they "are not representative of the general population"
  • But people who live in glass houses shouldn't throw stones
• Internet activity correlates strongly with people activity

Bigger questions

• Big data and data science make it possible to conduct research on a larger and more dynamic scale
• For example, one could collect tweets about political candidates daily and analyze change in sentiment over time

Research questions

• Measurement error is not always a problem, but sometimes it is
• Data science makes it possible to examine phonomena without measurement error
• For example, people tend to overestimate the degree to which their social media networks are politically heterogeneous

User networks

• In my dissertation, I tracked user networks on Twitter during the 2016 election
• Republican group: users who followed one or more of the following:
  • Sean Hannity, Sarah Palin, Fox News Politics, and Drudge Report
• Democrat group: users who followed one or more of the following:
  • Rachel Maddow, Paul Krugman, HuffPost Politics, and Salon

Network analysis

• Network analysis makes it possible to more accurately create a visual depiction of network clusters
• The degree to which Twitter networks are organized by partisanship is especially striking when examined via network analysis

Big data baggage

• In addition to computational demands, big data creates new ethical concerns for researchers
• Data science is becoming the intersection of prediction-centered machine learning and theory-centered statistics (Vasant, 2013)
• To better understand these two approaches, it helps to compare the goals of statistics with the goals of computer science

Two approaches

Computer science examines whether a certain pattern can be found in data.

• e.g., hey Alexa, what is the capital of Chile?

Statistics examines whether data can be found in a certain pattern.

• e.g., lack of sleep correlates with decreased cognitive performance

Two approaches

• Statistics, more than computer science, is condusive to knowledge discovery (Vasant, 2013)
  • Computer science provides tools for summarizing and subsetting "databases"
  • Statistics provides tools for testing and creating theories with "data frames"

Competing models

• Quantitative statistics models are judged by their explanatory potential
• Machine learning models are statistical models judged solely by accuracy of their predictions
  • If we had 100 cases of data, then we would fit a model on 50 of those cases and test the model on the other 50 cases
• Predictive models do not explain what is happening
  • Impossible to know whether variables in model actually reflect some underlying construct (Vasant, 2013)

Machine learning problems

• Machine learning is agnostic toward theoretical considerations, which leads to at least 2 problems
  • Predictive models can be misinterpreted as explanatory, e.g., racial profiling
  • Algorithms can be manipulated (Verge, "Twitter taught Microsoft’s AI chatbot to be a racist asshole in less than a day", 2016)

Theory hangs in the balance

• Institutions will increasingly make decisions informed by data science
• The future of data science will be shaped by machine learning and statistics
• Communication researchers are uniquely suited to contribute to best practices of data science

Communication studies

• Communication studies is a natural fit for data science
• Digital communication means "messages" can be directly captured using computational methods
  • So the "data" in data science is communication
• Methodological diversity in communication studies can provide insights to both applied and theoretical understandings of big data and data science

References

Vasant, D. (2013). Data science and prediction. Communications of the ACM, 56, 64-73.

Provost, F., & Fawcett, T. (2013). Data science and its relationship to big data and data-driven decision making. Big Data, 1, 51-59.