Introduction to Social Media Research 16 October 2013
Annie Pettit, Peanut Labs This webinar was broadcast on 16 October, 2013, as part of the NewMR Lecture Series. You can access the other lectures, and the materials from other events via NewMR’s Play Again http://newmr.org/play-again/
All copyright owned by The Future Place and the presenters of the material For more information about NewMR events visit http://newmr.org Speaker: Annie Pettit, Peanut Labs NewMR Lecture Series 2013 – Introduction to Social Media Research
An Introduction to Social Media Research By Annie Pettit Chief Research Officer, Peanut Labs @LoveStats @PeanutLabsMR
Speaker: Annie Pettit, Peanut Labs NewMR Lecture Series 2013 – Introduction to Social Media Research
More tools in our toolbox
MROCs Surveys Ethnography Mobile qualitative Online focus groups Social media research Speaker: Annie Pettit, Peanut Labs NewMR Lecture Series 2013 – Introduction to Social Media Research
Social media listening research is NOT • • • • •
Online surveys Online focus groups Online communities Facebook polls Sample from social networks
Speaker: Annie Pettit, Peanut Labs NewMR Lecture Series 2013 – Introduction to Social Media Research
What is social media listening research? • Market research using naturally occurring social media conversations as the data source
• Distinctive because: • NOT permission based • NOT elicited • Massive potential for individual harm
Speaker: Annie Pettit, Peanut Labs NewMR Lecture Series 2013 – Introduction to Social Media Research
What is individual harm?
@UserName If my rabbi finds out I just ate pork, I’m in huge trouble @UserName I’m doing crap all at work today #FreeMoney
@UserName If anyone at school says anything about me today, I will finally kill myself @UserName I hate pretending to be happy. &^%& %#&@ $*@& Speaker: Annie Pettit, Peanut Labs NewMR Lecture Series 2013 – Introduction to Social Media Research
Be knowledgeable about ethics guidelines • Separate guidelines developed by MRA, ESOMAR, CASRO • Essence is the same as traditional guidelines • Respect for privacy
• Be aware of implications re contacting/sharing without permission •
http://www.mra-net.org/rq/documents/MRA_IMRO_SMR16.pdf
•
http://www.casro.org/pdfs/1011/Social_Media_Research_Guidelines.pdf
•
http://www.esomar.org/uploads/public/knowledge-and-standards/codesand-guidelines/ESOMAR_Guideline-on-Social-Media.pdf
Speaker: Annie Pettit, Peanut Labs NewMR Lecture Series 2013 – Introduction to Social Media Research
Social Media Research Method
Create search Clean Search Crawl Internet Variable Cleaning Content Analysis Sentiment Scoring Sample Weight Analyze Results Speaker: Annie Pettit, Peanut Labs NewMR Lecture Series 2013 – Introduction to Social Media Research
Limitations
• All research methods have limitations • Surveys are difficult to write and may include poorly worded questions and answers • Focus groups employ small sample sizes of carefully selected people whose opinions might not generalize outside the group • Mobile surveys need to be short and simple
• Social media research also has limitations
Speaker: Annie Pettit, Peanut Labs NewMR Lecture Series 2013 – Introduction to Social Media Research
Cannot identify geo/demographics
Female, White, Not Hispanic, Canadian
Male, White, Not Hispanic, Belgium
Female, White, Not Hispanic, American
Speaker: Annie Pettit, Peanut Labs NewMR Lecture Series 2013 – Introduction to Social Media Research
Hard for unknown companies to research their own brand • • • • • • • • • • • •
Gazprom McKesson Sinopec AXA Paribas ThyssenKrupp State Grid Dexia Group Sysco Repsol YPF BNP Fortis
Speaker: Annie Pettit, Peanut Labs NewMR Lecture Series 2013 – Introduction to Social Media Research
Hard for untalked about companies to research their own brand • Procter & Gamble
I prefer to chat about Olay
• Dow Chemical
I prefer NOT to chat about Dow
• Pfizer
I prefer to chat about Neosporin
• Unilever
I prefer to chat about Lipton
• Siemens
What exactly do they make?
Speaker: Annie Pettit, Peanut Labs NewMR Lecture Series 2013 – Introduction to Social Media Research
Harder for local/small businesses to research their own brand via quant •
Annie’s Bakery
•
Joe’s Corner Store
•
Little Rock Tackle Shop
•
Lexington Spa and Nails
•
Jacksonville Coffee Stop
•
Rockland Best Ice Cream Shoppe
Speaker: Annie Pettit, Peanut Labs NewMR Lecture Series 2013 – Introduction to Social Media Research
Large consumer facing companies can measure anything • • • • • • • • • •
Burger King WalMart Fruit Loops Pepsi Kroger Nike Microsoft Toyota iPad Pringles Speaker: Annie Pettit, Peanut Labs NewMR Lecture Series 2013 – Introduction to Social Media Research
SMR cannot measure incidence nor awareness • People may not be aware of the brand • People who ARE aware aren’t chatting in social media • People may not feel like talking about the brand • People may not feel like talking about ANY brands
Speaker: Annie Pettit, Peanut Labs NewMR Lecture Series 2013 – Introduction to Social Media Research
The Good Stuff: Time travel!
Speaker: Annie Pettit, Peanut Labs NewMR Lecture Series 2013 – Introduction to Social Media Research
Speaker: Annie Pettit, Peanut Labs
NewMR Lecture Series 2013 – Introduction to Social Media Research 16-Feb-12
14-Feb-12
12-Feb-12
10-Feb-12
08-Feb-12
06-Feb-12
04-Feb-12
02-Feb-12
31-Jan-12
29-Jan-12
27-Jan-12
25-Jan-12
23-Jan-12
21-Jan-12
19-Jan-12
17-Jan-12
15-Jan-12
13-Jan-12
11-Jan-12
09-Jan-12
07-Jan-12
05-Jan-12
03-Jan-12
01-Jan-12
The Good Stuff: Daily Tracking 5.00
4.50
4.00
3.50
3.00
2.50
2.00
1.50
1.00
The Good Stuff: Thousands of Variables Strongly Disagree
Somewhat Disagree
Neutral
Somewhat Agree
Strongly Agree
Honest Reliable Creative Determined
Fashionable Kind Funny Classy Confident Sociable Energetic Intelligent
Speaker: Annie Pettit, Peanut Labs NewMR Lecture Series 2013 – Introduction to Social Media Research
Finding the needle in the haystack
Speaker: Annie Pettit, Peanut Labs NewMR Lecture Series 2013 – Introduction to Social Media Research
This is the internet
Sales Pitches
Spam
Real People
Business Discussions
Speaker: Annie Pettit, Peanut Labs NewMR Lecture Series 2013 – Introduction to Social Media Research
This is spam • Adderall 20/30mg IR/XR online without prescription.Contact now if interested at
[email protected]. • ambien without prescription ambien online paypal buy ambien no rx online • buy ambien without prescription buy ambien bulk buy ambien online no rx • buy ambien without prescription buy ambien in uk buy ambien tijuana • buy ambien without prescription buy ambien online fast shipping generic ambien safe
• http://t.co/nQ3E8MAt Buy Eprex online - Generic Eprex legally - Purchase Eprex without script • zithromax cost buy canadian zithromax prescription online and antibiotics that treat stds Speaker: Annie Pettit, Peanut Labs NewMR Lecture Series 2013 – Introduction to Social Media Research
This isn’t Spam RT @AriesTerms: #Aries are the sexual energizer bunnies. They keep going, & going, & going...
http://twitter.com/personsname1/status/2177 75548674486273
RT @AriesTerms: #Aries are the sexual energizer bunnies. They keep going, & going, & going...
http://twitter.com/personsname2/status/2177 75584892293120
RT @AriesTerms: #Aries are the sexual energizer bunnies. They keep going, & going, & going...
http://twitter.com/personsname3/status/2177 75625186967554
RT @AriesTerms: #Aries are the sexual energizer bunnies. They keep going, & going, & going...
http://twitter.com/personsname4/status/2177 75643876786176
RT @AriesTerms: #Aries are the sexual energizer bunnies. They keep going, & going, & going...
http://twitter.com/cpersonsname5/status/217 775684737695744
RT @AriesTerms: #Aries are the sexual energizer bunnies. They keep going, & going, & going...
http://twitter.com/personsname6/status/2177 81225044193280
22
Speaker: Annie Pettit, Peanut Labs NewMR Lecture Series 2013 – Introduction to Social Media Research
These are sales pitches • •
• • • • •
23
This is an excellent movie, I saw it online for free in HD at: watchmovies4free {.} org This is a very cool film, I saw it online for free in HD here: viewfilmsfree {.} com This is a great movie, I downloaded it online for free here: VIEWFILMSFREE {.} COM This is an excellent movie, I watched it online for free here: WATCHMOVIES4FREE {.} ORG "Xbox 360 Messenger Kit (Chatpad & headset), Only 10 @ HMV http://t.co/vSJK2VUH #deal" "New comedy DVDs and Blu-rays at HMV http://t.co/XjsH130e #deal" "#Offers #Deals Amazing Christmas deals at HMV Blackadder Remastered: Ultimate Edition: 6 DVD 17.99 and fr... http://bit.ly/eS2GZd" Speaker: Annie Pettit, Peanut Labs NewMR Lecture Series 2013 – Introduction to Social Media Research
These are not sales pitches • • • • • • • •
I unlocked the The Glee Project: Sexuality sticker on @GetGlue! http://t.co/65a7aB6E I unlocked the The Glee Project: Sexuality sticker on @GetGlue! http://t.co/756xG3mR I unlocked the The Glee Project: Sexuality sticker on @GetGlue! http://t.co/8FFS3Xzj I unlocked the The Glee Project: Sexuality sticker on @GetGlue! http://t.co/9i9OZG1i I unlocked the The Glee Project: Sexuality sticker on @GetGlue! http://t.co/aa1Xbb1F I unlocked the The Glee Project: Sexuality sticker on @GetGlue! http://t.co/AmAgrCL1 I unlocked the The Glee Project: Sexuality sticker on @GetGlue! http://t.co/BKCM29b6 I unlocked the The Glee Project: Sexuality sticker on @GetGlue! http://t.co/D8wLvPzR
•
"Be grateful if you used my referral link for this deal http://share.livingsocial.com/deals/...4&rui=95087725" "http://www.hotukdeals.com/deals/batt.../817097?page=4 stock on order but hukd seems to think its a very hot deal" "its part of the 5 for 10 pounds universal 100 deal on hmv http://hmv.com/hmvweb/displayProduct...RODUCT-620866#" "Picked this up at HMV though it was a steal. http://yfrog.com/h22plaj Still can't find a Kinect Sports deal tho"
• • •
24
Speaker: Annie Pettit, Peanut Labs NewMR Lecture Series 2013 – Introduction to Social Media Research
These are business discussions • • • • • • •
25
"HMV: what the analysts say - # Telegraph ow.ly/84WHA # HMV" "UK: HMVs New Loan Facilities Have Retailer Over A Barrel, Says Analyst http://goo.gl/fb/pzKi9 2:33 AM Jun 8th via Google" "#HMV announces another profit warning, #VerdictRetail analyst comment http://bit.ly/fD301z @ hmvtweets @ retailfocus 4:14 AM Apr 6th via bitly" "HMV warns on covenants as debt climbs: Embattled music and entertainment retailer says it will miss analysts... http://on.ft.com/efK5cu 3:56 AM Mar 1st via twitterfeed" "HMV: What the analysts say: Today's grim profits warning from HMV Group has reignited debate over the comp... http://tinyurl.com/2cdozec 2:20 AM Jan 5th via twitterfeed" "HMV struggles to survive amid tumbling sales: With more than 400 stores across the country, industry analysts sa... http://bit.ly/dVdgHY 12:27 PM Dec 9th via twitterfeed" "Jan. 10 (Bloomberg) -- Natalie Berg, an analyst at Planet Retail Ltd., discusses holiday sales reports by Debenhams Plc, Marks & Spencer Group Plc and HMV Group Plc, and the outlook for retailers. She speaks with Linzie Janis and Mark Barton on..."
Speaker: Annie Pettit, Peanut Labs NewMR Lecture Series 2013 – Introduction to Social Media Research
This is hidden gold • CaDbury ;p tehehe !!! whch one z ur fav ?eish i lyk them all • On a fyn weathr lyk ths......smbdy z cravin 4 a Cadbury chocolate • Neva eva lyk neva chew cadbury swt infrnt of ur man or any otha cute fellow! Nt cute at al! • My favorite player Randy Moss mite be makin ah cum back to tha NFL • wtf would i say sheesh? "o sry bout ur balls dude“ • I shlda jus taken the credit smh ;) • dnt giv a faak boy...they will do what ever to get a follower; twar, steal tweets, diss a country, screenmunch, dey r relentles • Slick talkn , sneek diss n, all that ole under the table @!! Ish,, bring it 2 a brother face!!
26
Speaker: Annie Pettit, Peanut Labs NewMR Lecture Series 2013 – Introduction to Social Media Research
These are always consumers
27
Speaker: Annie Pettit, Peanut Labs NewMR Lecture Series 2013 – Introduction to Social Media Research
Data quality is top priority • Every research method has data quality processes • Seek out and fix leading questions • Seek out and recruit the right target not the most convenient target • Seek out and shorten overly long surveys
• SMR begins with quality extraction
Speaker: Annie Pettit, Peanut Labs NewMR Lecture Series 2013 – Introduction to Social Media Research
What does HFCS stand for?
Speaker: Annie Pettit, Peanut Labs NewMR Lecture Series 2013 – Introduction to Social Media Research
What does HFCS stand for?
Speaker: Annie Pettit, Peanut Labs NewMR Lecture Series 2013 – Introduction to Social Media Research
What does BP Stand For?
Speaker: Annie Pettit, Peanut Labs NewMR Lecture Series 2013 – Introduction to Social Media Research
What does BP Stand For?
Speaker: Annie Pettit, Peanut Labs NewMR Lecture Series 2013 – Introduction to Social Media Research
Acronyms need special care
Speaker: Annie Pettit, Peanut Labs NewMR Lecture Series 2013 – Introduction to Social Media Research
Quality in sample sources • Traditional research sample data sources • Are you interested in opinions from women • Are you interested in opinions from younger people • Are you interested in opinions from lower income people
• SMR samples data sources
Speaker: Annie Pettit, Peanut Labs NewMR Lecture Series 2013 – Introduction to Social Media Research
Social Media Sampling Matrix
Speaker: Annie Pettit, Peanut Labs NewMR Lecture Series 2013 – Introduction to Social Media Research
Same brand, different result 100%
75%
50%
25%
0% Walmart Adidas
BP
iPod
Kindle Fire
Social Networks - facebook.com
Kroger Poptarts Sam's Tablet - Tiger Club iPad 2 Woods
Video - YouTube.com
Top 2 Box (Positive) Speaker: Annie Pettit, Peanut Labs NewMR Lecture Series 2013 – Introduction to Social Media Research
Quality in Coding • Traditional research organizes content • Q1: Are you a man or woman? • Q12: How likely are you to purchase coffee? • Q15: Which of the following brands do you use?
• SMR organizes content
Speaker: Annie Pettit, Peanut Labs NewMR Lecture Series 2013 – Introduction to Social Media Research
SMR Variables
in my college, people voted against this strike... wonder if their tuition fees gonna be paid in cash or cards.
Strike Schooling
Voting
Speaker: Annie Pettit, Peanut Labs NewMR Lecture Series 2013 – Introduction to Social Media Research
Content Analysis Validity
Beefy Heavy Weighs a ton Pounds sterling Pound puppy Tons of stuff Speaker: Annie Pettit, Peanut Labs NewMR Lecture Series 2013 – Introduction to Social Media Research
Content Analysis Validity
Blue ray Blue tooth Bobbi Brown Charlie Brown Fifty Shades of Grey Grey Goose
Speaker: Annie Pettit, Peanut Labs NewMR Lecture Series 2013 – Introduction to Social Media Research
Quality in Sentiment Analysis • Ethnographies are scored by researchers • How long can you run a country by holding all the poor provinces hostage and saying "were gonna cut out your equalizations unless you do what we say". cuts are gonna be made, people will be unhappy when provinces cut jobs education and you're gonna have a hard time getting re-elected because you were too busy implementing your nonsense.
• SM data are scored by researchers • “I would totally buy that” • “No way in hell am I buying that crap”
Speaker: Annie Pettit, Peanut Labs NewMR Lecture Series 2013 – Introduction to Social Media Research
Automated sentiment analysis requires care • This fabulous movie delicately portrayed the horrible atrocities of war, including terrifying images of torture and rape
• This fabulous movie delicately portrayed the horrible atrocities of war, including terrifying images of torture and rape
Speaker: Annie Pettit, Peanut Labs NewMR Lecture Series 2013 – Introduction to Social Media Research
Sentiment analysis is never perfect
New emoticons New acronyms New slang Sarcasm Strange typos Strange grammar Descriptions vs Sentiment Human scoring is only 85% accurate. Machine scoring strives for 75% accuracy. Speaker: Annie Pettit, Peanut Labs NewMR Lecture Series 2013 – Introduction to Social Media Research
Use of emoticons: How smart are you?
:) :(
:o) =] =) :^)
Speaker: Annie Pettit, Peanut Labs NewMR Lecture Series 2013 – Introduction to Social Media Research
Use of emoticons: How smart are you?
:)
;)
d^_^b
m9(^Д^)
:(
:|
(¬_¬)
(`･ω･´)
:o)
>;]
d(>w<)b
＿|￣|○
=]
>:X
(,_,)
(｀-´)>
=)
\,,/
( °٢° )
(´；ω；`)
:^)
\m/
(V)!_!(V)
ヽ(`Д´)ﾉ
Speaker: Annie Pettit, Peanut Labs NewMR Lecture Series 2013 – Introduction to Social Media Research
Use of acronyms
LOL FTW
WFT ROFL SMH
Speaker: Annie Pettit, Peanut Labs NewMR Lecture Series 2013 – Introduction to Social Media Research
Use of acronyms: Still feeling smart?
LOL
2U2
AWHFY
JK
FTW
2l8
IMHO
KMA
WFT
2mr
BION
OMG
ROFL
4ever
CUL
POS
SMH
2C4U
FCFS
RTF
Speaker: Annie Pettit, Peanut Labs NewMR Lecture Series 2013 – Introduction to Social Media Research
Use of Slang
Sick Whack
Dope Mafan Crush
Speaker: Annie Pettit, Peanut Labs NewMR Lecture Series 2013 – Introduction to Social Media Research
Use of Slang: Feeling inadequate now?
Sick
Clutch
Bomb
Sup
Whack
Koota
Burned
Pwn
Dope
Slub
Dough
Kicks
Mafan
Scrub
Aiight
Dat
Crush
Wassock
Krunk
Lekker
Speaker: Annie Pettit, Peanut Labs NewMR Lecture Series 2013 – Introduction to Social Media Research
How non-researchers validate scoring Verbatim
Step 1: Auto Score
Step 2: Human Validation
What a horrible thing to happen. Can’t believe people died on a costa cruise.
Negative
Agree
What a joke. Costa is giving people a big discount after the whole sinking ship thing.
Positive
Disagree
We had our honeymoon on the costa concordia. It was wonderful.
Positive
Agree
I hope the costa concordia captain goes to hell.
Negative
Agree
Plus 996 other evaluations…
The “yeah, yeah” method Speaker: Annie Pettit, Peanut Labs NewMR Lecture Series 2013 – Introduction to Social Media Research
How researchers validate scoring Verbatim
Step 1: Blind Score
Step 2: Blind Score
Step 3: Concurrence
What a horrible thing to happen. Can’t believe people died on a costa cruise.
Negative
Negative
Yes
What a joke. Costa is giving people a big discount after the whole sinking ship thing.
Negative
Positive
No
We had our honeymoon on the costa concordia. It was wonderful.
Positive
Positive
Yes
I hope the costa concordia captain goes to hell.
Negative
Negative
Yes
Plus 996 other evaluations…
The Correct Blind Method Speaker: Annie Pettit, Peanut Labs NewMR Lecture Series 2013 – Introduction to Social Media Research
Words in context are much easier? 1.
I hate hate HATE grocery shopping at Walmart!
2.
beautiful. loved it. amazing, keep up the excellent work!
3.
Im eating a big mac meal as i type , and it's fkin lovely
4.
just ate mcds...i feel ten pounds heavier..its a love hate
5.
I love Rebecca Black
6.
Yeah right. Keep arguing, at least you look smarter
7.
Bouta crush this big mac...
8.
dirk is just too clutch its scary
9.
the black eyeliner is the bomb
10. expect 400% from you dude--no excuses. >;p
Speaker: Annie Pettit, Peanut Labs NewMR Lecture Series 2013 – Introduction to Social Media Research
Is social media data normally distributed? 1500
8000 3000 6000
1000
2000
500
4000
1000
0
2000
0
1
1.5
2
2.5
3
3.5
4
4.5
5
Junior Seau
0
1
1.5
2
2.5
3
3.5
4
4.5
5
Fuel Prices
1
1200
6000
4000
800
4000
2000
400
2000
0
0
0
1.5
2
2.5
3
3.5
4
4.5
Nissan Rogue
5
1
1.5
2
2.5
3
3.5
4
4.5
5
Toronto Transit Commission
1
1200
2250
2000
800
1500
1000
400
750
0
0
0
1.6
2.1
Neiman Marcus
53
2.6
3.1
3.6
4.1
4.6
1
1.5
Sephora
2.5
3
3.5
4
4.5
5
2
2.5
3
3.5
1.5
2
2.5
3
3.5
4
4.5
5
4
4.5
5
Toilet Paper
3000
1
2
Bullying
6000
1
1.5
4
4.5
5
1
1.5
2
2.5
3
3.5
Cake Boss
Speaker: Annie Pettit, Peanut Labs NewMR Lecture Series 2013 – Introduction to Social Media Research
Branded Social Media Data is (Mostly) Normally Distributed (n=60 million) 12000
10000
8000
6000
4000
2000
0
-100
-89
-79
Extremely Negative 54
-69
-59
-49
-39
-29
-19
-9
1
11
21
31
41
51
61
Neutral
71
81
91
Extremely Positive
Speaker: Annie Pettit, Peanut Labs NewMR Lecture Series 2013 – Introduction to Social Media Research
What CAN’T we do with the data? • • • •
Measure census rep(representative) Measure frequencies and quantities precisely Measure geographics Ask follow up questions • But we can do so much…
55
Speaker: Annie Pettit, Peanut Labs NewMR Lecture Series 2013 – Introduction to Social Media Research
What other brands should you ask about? Identify the other top of mind brands that come up during conversations of your target topic. Make sure you do follow up research with them. Make sure they are included on your survey
Speaker: Annie Pettit, Peanut Labs NewMR Lecture Series 2013 – Introduction to Social Media Research
Which celebrity should endorse the Blackberry? These are the celebrities YOUR consumers want to talk about. These celebrities are already meaningful. Are they the RIGHT celebrities?
57
Speaker: Annie Pettit, Peanut Labs NewMR Lecture Series 2013 – Introduction to Social Media Research
Find the positives Do more of these. Don’t get rid of these. Brag about these.
58
Speaker: Annie Pettit, Peanut Labs NewMR Lecture Series 2013 – Introduction to Social Media Research
Find the negatives Solve these problems. Make sure consumers know you’re working on this.
59
Speaker: Annie Pettit, Peanut Labs NewMR Lecture Series 2013 – Introduction to Social Media Research
Find the most common topic Which issue is top of mind for your consumers? Which issue should you address first? Which great idea should you brag about the most?
60
Speaker: Annie Pettit, Peanut Labs NewMR Lecture Series 2013 – Introduction to Social Media Research
Find the least common topic Is a problem about to explode? Are your consumers unaware of or uninterested in your bragging points?
61
Speaker: Annie Pettit, Peanut Labs NewMR Lecture Series 2013 – Introduction to Social Media Research
Thank you!
http://ca.linkedin.com/in/anniepettit http://lovestats.wordpress.com/
62
https://www.facebook.com/anniepettit http://twitter.com/lovestats
Speaker: Annie Pettit, Peanut Labs NewMR Lecture Series 2013 – Introduction to Social Media Research
Thank you
Annie Pettit Peanut Labs Speaker: Annie Pettit, Peanut Labs NewMR Lecture Series 2013 – Introduction to Social Media Research
