Making of
fbreactions.io
港台媒體FB心情數據分析
Data Analysis on Facebook Reactions of
Hong Kong vs Taiwan Media pages
API Released
for FB Reactions
(@F8, May, 2016)
抓高流量媒體FB數據
Scrapping Facebook Pages of Media
蘋果日報 東森新聞 ETtoday新聞雲 自由時報 快點報報 The News Lens 關鍵評論網 udn.com 聯合新聞網
蘋果日報 South China Morning Post 香港獨立媒體網 熱血時報 明報即時新聞 信報財經新聞
香港經濟日報 am730 香港01 頭條日報 852郵報 《都市日報》
算流量基本上都是蘋果
時事情緒指數
香港 vs 台灣
an Index on HK vs TW's Emotion
發現 Findings
台灣心情比香港好?
台灣人不太用心情功能?
每天都好像很不開心?
Taiwan has a better mood?
Taiwan folks don't use reaction much?
Angry everyday?
Architecture
Scrapping Problems
FB API as bottleneck
Even scraping for all pages' latest posts
No /_changes feed nor time-cursor at FB API
Effective scraping: less for old posts
Architecture
Post Ids
/page
/reactions
/analyze
Aggregated
Reactions by Post
analyzed
result
Redis
API
Why AWS Lambda
Simple Enough
Stateless
Easy to Scale up / Down
Easy to backfilling
Limited Capacity Required
dumb API
Design
I'm not a graphic designer....
Telling story with the
Emojis
X over-interpretation
X reason about detailed figures
d3.Js Force Layout
simple rules but beautiful emergent behaviour
caveat:
deal with boundaries and indifferent size
visually dynamic has huge effect
Need of
Conceptual Model
Association
Identify the relationship
(even when drunk)
VS
Needa..
highlight the story
start a conversation
Responsive is hard...
especially at Data Viz...
Hard to transpose
the table
Hack: overflow-x:scroll;
Even Better: blur to suggest scrollable
d3.Js is Scalable Vector Graphics (SVG)
What's Next?
TODO
-
2D clustering and visualization
-
Tune for better API throughput
-
Real-time streaming
- IA
-
Quote Plugin
- Semantic Url by Date
-