Social Media Observatory

Gain the technical skills to monitor millions of social media accounts for a panoramic view of online communities.

Social media has become our modern-day agora, the place where we get our news and engage in political discourse. With disinformation, hate speech, foreign interference, authoritarianism, market monopolization, and political polarization all on the rise, monitoring social media in the public interest has never been so difficult nor so urgent. Social Media Observatory is an in-depth beginner’s guide to coding full-stack web applications in Python to track and analyze social media activity at scale, giving you unparalleled insights into the cultural and political conversations of our time, then empowering you to reach your own conclusions.

You’ll start by coding a web app to monitor Russian disinformation channels discussing Ukraine on Telegram, then learn more adversarial methods to scrape the Wayback Machine, Instagram, and TikTok. You’ll detect Twitter bots following Saudi influencers, extract text-embedded URLs to reveal a white supremacist blogosphere, parse archived tweets to reveal the discourse leading up to the January 6 insurrection, and learn to circumvent Meta's and ByteDance's defenses to scrape Instagram and TikTok, collecting evidence of war crimes in the Middle East. Finally, you’ll write an API and deploy your observatory to a cloud server to collaborate with other investigators. By the end of the book, you’ll have a working website that allows collaborators to log in, request data collection, and perform analysis of collected data, via a point-and-click interface. 

Along the way, you’ll learn to:


Use Python and RabbitMQ to retrieve Telegram data from a remote API

Store data with PostgreSQL and query it with SQLAlchemy

Analyze the data you’ve retrieved with Pandas, NetworkX, Gephi, and other tools

Use state-of-the-art AI tools to transcribe audio and classify text

Deploy a website with an interactive dashboard using FastAPI, Dash, Cytoscape, Uvicorn, Gunicorn, Nginx, AWS, and Let’s Encrypt

Containerize your apps with Docker and provision infrastructure with Pulumi



Whether you’re a citizen scientist, a security researcher, a journalist, an activist, or a scholar, Social Media Observatory is an essential blueprint for building your window into online worlds.