The page for MS&E 237, Spring 2010 will be up in March 2010.
Looking for the notes for this course? All on the course wiki!
Want to listen to the lectures? Download the mp3 recordings and transcripts.
Data Mining and Electronic Business: The Social Data Revolution
Extract Insights from Twitter
In the last year, your location data and personal medical information have become the latest streams in the river of data, joining email, clicks, searches, social networking, and buying patterns. This course will dramatically change how you think about your data.
How can these data sources make our lives easier, more effective, more interesting? How can we get better recommendations, based on our behavior and the behavior of our friends? How can reputation systems help with decisions about who to trust?
Gathering, sharing, and storing data has become trivial. But what shall we collect, and what applications can we build that users really want?
Moving beyond graph and guess, push and pray, launch and learn, and so on, this course gives you tools and strategies for successful applications. How can you optimize virality and engagement, and spot weaknesses early? How can you entice users to interact with the app, and recommend it to their friends?
Each class is structured according to PHAME: define relevant Problems, invent Hypotheses, create Actions, design Metrics, and conduct Experiments. We also introduce a key driver to encourage users to provide critical data: Return on Personal Engagement (ROPE). Users who gain a benefit (tangible or psychological) from participating are far more likely to do so, and we discuss how to design incentives to encourage participation.
Course time is enriched by notable speakers, from notable companies. In addition to discussing applications that succeeded, we also discuss applications that failed, and try to distill out the reasons for success and failure. This course also includes highlights from two courses I developed and taught at Berkeley last year, including the popular"Marketing 2.x" at the Haas School of Business.
Data mining is no longer the process of digging through data morgues to uncover scraps of still-viable information. Success in the online marketplace now hinges on people and the data they create. E-business is no longer about selling books.
Assignments and Readings
Hands-on assignments include leveraging web analytics, applying geolocation, creating a recommender system for Twitter, and building a Facebook app.
Students are expected to actively engage in class discussions, to have their assumptions challenged, and to bring their diverse backgrounds to bear. After each class, a detailed write-up is created by the students as the course wiki (see 2009, 2008, 2007).
The reading material is very recent, originating from several academic disciplines. Besides statistics and computer science, it discusses modern marketing techniques, behavioral economics, social network analysis ideas and other concepts. For background reading, the following books might be useful:
After our crazy Data Safari, now in London. Will be back for the exciting 10/13 Berkeley class Identity and Trust with Reddit's amazing Steve Huffman. See you there! Mobile: +1 650 906-5906. Email:
Click on an airport to see when I'll be there : ASE : BKK : DFW : JHM : LHR : LON : MEX : MUC : NTE : OGG : PAO : SFO : SHA : SJC : UCB : YVR : ZRH : all