Thursday, August 6, 2009

Netflix Contest Points to Statistical Future

A lot of these techniques will propagate across the internet.

- Lester Mackey, Ph.D. Candidate at the Statistical Artificial Intelligence Lab at UC Berkeley

In Steve Lohr, "Netflix Competitors Learn the Power of Teamwork," New York Times, July 27, 2009.

Netflex has since 2006 offered $1 million to the person or group that can improve its recommendation engine by at least 10%. A recommendation engine is a computing medel that predicts "what a person might enjoy based on statistical scoring of that person's stated preferences, past consumption patterns, and similar choices made by others." Netflix and other firms make these predictions based on what their customers do; Google offers search results and ads based on analyses of billions of clicks on its results. The contest is notable for encouraging the creation of teams of contestants with different types of statistical skills. The team with the best chance to win, for example, has members with a variety of skills who live in US, Canada, Austria and Israel.

Statistical evaluations of media audiences are on the upswing, and they will affect how companies and media firms think about us, and what they send us. The collection of such data often may also infringe on some people's notions of privacy.

No comments: