NY Times & Jigsaw
Utilizing AI to foster healthy online conversations.
Overview
The New York Times knew that many readers of their publication found the comment section almost as valuable as the articles. However, they only had the ability to moderate comments for 10% of articles, forcing them to turn off the feature for the rest. I worked with Jigsaw, a subsidiary of Alphabet, to partner with the Times for a pilot program that would use AI to make moderation of comments much more efficient.
My role
Interviews with NY Times stakeholders
Product requirements and definition with Jigsaw engineers
UX design concepts for dashboard
Result
300% increase in the number of articles offering comments
The pilot program made the process of moderation so much more efficient that the New York Times opened 300% more stories for comments. The program has also been rolled out to other major content publishers such as the Wall Street Journal, OpenWeb, Reddit, Financial Times and many others.
User Testing
Finding the right workflow
Moderators were typically journalists early in their career, so increasing moderation speed and efficiency would not only help the New York Times maintain a vibrant community, but also help our end users focus on writing more stories. After studying their workflow it was clear that the best way to support them was to sort comments into batches based on perceived toxicity. Human intervention could then be focused on the gray areas where the AI couldn’t make a definitive decision.
UX Design
Balancing AI and human intervention
The dashboard needed to strike a balance between allowing the AI to rapidly increase the rate of moderation while giving moderators an understanding of what was happening and the agency to override automated decisions. The design puts the moderator in control by making clear each decision that is being made by the AI with a chance to review it as necessary.