OpenWeb, using Jigsaw’s Perspective API, releases case study measuring the effects of real-time feedback and “nudges” in decreasing toxicity in online discussions.
The study involved nearly half a million comments and 50k users across sites including AOL, Salon, Newsweek, RT, and Sky Sports; demonstrates the ability of technology to positively affect the quality of conversation.
NEW YORK, September 21, 2020 /PRNewswire/ — OpenWeb, a leading audience engagement and conversation platform released the findings of an extensive study done in collaboration with Jigsaw’s Perspective API. Jigsaw is a unit within Google that forecasts and confronts emerging threats, creating future-defining research and technology to keep our world safer. Perspective API is a tool developed by Jigsaw that makes it easier to host better conversations, by using machine learning models to detect the potential toxicity of a comment.
The study measured the impact of deploying a “nudge” to potentially offensive or profane comments, encouraging commenters to reevaluate their message. The goal of the test was to measure if such a nudge could reduce toxicity within conversations. The study involved more than 400,000 comments across a select number of OpenWeb’s partner publishers, including AOL, Salon, RT, and Newsweek.
In addition to testing the effect of the “nudge,” OpenWeb created a variety of messages to measure how users react to different statements. The study then assessed the effect that responses to the nudge had on overall community behavior and toxicity.
Highlights from the study:
- 400,000 comments and 50k users were analyzed over 3 months, from May through July 2020
- 34% of commenters who received a nudge edited their comments. Of those who edited:
- 54% changed it to be immediately permissible
- 45% removed or replaced the toxic element
- 8% reshaped their entire comment
- 12.5% lift in civil and thoughtful comments being published overall
“The results of this case study are further evidence that technology can have an effect on human behavior,” said Nadav Shoval, CEO and co-founder of OpenWeb. “There are ways machines can learn and respond to behavior and create safer environments for us all, without suppressing speech. That’s what we’re working to achieve.”
“We’re thrilled to see more platforms using Perspective to help host better conversations online, especially platforms with such diverse and comprehensive audiences as OpenWeb. Machine learning has the potential to open up more spaces online for people to express themselves and interact with the community, but the technology is only as useful as the partners who implement it and bring it to users. This case study shows how Perspective can deliver measurable improvements in reducing toxicity. We’re proud to be working with OpenWeb to bring better conversations to millions of users across the web,” said Jared Cohen, Founder and CEO of Jigsaw.
“Ensuring that every member of the Salon community has an engaging experience with our features, videos, and our comments has always been a priority for us” said Mary Elizabeth Williams, Director of Community at Salon. “OpenWeb’s Real-time Feedback encourages commenters to contribute their ideas and opinions in a meaningful way. News today relies on sharing, and we want that experience to be honest, insightful, and far from toxic.”.
OpenWeb aims to improve the quality of online discourse through extensive, layered, moderation technology and proprietary AI, and runs Perspective ML models on millions of comments to catch abuse and toxicity in real-time. This Real-Time Feedback feature gives users an opportunity to change their message if it is suspected to break the Community Guidelines, using the nudge concept, a known theory in behavioral sciences that proposes positive reinforcement and indirect suggestions as ways to influence the behavior and decision making of groups or individuals.
“We believe encouraging good behavior is equally as important as removing the bad actors to support a thriving community,” said Nadav Shoval, CEO and co-founder of OpenWeb. “We want to inspire an open exchange of ideas across the web — with less hostility and toxicity – and are constantly experimenting with ways to encourage and develop this.”
While initially deployed as a test, due to the findings of the study, OpenWeb has begun rolling out the feature to its entire network of publishers. Read the complete case study here.
