Polisis: AI-based framework for analyzing privacy policies in real time

It has been known for a while that the overwhelming majority of Internet users doesn’t read privacy policies and terms of service before agreeing to them. Those few that do usually skim over them.

That’s mostly because these documents and agreements are extremely long and – intentionally or unintentionally – written in a way that makes them unintelligible to the great majority of users.

Companies’ privacy policies and terms of service also change through time, and often in a direction that’s unwelcome to many users. But most users simply don’t have the time to peruse these compendiums each time a change is made, even if they are able to understand the legalese in which they are written.

Shortcuts to knowledge

There have been several efforts to decipher these documents and present the most crucial information to users in a way they can understand it without having to waste too much of their precious time.

TOS;DR (Terms of Service; Didn’t Read) is one well-known example, but it depends on humans reading the documents and pointing out the crucial things in them.

The initiative has shared their insight on the privacy policies and terms of service of some of the most widely used online services, but it will take a lot of time and effort to do the same for a never-ending stream of less-used and newly introduced offerings.

“Short notices based on information extracted from privacy policies have been shown to be useful and more usable, but face a significant scalability hurdle, given the number of policies and their evolution over time. Companies, users, researchers and regulators still lack usable and scalable tools to cope with the breadth and depth of privacy policies,” a group of researchers from École Polytechnique Fédérale de Lausanne (EPFL), the University of Wisconsin and the University of Michigan, have noted.

Their solution to the problem? Polisis, an AI-powered automated framework for privacy policies analysis, and PriBot, a free-form Question Answering system for privacy policies.

About Polisis and PriBot

“At the core of Polisis is a privacy-centric language model, built with 130K privacy policies, and a novel hierarchy of neural-network classifiers that caters to the high-level aspects and the fine-grained details of privacy practices,” the researchers explained.

For more technical details on how they went about creating it I suggest reading their paper, but users who want to immediately take advantage of what Polisis can offer can go here and here.

The first link will take you to an app that will give you a visual overview of the different aspects of the privacy policy you want examined:

The second link will lead you to a chatbot for answering user questions in real-time:

How accurate is Polisis?

The researchers made sure to note that “Polisis is not intended to replace the privacy policy – as a legal document – with an automated interpretation.”

Nevertheless, it seems to offer a pretty good snapshot of the information that is of significance to end users.

“Polisis can be used to assign privacy icons to a privacy policy with an average accuracy of 88.4%. This accuracy is computed by comparing icons assigned with Polisis’s automatic labels to icons assigned with manual annotations by three legal experts. To put this result in context, it is similar to the agreement level between trained human judges assessing privacy policies,” the researchers shared.

They also noted that the Polisis framework could ultimately be used to enable comparative shopping applications that advise the consumer by comparing the privacy aspects of multiple applications they want to choose from.