Enhancing Permission Slip, a digital data privacy assistant by Consumer Reports: Unveiling Consumer Insights and Ensuring Robust Validation
by Tuan-he Lee
Background
CR's mission is to empower consumers, foster responsible corporate conduct, and aid policymakers in promoting consumer-centric marketplaces. The CR Innovation Lab is dedicated to designing, prototyping, and scaling new solutions that enable consumers to make informed choices. One of their key initiatives is Permission Slip, an app designed to assist consumers in regaining control over their data and enhance data privacy. This summer, I joined the team as a Siegel Family Endowment PiTech PhD Impact Fellow in summer 2023 to address two questions regarding Permission Slip’s development.
The Challenge
First of all, it is still unclear what people actually want to do when they attempt to manage and protect their privacy, such as which types of data they care about and which companies concern them. To tackle this first issue, I conducted a Consumer Study to learn about consumers’ main concerns and priorities regarding data privacy management. This helped shape the app's design to better match user needs
Second, the Innovation Lab is exploring large language models (LLM) to analyze privacy policies. Large language models can read and synthesize vast amounts of text, but they can also hallucinate inaccurate content, which led to a new challenge: How can we ensure that these models understand the policy correctly? What’s the process of validation?
Tuan-he Lee
The Consumer Study
This study delved into the factors that influence consumer decisions around personal data management, inquired about the specific companies and data types that trigger high levels of concern, and sought to discern consumers' expectations for privacy tools.
Key Finding #1 Companies that collect "sensitive" data, such as that from camera and sensors, online-tracking and location and financial information, are of high priority for managing for consumers
Consumers prioritize managing data with companies that either handle sensitive data, or over-collect information that could jeopardize their security, or with whom they engage on a regular basis. The major concern lies with companies gathering what participants called “sensitive” and “private” data, such as financial, health, camera, and location-related information. They are concerned about the possible risk associated with sharing such information. Furthermore, excessive data gathering, especially unrelated to services, raises questions about necessity.
A subsequent closed-ended question added further color to the precise data categories that consumers desire to manage. About 90% of surveyed consumers expressed worry about Cameras & Sensors, Online Tracking, and Financial Data. The rankings also mirrored their reasons for grouping, with sensitive categories like financial and camera data taking priority. Furthermore, the perceived over-collection seems to be linked to worries about online tracking data.
Key Finding #2: Consumers’ privacy and security goals are intermingled
We found that consumers expect to feel protected, informed, and in control from using a tool that safeguards their personal data. Consumers' privacy goals are divided into five layers: prevent harm, feel more secure, be better informed, take extra control, and maintain data autonomy. (Figure 1).
Figure 1: Expectations toward privacy tools for safeguarding personal data
Preventing harm and enhancing security emerged as top priorities. In addition, consumers desire to learn from the tool, such as by understanding data collection practices. People also want to be able to take action, such as preventing data from being sold. Finally, the tool should be secure and not shady itself.
Ultimately, customers desire a secure and user-friendly app that provides an analysis of organizations' practices and assists them in taking real steps to restrict data collection.
The LLM validation research
The Permission Slip app contains content about the data collection practices of many different companies. Historically, CR derived this content by carefully reading companies’ privacy policies, asking 24 standard questions about each policy and hand-coding the answers. With the advancement of large language models (LLM), CR has seized the potential of large language models and generative AI, and started experimenting with using these technologies to expedite our process of reading and synthesizing companies’ privacy policies. Therefore, my role was to assist the team in analyzing the LLM-generated content and develop a validation framework that would guide CR’s validation process.
Our analysis has led to a validation framework for evaluating the content’s quality and refining the LLM’s output (Figure 2). This framework involves:
Human-generate a baseline;
Compare outputs: comparing LLM output against baseline;
Validate in house: leverage the research prototype for a consistent validation process;
Solicit feedback and continuously improve: the sources of additional accuracy checks include CR internal reviewers, stakeholders from the companies in question, and end-users;
Fine-tune models with validated outputs. With the validation framework, the prototyping team will strengthen in-house validation and foster and systematic validation process.
Figure 2: Validation framework v1
Our work has provided valuable insights and strengthened our validation process. Here are key takeaways from our initial validation round:
Prioritization: We've identified criteria for prioritizing validation, focusing on areas where the LLM is likely to struggle, where content will be visible to users, and where users are likely to take action based on the content they see.
Collaborative Approach: The first layer of collaboration relates to dividing tasks between human and machine, allowing us to automate some tasks. The second layer of collaboration involves internal parties (internal content experts) versus external parties such as stakeholders from the companies, and the end-users of the app.
Version Complexity: Handling diverse input and output versions during validation requires careful consideration.
Documentation and Comparison: Documentation helps us systematically examine improvements and understand the nuances of different prompts and inquiries.
Impact and Path Forward
Consumer insights, including consumers’ mental models, privacy goals, and distinct privacy personas, will guide content prioritization and strategic planning for the app.
The validation framework will enhance in-house validation and extend to other LLM-assisted projects.
For a deeper dive into my journey, check out this article on the CR Innovation Lab blog, where I talked about more findings from the consumer study.
