How Can People With Intellectual and Developmental Disabilities Be Supported With Computer Vision AI in the Future?
by Hauke Sandhaus
The Challenge
People with intellectual and developmental disabilities (I/DD) seek to live independent and self-determined lives. However, they often need support to succeed in their daily routines. The organization YAI trains and employs direct support professionals (DSPs), who are the backbone of that support. Depending on the amount of support they need, people with I/DD can be on labor-intensive one-on-one (and even two-on-one) staffing for long periods of their day and night and, to make matters more complicated, they may have other health conditions, such as epilepsy.
YAI is therefore looking into technology that would help people with I/DD organize their lives, live happily and safely, and assist DSPs in their job. Some accessibility technology for people with I/DD already exists, however, common consumer products are typically not directly accessible for people with I/DD due to their unique conditions. Novel artificial intelligence-based technologies are enabled by data. What are the main challenges for developing and embedding data-driven assistive technology into the living environments of this underserved community?
Fig 1. A YAI program location
Discovery and Exploration Process
During my time at YAI, I worked on a case study to better understand this complex problem space. Together with my supervisor from YAI’s Center for Innovation and Engagement, we picked AI-assisted video monitoring of people with I/DD who have a high risk of seizures as a particularly knotty case study to disentangle.
First, I interviewed and sought information from YAI's stakeholders such as behavioral specialists, technology specialists, and program supervisors as they were knowledgeable about the organization's specific needs and processes for information technology, behavior intervention, human rights, direct support, and supervision at residential programs. Next, I researched and tinkered with off-the-shelf solutions, open-source software and AI models for recognizing seizures and other health-critical events. As the last step of the discovery process, I read research papers and got in touch with leading researchers to learn more about epileptic seizure recognition and classification.
Fig 2. Testing the pose recognition in bed with blurred image.
My Proposed Solution
he work and learnings from this process culminated in my creating a design prototype that was deployed in a brief field study. The camera AI-based prototype I built uses pose and motion recognition to identify the location and repetitive motion of the person with I/DD to help identify seizures. Pose and motion recognition was built on available models trained on open internet data. The prototype integrates with a smartphone application that allows real-time monitoring by DSPs and sends recognized seizure events as audible push notifications. The camera explains the monitoring state through an intuitive status indicator that also blocks the camera sensor when disabled. The design prototype illustrates an AI-assisted future of direct support provision. It is the first step towards conjointly defining a “tangible target”’ endeavored by all.
Hauke Sandhaus
Impact and Path Forward
My work outlined practical steps towards a (computer vision) AI-assisted future of direct support. By prototyping, I could show that for many use cases, readily available AI models are sufficient. As revealed by this case study, the problems of developing and embedding AI in this context appear to largely lie in ethical, organizational, educational and design challenges; not in technical ones. However, current AI models for seizure recognition are still limited in their accuracy by a lack of domain data.
YAI has already begun implementing methods to gather some quantitative data from smart hubs placed in participating program residencies. Moving forward, YAI will look into methods to collect data that go beyond the current logging of usage. YAI’s senior leadership acknowledged that there is value and need in collecting data to enable AI applications that can improve the lives of people with IDD. The next steps are to design processes and generate frameworks that help YAI to collect helpful data from people with IDD in a serviceable and ethical manner.
