Interactive Plush Toys: Exploring Machine Learning Through Storytelling

Discussion on PlushPal: Storytelling with Interactive Plush Toys and Machine Learning CSCE 634 Intelligent User Interface Zihao Wang 11/15/2024

Introduction to PlushPal What is PlushPal? A web-based design tool for children aged 8-14 to make stuffed toys interactive. Uses machine learning (ML) for gesture recognition, allowing toys to respond to custom movements and sounds. Objective of PlushPal: To introduce children to basic ML concepts such as data sampling, model training, and evaluation. Fosters creativity through interactive storytelling, combining play and learning. Research Focus: Exploring how children engage with ML concepts. Examining their debugging practices and approach to creating custom ML models.

Relevance of Research Today Trends in AI Education: Increased focus on accessible ML tools to introduce AI literacy at younger ages. Aligns with current initiatives for computational thinking and data literacy in K-12 education. Examples of Related Tools: Google s Teachable Machine: Easy-to-use platform for creating ML models based on images, sounds, and poses without coding. MIT App Inventor: Empowers children to design mobile apps using basic ML concepts.

Relevance of Research Today Why it Matters: Data Literacy: Prepares children to understand and interact with AI in their daily lives. Creativity and Agency: Children gain hands-on experience designing technology, giving them control over how toys interact and respond. Long-term Impact: Equips young learners with foundational AI skills. Cultivates critical thinking about the role and limitations of AI.

Simons Theoretical Perspective Concept from Herbert Simon s Sciences of the Artificial Simon views artificial systems as tools that mimic real-world actions and intelligence. Emphasizes design as a process that extends human cognitive ability, allowing users to modify and control their environment. Connection to PlushPal: Mimicking Reality: PlushPal toys perform actions (like moving or making sounds) through gestures, mimicking real-world behavior. User Control and Customization: PlushPal empowers children to define gestures and sounds, creating personalized interactions with their toys. Feedback Loop and Iterative Design: Aligns with Simon s idea that design is a feedback-driven process where users experiment, refine, and adapt models based on interaction.

Simons Perspective Applied in PlushPal Design as a Cognitive Extension: PlushPal s interface encourages children to iterate and refine their models, enhancing their problem-solving and critical thinking skills. By customizing toy behavior, children learn how to manage and adapt technology to serve specific purposes. Interactive Feedback Loops: As children test gestures, they see real-time feedback, learning to adjust model parameters (e.g., more samples for accuracy). Reinforces Simon s belief in trial-and-error as a critical part of learning and design.

Suchmans Theoretical Perspective Concept from Lucy Suchman s Human-Machine Reconfigurations Emphasizes that human actions and machine responses are situational meaning emerges from specific contexts and interactions. Connection to PlushPal: Situated Actions: PlushPal lets children customize toys in unique ways, making each toy's interaction personalized and relevant to the child s own experience. Human-Machine Adaptation: Children s gestures and stories shape the ML model, allowing the toy s behavior to reflect individual personalities and contexts. Supporting Play as Contextual Interaction: Suchman s theory aligns with how PlushPal models user-specific behavior, creating individualized experiences and adapting machine responses to child-driven interactions.

Suchmans Perspective Applied in PlushPal Customized and Contextualized Interaction: Children s individual interactions drive the customization, as they connect gestures and sounds that reflect personal memories or imaginative stories. PlushPal as Situated Technology: ML models in PlushPal adapt based on how children teach their toys, turning the plush toy into a personalized companion with unique responses. Suchman s perspective highlights that toys are not one-size-fits-all; PlushPal enables dynamic, adaptable engagement based on each user s context.

Lessons for Our Project Our Project Objective: Building an AI-powered coding assistant, similar to Claude or GitHub Copilot, tailored for programming tasks. Key Takeaways from PlushPal: User-Centered Interaction: Just as children interact directly with their toys, our AI assistant can allow programmers to customize suggestions and refine the assistant s responses. Real-Time Feedback Integration: PlushPal s feedback-driven model allows children to adjust actions based on toy responses; similarly, our assistant could improve by incorporating user feedback on suggestions accuracy. Simplified Customization Interface: PlushPal uses an intuitive interface for gesture and sound linking. Our project can also benefit from a user-friendly interface where users specify coding preferences or style, making the AI more responsive to specific needs. Future Directions: Investigate ways to allow users to teach the AI assistant, similar to PlushPal s model training, for more accurate and contextually relevant coding assistance.