Ethics of Autonomy in AI: Examining Healthcare Diagnosis & Decision-Making

CS 312 AI ethics of Autonomy Part 2 Isabel Fabio, Isabella Esquilin, Jason Forrisi

I. Introduction to Autonomy in AI A. Definition of Autonomy in AI Autonomy as self-determination: the ability to make independent choices Ethical concerns surrounding AI systems influencing human autonomy B. Importance of Autonomy in AI Ethics How AI impacts real-world decision-making The fine line between assistance and control in AI-driven technologies

I. Introduction to Autonomy in AI C. Use Cases Examined Case 1: AI-powered healthcare diagnosis (Cardiso) Case 2: AI-driven personalized insurance pricing (Dynamic Insurance) D. Key Ethical Questions How does AI impact user autonomy? Where should AI decision-making end and human choice begin? Can AI systems maximize both autonomy and practical utility?

II. Case Study 1: Autonomy in AI-Powered Healthcare (Cardiso) A. Overview of Cardiso s AI Technology Private healthcare company using AI to analyze EKG data No doctor consultation required; AI predicts heart health risks Use of vast datasets to detect patterns in heart rhythms B. The Strengths and Limitations of AI in Healthcare AI surpasses human doctors in handling large datasets AI lacks holistic understanding (patient-reported symptoms, lifestyle, etc.)

II. Case Study 1: Autonomy in AI-Powered Healthcare (Cardiso) C. Ethical Dilemmas in AI-Driven Healthcare The Problem of Risk Categorization: Green: No concerns, normal heart patterns Red: Serious risk of heart failure, urgent medical attention needed Yellow (Proposed Solution): Moderate risk but not immediate danger The Difficulty of Setting Thresholds: Should a 10% chance of a heart attack in 10 years be red? Should a 50% chance of a heart attack in 5 years be red?

II. Case Study 1: Autonomy in AI-Powered Healthcare (Cardiso) D. The Impact of AI Predictions on Autonomy AI-driven predictions could cause lifestyle restrictions: People may stop traveling due to fear of sudden heart failure Users may adopt restrictive diets and avoid social drinking AI warnings may induce unnecessary anxiety and fear Ethical conflict: Maximizing biological health vs. preserving user autonomy Medical perspective: Lower all risks by advising caution Autonomy perspective: Let individuals decide their risk tolerance

II. Case Study 1: Autonomy in AI-Powered Healthcare (Cardiso) E. Solutions for Balancing AI and User Freedom Dynamic Risk Levels: Introduce yellow between green and red User-Informed Threshold Adjustments: Allow patients to set personal risk tolerance levels Provide optional risk explanations rather than absolute warnings Transparency and Consent: AI should explain why a user is classified as red/yellow/green Users should be allowed to opt in or out of detailed risk analyses

II. Case Study 1: Autonomy in AI-Powered Healthcare (Cardiso) F. Business and Ethical Considerations Overly cautious AI classifications harm customer satisfaction Ethical AI design ensures better user experience and company success AI ethics is essential to align business goals with public trust

III. Case Study 2: Autonomy and Dynamic Insurance A. Introduction to AI in Personalized Insurance AI-driven pricing models adjust rates based on user behavior Data collection from: Car sensors (speed, braking habits) B. The Initial Appeal of AI-Driven Insurance Users voluntarily opt-in to reduce premiums AI incentivizes safer driving and healthier lifestyles Autonomy appears enhanced: Users actively manage their risk

III. Case Study 2: Autonomy and Dynamic Insurance C. Risk of an Autonomy Trap Once enrolled, users may feel restricted in their actions: Driving slower or less frequently out of fear of premium hikes Avoiding exercise risks (Extreme Sports, Skiing) Continuous AI monitoring leading to self-imposed constraints Ethical Dilemma: Does constant AI feedback erode autonomy? D. The challenge of Setting Limits: AI must balance useful warnings vs. overbearing alerts At what point does personalized risk assessment become manipulative? E. Business vs Autonomy: Insurance companies benefit from risk-based pricing Overuse of AI-Driven interventions can cause customer dissatisfaction AI developers must ensure fairness, transparency, and user-trust

IV. Challenges to Autonomy in AI: Nudges and Dark Patterns A. Understanding Nudges: Definition: Alters behavior without forcing a choice, rewarding desired action, or prohibiting alternatives Example: Encouraging stair use with signs depicting calorie burning benefits AI-Enhanced nudging: Personalized behavioral suggestions: Not everyone motivated by burning calories, so signs display cardio benefits or energy benefits to fit the user B. The Ethical Debate Over AI Nudges: Can AI-driven nudges empower users by offering informed choices? When does nudging become psychological manipulation?

IV. Challenges to Autonomy in AI: Nudges and Dark Patterns C. Dark patterns in AI designs Definition: UI/UX choices that coerce, deceive or mislead users into making an uninformed decision Examples: Pre-checked boxes for unwanted services Tricky opt-out processes (making cancellations difficult) D. AI and Psychological Manipulation Example: Facebook s dopamine-based engagement model AI controls like distribution to maintain platform usage AI manages emotional engagement to influence user behavior E. Free Will vs Algorithmic Coercion Users choose to engage, but AI influences their decision-making Ethical Concern: Is AI enhancing or eroding human autonomy?

V. Theoretical Perspectives on Autonomy and AI A. Historical Philosophical Insights Gilles Deleuze s Society of Control Control via nudges rather than direct prohibitions AI s subtle guidance as a form of modern governance Jean-Paul Sartre s Existentialist Perspective Autonomy as fundamental to human identity AI s role in shaping individual freedoms and choices

V. Theoretical Perspectives on Autonomy and AI B. Future Ethical and Business Implications AI must balance user autonomy with responsible guidance Overly restrictive AI policies can lead to public backlash Ethical AI fosters trust, transparency, and long-term success

VI. Conclusion A. Summary of Key Autonomy Dilemmas in AI The challenge of balancing AI-driven assistance and user independence The ethical implications of nudging, dark patterns, and algorithmic influence B. Open Questions for AI Development How should AI companies define ethical boundaries when nudging? Can AI promote autonomy while minimizing risk?

VI. Conclusion C. The Need for Ethical AI Design Companies must align AI-driven insights with user freedoms Ensuring fairness and choice in AI-driven decision-making