Artificial Intelligence and Sensitive Data

Posted On 6 Marzo 2025 By kallistamoonadmin In Intelligenza Artificiale e tutela della Privacy, PRIVACY LAW, Tutela del consumatore /  
Facebooktwitterredditpinterestlinkedinmail

Artificial Intelligence and Sensitive Data: A Dialogue Between Innovation, Law, and Vulnerability

The advent of artificial intelligence (AI) applications has redefined the boundaries of technology, economics, and, not least, law. These systems, powered by unprecedented amounts of data, promise efficiency, personalization, and innovative solutions to complex problems, from healthcare to smart city management. However, at the core of their functionality lies a crucial paradox: to provide increasingly sophisticated services, AI needs to access, process, and store information that is often classified as “sensitive” – biometric data, political or religious preferences, health conditions, sexual orientation, and more.

This interplay between technological progress and privacy protection raises legal and ethical questions of global significance. On the one hand, national and supranational legislations attempt to curb the risks of misuse, leveraging tools such as the GDPR in Europe or the CCPA in the United States. On the other hand, the relentless dynamism of AI challenges the static nature of regulations, creating tensions between innovation, fundamental rights, and collective security.

The issue, however, is not merely regulatory. The collection and use of sensitive data by AI call into question the very notion of individual autonomy: to what extent are users aware of the implications of surrendering their data? How can transparency be ensured in increasingly opaque (“black box”) algorithms? And, above all, how can we balance commercial interests, social utility, and the protection of vulnerable minorities, who are often subjected to systemic discrimination perpetuated precisely by AI systems?

AI Systems and Personal Data: Regulatory Principles and Frameworks in the EU and the U.S.

The use of personal data by artificial intelligence (AI) systems raises critical questions about privacy, accountability, and ethical governance. Below, we analyze the core principles and legal frameworks governing this issue in the European Union (EU) and the United States (U.S.), drawing on official sources.

I. Foundational Principles for AI and Personal Data

AI systems processing personal data must adhere to principles established in international and regional frameworks. Key principles include:

  1. Lawfulness, Fairness, and Transparency
    • Data processing must have a legal basis (e.g., consent, contractual necessity) and be transparent to users (GDPR, Art. 5(1)(a)).
    • AI decisions affecting individuals must be explainable (EU AI Act, Art. 13).
  2. Purpose Limitation
    • Data collected for specific purposes (e.g., fraud detection) cannot be repurposed without consent (GDPR, Art. 5(1)(b)).
  3. Data Minimization
    • Only data strictly necessary for the AI’s function should be collected (GDPR, Art. 5(1)(c); U.S. FTC Guidelines, 2023).
  4. Accuracy
    • Systems must ensure data accuracy and allow corrections (GDPR, Art. 5(1)(d); NIST AI Risk Management Framework).
  5. Storage Limitation
    • Data retention periods must be justified (GDPR, Art. 5(1)(e)).
  6. Integrity and Confidentiality
    • Robust security measures are required to prevent breaches (GDPR, Art. 5(1)(f); U.S. Health Insurance Portability and Accountability Act (HIPAA)).
  7. Accountability
    • Organizations must demonstrate compliance with the above principles (GDPR, Art. 5(2); U.S. Executive Order on Safe AI).

II. European Union: The GDPR and the AI Act

The EU’s approach combines strict data protection rules with emerging AI-specific regulations.

  1. General Data Protection Regulation (GDPR)
    • Scope: Applies to all entities processing EU residents’ data, regardless of location (Art. 3).
    • Key Provisions:
      • Consent: Must be explicit, informed, and revocable (Art. 7).
      • Automated Decision-Making: Individuals have the right to opt out of decisions made solely by AI (Art. 22).
      • Data Protection Impact Assessments (DPIAs): Required for high-risk AI systems (Art. 35).
    • Source: GDPR Full Text.
  2. EU AI Act (2024)
    • Risk-Based Classification: Prohibits AI systems posing “unacceptable risk” (e.g., social scoring) and imposes strict requirements on “high-risk” AI (e.g., hiring algorithms) (Art. 5).
    • Transparency Obligations: Users must be informed when interacting with AI (Art. 52).
    • Source: EU AI Act Provisional Agreement.
  3. Enforcement
    • Supervised by national Data Protection Authorities (DPAs) and the European Data Protection Supervisor (EDPS).
    • Penalties: Up to 4% of global turnover for GDPR violations (Art. 83).

III. United States: Sectoral Laws and Emerging Frameworks

The U.S. lacks a comprehensive federal data protection law but regulates AI through sectoral laws and voluntary guidelines.

  1. Existing Laws
    • Federal Trade Commission Act (FTC Act): Prohibits “unfair or deceptive practices,” including misuse of personal data by AI (Section 5).
    • California Consumer Privacy Act (CCPA): Grants Californians rights to access, delete, and opt out of the sale of their data (amended by CPRA, 2023).
    • Health Data: HIPAA regulates AI in healthcare, requiring anonymization and patient consent.
  2. Proposed Federal Legislation
    • Algorithmic Accountability Act (2023): Requires impact assessments for AI systems in housing, employment, and healthcare.
    • AI Bill of Rights (2022): Non-binding framework emphasizing:
      • Safe and effective systems.
      • Protection against algorithmic discrimination.
      • Transparency and explainability.
      • Source: White House AI Bill of Rights.
  3. Agency Guidelines
    • NIST AI Risk Management Framework (2023): Voluntary standards for managing AI risks, including data privacy.
    • FTC Enforcement: Recent actions against companies like Amazon and OpenAI for data misuse.

IV. Comparative Analysis: EU vs. U.S.

Aspect European Union United States
Regulatory Approach Comprehensive (GDPR + AI Act) Sectoral + state-level laws
Consent Requirements Explicit and granular (GDPR) Varies by state (e.g., CCPA)
AI Transparency Mandatory (AI Act) Voluntary (NIST Framework)
Enforcement Centralized (DPAs) + heavy fines FTC litigation + state-level penalties
Focus Fundamental rights and prevention Innovation + mitigating harm ex-post

V. Challenges and Criticisms

  • EU: Overly restrictive rules may stifle innovation (e.g., AI Act’s “high-risk” classification).
  • U.S.: Fragmented laws create compliance complexity (e.g., CCPA vs. Virginia’s CDPA).
  • Global Tensions: Cross-border data transfers (e.g., EU-U.S. Data Privacy Framework) remain contentious.

Suggested Citations for Legal Texts:

  • EU GDPR: Regulation (EU) 2016/679.
  • EU AI Act: COM/2021/206 final.
  • U.S. AI Bill of Rights: White House Office of Science and Technology Policy (2022).
Dott.ssa Luana Fierro
Read More