Search
Buy your pass
OnlineDiscovery - AI for Earth and Sustainability Science Badge available

Do AI models know what they know? How explainable AI can help us trust what machines learn

  • Register

    Watch

     

    Watch and get your discovery badge

    * Register (or log in) to the Neural Network to add this session to your agenda or watch the replay

    • Date
    24 November 2025
    Timeframe
    17:00 - 18:00 CET Geneva
    Duration
    60 minutes

    AI models can make powerful predictions, from diagnosing diseases to forecasting climate extremes, but understanding why they make those predictions remains one of the biggest challenges in deploying them responsibly.

    In this talk, the speaker explores what it really means for an AI system to “know” something. Using a synthetic benchmark dataset inspired by climate prediction tasks, where the true drivers of the target variable are known, the assessment focuses on explainable AI (XAI) tools and their ability to recover these drivers under varying levels of noise and data availability – conditions that mirror many real-world scientific and societal applications. Two clear insights emerge: first, explanations become reliable only when models truly learn signal rather than noise; and second, agreement among different explanation methods or models can serve as a practical indicator of whether an AI system has learned something meaningful or is still operating in a state of “epistemic ignorance”.

    These findings offer guidance for building AI systems that are not only powerful but also aware of their own limits, a key step toward AI that supports science and society with confidence and transparency.

     

    Session Objectives
    By the end of this session, participants will be able to:

    • Explain the concept of explainable artificial intelligence (XAI) and why explanation reliability depends on whether a model learns signal rather than noise.
    • Describe how controlled synthetic benchmarks with known ground-truth drivers can be used to objectively evaluate the faithfulness of XAI methods.
    • Analyze how variations in data noise and sample size affect both model performance and the reliability of explanations across different XAI methods.
    • Evaluate when and why cross-method or cross-model agreement can serve as a practical proxy for explanation trustworthiness in the absence of ground truth.
    • Apply these insights to critically assess the use of XAI in scientific, medical, or policy-relevant contexts, identifying situations where explanations are likely meaningful versus misleading.

     

    Recommended Mastery Level / Prerequisites:

    This talk is designed for an interdisciplinary audience and requires no advanced technical background, but participants will benefit from having: 

    • Basic familiarity with AI/ML concepts, such as supervised learning and neural networks. 
    • General understanding of uncertainty and noise in data (no formal statistics required).
    • Interest in the ethical and scientific implications of AI transparency—especially how to assess when AI predictions can be trusted. 

    Optional (for deeper engagement):

    • Some exposure to model interpretation tools (e.g., saliency maps, SHAP, or LIME) will enhance understanding of the examples.
    FAQ's
    • Discovery badge available
    Showcase it on your LinkedIn profile or your professional portfolio
    Verified badge is available for CHF 10
    Learn more
    Share this session with your network
    Discover more from programme stream
    Discovery - AI for Earth and Sustainability Science
    Risks and solutions when using non-stationary training data in earth...

    Risks and solutions when using non-stationary training data in earth...

    8 December 2025 - 17:00 to 18:00

    Do AI models know what they know? How explainable AI...

    Do AI models know what they know? How explainable AI...

    24 November 2025 - 17:00 to 18:00

    AI for humanity: Using earth observation for solving global challenges

    AI for humanity: Using earth observation for solving global challenges

    3 November 2025 - 17:00 to 18:00

    Forecasting and understanding bird migration with process-guided deep learning

    Forecasting and understanding bird migration with process-guided deep learning

    8 October 2025 - 16:00 to 17:00

    AI for early warnings addressing floods and droughts

    AI for early warnings addressing floods and droughts

    22 September 2025 - 16:00 to 17:00

    Related stories

    Are you sure you want to remove this speaker?