Endogenous Macrodynamics in Algorithmic Recourse

IEEE Conference on Secure and Trustworthy Machine Learning

Patrick Altmeyer

Delft University of Technology

Giovan Angela
Aleksander Buszydlik
Karol Dobiczek
Arie van Deursen
Cynthia C. S. Liem

May 1, 2024

Background

Counterfactual Explanation (CE) explain how inputs into a model need to change for it to produce different outputs.

Provided the changes are realistic and actionable, they can be used for Algorithmic Recourse (AR) to help individuals who face adverse outcomes.

Example: Consumer Credit

In Figure 1, arrows indicate changes from factuals (loan denied) to counterfactuals (loan supplied).

Figure 1: Counterfactuals for Give Me Some Credit dataset (Kaggle 2011).

Our work in a nutshell …

Figure 2: Dynamics in Algorithmic Recourse.

  • We show that counterfactuals can induce substantial domain and model shifts like in Figure 2.
  • We propose a novel perspective on AR that explicitly addresses this issue.
  • We open-source a Julia package to study the dynamics we point to.

Proof-of-Concept

Figure 3: A bank has trained a model to evaluate credit applicants. Credit risk is highest in bottom-right corner.
Figure 4: The bank has provided recourse to unsuccessful applicants: endogenous domain shift.
Figure 5: The bank has retrained the classifier: endogenous model shift.
Figure 6: The outcome after the process has been repeated a few times. Average default risk has increased.

Questions …

  • Who should bear the cost?
  • Are the counterfactuals genuinely valid in practice?
  • What about fairness and privacy concerns?
Figure 7: Indiviudals who received recourse are clearly distinguishable.

Experiments

Empirical Setup

Evaluation: we propose metrics to measure domain shifts and model shifts.

Models: we use linear classifiers, deep neural networks and deep ensembles.

Data:

Synthetic
  • Overlapping, Linearly Separable, Circles, Moons.

Real-World

Generators: Wachter (Wachter, Mittelstadt, and Russell 2017), REVISE (Joshi et al. 2019), DiCE (Mothilal, Sharma, and Tan 2020), Greedy (Schut et al. 2021)

Principal Findings — Synthetic

Domain shifts for overlapping synthetic data using deep ensemble.

Performance deterioration for overlapping synthetic data using deep ensemble.

Principal Findings — Real-World

Model shifts for Credit Default data using deep ensemble.

Performance deterioration for Credit Default data using deep ensemble.

Gradient-Based Recourse Revisited

From Individual Recourse …

Many existing approaches to CE and AR work with the following baseline:

\[ \begin{aligned} \color{lightgrey} \mathbf{s}^\prime &\color{lightgrey}= \arg \min_{\mathbf{s}^\prime \in \mathcal{S}} \{ {\text{yloss}(M(f(\mathbf{s}^\prime)),y^*)} \\&\color{black} + \lambda {\text{cost}(f(\mathbf{s}^\prime)) } \color{lightgrey} \} \end{aligned} \tag{1}\]

Typically concern has centred around minimizing costs to a single individual.

… towards collective recourse

We propose to extend Equation 1 as follows:

\[ \begin{aligned} \color{lightgrey}\mathbf{s}^\prime &\color{lightgrey}= \arg \min_{\mathbf{s}^\prime \in \mathcal{S}} \{ {\text{yloss}(M(f(\mathbf{s}^\prime)),y^*)} \\ &\color{lightgrey}+ \lambda_1 {\text{cost}(f(\mathbf{s}^\prime))} + \color{black}\lambda_2 {\text{extcost}(f(\mathbf{s}^\prime))} \color{lightgrey}\} \end{aligned} \tag{2}\]

The newly introduced term \(\text{extcost}(f(\mathbf{s}^\prime))\) is meant to explicitly capture external costs generated by changes to \(\mathbf{s}^\prime\).

Background: Externalities

We borrow the concept of private vs. external costs from Economics.

Figure 8: Illustration of a negative externality. Source: Investopedia.

Externalities in our Context

Relating this bank to our opening example, the external cost is carried by the retail bank.

Figure 9: External cost is carried by bank.

Mitigation Strategies

  1. More Conservative Decision Thresholds
  2. Classifier Preserving ROAR (ClaPROAR)1
  3. Gravitational Counterfactual Explanations
Figure 10: Mitigation strategies.

Secondary Findings

Domain shifts for overlapping synthetic data using deep ensemble.

Performance deterioration for overlapping synthetic data using deep ensemble.

Discussion

Key Takeaways 🔑

  • State-of-the-art approaches to AR induce substantial domain and model shifts.
  • External costs of Individual Recourse should be shared across stakeholders.
  • Ad-hoc solution: penalize external costs in the counterfactual search objective function (Equation 2).
  • This gives rise to interesting cross-disciplinary research ideas (e.g. Pareto-optimal AR).

Counterfactual Explanations and Probabilistic Machine Learning

Methodologies and open-source tools that help researchers and practitioners assess the trustworthiness of predictive models.

Towards Trustworthy AI in Julia
  1. CounterfactualExplanations.jl (JuliaCon 2022)
  2. ConformalPrediction.jl (JuliaCon 2023 — I hope!)
  3. LaplaceRedudx.jl (JuliaCon 2022)
  4. AlgorithmicRecourseDynamics.jl

… contributions welcome! 😊

📚 More Reading

Image Sources

  • Copyright for stock images belongs to TU Delft.
  • All other images, graphics or animations were created by us.

References

Joshi, Shalmali, Oluwasanmi Koyejo, Warut Vijitbenjaronk, Been Kim, and Joydeep Ghosh. 2019. “Towards Realistic Individual Recourse and Actionable Explanations in Black-Box Decision Making Systems.” https://arxiv.org/abs/1907.09615.
Kaggle. 2011. “Give Me Some Credit, Improve on the State of the Art in Credit Scoring by Predicting the Probability That Somebody Will Experience Financial Distress in the Next Two Years.” https://www.kaggle.com/c/GiveMeSomeCredit; Kaggle. https://www.kaggle.com/c/GiveMeSomeCredit.
Mothilal, Ramaravind K, Amit Sharma, and Chenhao Tan. 2020. “Explaining Machine Learning Classifiers Through Diverse Counterfactual Explanations.” In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, 607–17. https://doi.org/10.1145/3351095.3372850.
Schut, Lisa, Oscar Key, Rory Mc Grath, Luca Costabello, Bogdan Sacaleanu, Yarin Gal, et al. 2021. “Generating Interpretable Counterfactual Explanations By Implicit Minimisation of Epistemic and Aleatoric Uncertainties.” In International Conference on Artificial Intelligence and Statistics, 1756–64. PMLR.
Upadhyay, Sohini, Shalmali Joshi, and Himabindu Lakkaraju. 2021. “Towards Robust and Reliable Algorithmic Recourse.” Advances in Neural Information Processing Systems 34: 16926–37.
Wachter, Sandra, Brent Mittelstadt, and Chris Russell. 2017. “Counterfactual Explanations Without Opening the Black Box: Automated Decisions and the GDPR.” Harv. JL & Tech. 31: 841. https://doi.org/10.2139/ssrn.3063289.
Yeh, I-Cheng, and Che-hui Lien. 2009. “The Comparisons of Data Mining Techniques for the Predictive Accuracy of Probability of Default of Credit Card Clients.” Expert Systems with Applications 36 (2): 2473–80. https://doi.org/10.1016/j.eswa.2007.12.020.