Dipin Khati

Dipin Khati

PhD Candidate in Computer Science | Deep Learning for Software Engineering | Interpretable AI & Large Language Models

Posts by Collection

portfolio

publications

Benchmarking Causal Study to Interpret Large Language Models for Source Code

ICSME Published in ICSME 2023 , 2023

One of the most common solutions adopted by software researchers to address code generation is by training Large Language Models (LLMs) on massive amounts of source code. Although a number of studies have shown that LLMs have been effectively evaluated on popular accuracy metrics (e.g., BLEU, CodeBleu), previous research has largely overlooked the role of Causal Inference as a fundamental component of the interpretability of LLMs’ performance. Existing benchmarks and datasets are meant to highlight the difference between the expected and the generated outcome, but do not take into account confounding variables (e.g., lines of code, prompt size) that equally influence the accuracy metrics. The fact remains that, when dealing with generative software tasks by LLMs, no benchmark is available to tell researchers how to quantify neither the causal effect of SE-based treatments nor the correlation of confounders to the model’s performance. In an effort to bring statistical rigor to the evaluation of LLMs, this paper introduces a benchmarking strategy named Galeras comprised of curated testbeds for three SE tasks (i.e., code completion, code summarization, and commit generation) to help aid the interpretation of LLMs’ performance. We illustrate the insights of our benchmarking strategy by conducting a case study on the performance of ChatGPT under distinct prompt engineering methods. The results of the case study demonstrate the positive causal influence of prompt semantics on ChatGPT’s generative performance by an average treatment effect of ≈3%. Moreover, it was found that confounders such as prompt size are highly correlated with accuracy metrics (≈0.412%). The end result of our case study is to showcase causal inference evaluations, in practice, to reduce confounding bias. By reducing the bias, we offer an interpretable solution for the accuracy metric under analysis.

Download here

Towards More Trustworthy and Interpretable LLMs for Code through Syntax-Grounded Explanations

Under Review Under Review (Major Revision) , 2024

Trustworthiness and interpretability are inextricably linked concepts for LLMs. The more interpretable an LLM is, the more trustworthy it becomes. However, current techniques for interpreting LLMs when applied to code-related tasks largely focus on accuracy measurements, measures of how models react to change, or individual task performance instead of the fine-grained explanations needed at prediction time for greater interpretability, and hence trust. To improve upon this status quo, this paper introduces ASTrust, an interpretability method for LLMs of code that generates explanations grounded in the relationship between model confidence and syntactic structures of programming languages. ASTrust explains generated code in the context of syntax categories based on Abstract Syntax Trees and aids practitioners in understanding model predictions at both local (individual code snippets) and global (larger datasets of code) levels. By distributing and assigning model confidence scores to well-known syntactic structures that exist within ASTs, our approach moves beyond prior techniques that perform token-level confidence mapping by offering a view of model confidence that directly aligns with programming language concepts with which developers are familiar. To put ASTrust into practice, we developed an automated visualization that illustrates the aggregated model confidence scores superimposed on sequence, heat-map, and graph-based visuals of syntactic structures from ASTs. We examine both the practical benefit that ASTrust can provide through a data science study on 12 popular LLMs on a curated set of GitHub repos and the usefulness of ASTrust through a human study. Our findings illustrate that there is a causal connection between learning error and an LLM’s ability to predict different syntax categories according to ASTrust – illustrating that our approach can be used to interpret model effectiveness in the context of its syntactic categories. Finally, users generally found ASTrust’s visualizations useful in understanding the trustworthiness of model predictions.

Download here

Mapping the Terrain: LLMs in Software Engineering Insights and Perspective

TOSEM Published in TOSEM (ACM Transactions on Software Engineering and Methodology) , 2025

This paper provides a comprehensive survey and perspective on the use of Large Language Models (LLMs) in software engineering, mapping the current landscape of research and applications. We examine how LLMs are being applied across various software engineering tasks, identify key insights from existing work, and provide perspectives on future directions and challenges in this rapidly evolving field.

Download here

On Explaining (Large) Language Models for Code Using Global Code-Based Explanations

Under Review Under Review (Major Revision) , 2025

Understanding how Large Language Models (LLMs) generate and reason about code is crucial for building trust and improving their performance in software engineering tasks. This paper introduces a novel approach to explain LLMs for code using global code-based explanations, moving beyond local explanations to provide a comprehensive understanding of model behavior across different code contexts and patterns.

Download here

A Causal Perspective on Measuring, Explaining and Mitigating Smells in LLM-Generated Code

ICSE Published in ICSE 2026 , 2025

Recent advances in large language models (LLMs) have accelerated their adoption in software engineering contexts. However, concerns persist about the structural quality of the code they produce. In particular, LLMs often replicate poor coding practices, introducing code smells (i.e., patterns that hinder readability, maintainability, or design integrity). Although prior research has examined the detection or repair of smells, we still lack a clear understanding of how and when these issues emerge in generated code. This paper addresses this gap by systematically measuring, explaining and mitigating smell propensity in LLM-generated code. We build on the Propensity Smelly Score (PSC), a probabilistic metric that estimates the likelihood of generating particular smell types, and establish its robustness as a signal of structural quality. Using PSC as an instrument for causal analysis, we identify how generation strategy, model size, model architecture and prompt formulation shape the structural properties of generated code. Our findings show that prompt design and architectural choices play a decisive role in smell propensity and motivate practical mitigation strategies that reduce its occurrence. A user study further demonstrates that PSC helps developers interpret model behavior and assess code quality, providing evidence that smell propensity signals can support human judgement. Taken together, our work lays the groundwork for integrating quality-aware assessments into the evaluation and deployment of LLMs for code.

Download here

talks

teaching

Teaching experience 1

Undergraduate course, University 1, Department, 2014

This is a description of a teaching experience. You can use markdown like any other post.

Teaching experience 2

Workshop, University 1, Department, 2015

This is a description of a teaching experience. You can use markdown like any other post.