Skip to content

Reading List: The Paper Pool

The paper pool for Project 2, and a suggested reading list for the course. It is mostly software engineering research, with the AI papers you need to make sense of it. Not a complete bibliography, and not everything the slides cite: pick what your project needs.

One Project 2 route is to start from a paper: pick one from this pool and replicate, extend, or challenge one of its claims. Either way, your research abstract (due Fri, Oct 2) has to name the papers whose open question you are answering. Start from this page.

Venues and years are given for convenience; check the canonical version before you cite anything.

Start here: where the field is going

  • Hassan et al., Agentic Software Engineering: Foundational Pillars and a Research Roadmap, arXiv:2509.06216, the agenda the ⚙ thread follows; most sections end in an open question sized for a course project
  • Hassan et al., Towards AI-Native Software Engineering (SE 3.0), arXiv:2410.06107, the shorter vision paper
  • Fan et al., Large Language Models for Software Engineering: Survey and Open Problems, ICSE-FoSE 2023, the map of the whole area
  • Hou et al., Large Language Models for Software Engineering: A Systematic Literature Review, TOSEM 2024, the exhaustive version; use it to find work near your topic

Living collections

The instructor's papers

Extending, replicating, or challenging one of these is a legitimate Project 2 route, and it comes with built-in supervision: the person who wrote the paper is the person giving you feedback.

  • Wu & Fard, HumanEvalComm: Benchmarking the Communication Competence of Code Generation for LLMs and LLM Agent, TOSEM 2025: what happens when the problem description is ambiguous, inconsistent or incomplete, and whether the model asks
  • Joel, Wu & Fard, A Survey on LLM-based Code Generation for Low-Resource and Domain-Specific Programming Languages, TOSEM 2025: where code LLMs stop working, and why
  • Wu et al., Investigating the Efficacy of Large Language Models for Code Clone Detection, ICPC ERA 2024: an SE task LLMs are unexpectedly awkward at
  • Wu, An Exploratory Study of V-Model in Building ML-Enabled Software: A Systems Engineering Perspective, CAIN 2024: what a classical process model does and does not survive

Code generation and how we evaluate it

  • Chen et al., Evaluating Large Language Models Trained on Code, arXiv 2021. Codex, HumanEval, and the definition of pass@k
  • Liu et al., Is Your Code Generated by ChatGPT Really Correct? (EvalPlus), NeurIPS 2023: weak tests inflate benchmark scores; the correction matters
  • Austin et al., Program Synthesis with Large Language Models (MBPP), arXiv 2021
  • Li et al., StarCoder: May the Source Be With You!, TMLR 2023, what an open, documented code-model release looks like
  • Hindle et al., On the Naturalness of Software, ICSE 2012: the paper that started it all; code is repetitive enough to model statistically
  • Pan et al., Lost in Translation: A Study of Bugs Introduced by Large Language Models while Translating Code, ICSE 2024: a bug taxonomy for LLM code translation, a shape many projects can reuse

Agents for software engineering

  • Jimenez et al., SWE-bench: Can Language Models Resolve Real-World GitHub Issues?, ICLR 2024, how agent progress is scored
  • Yang et al., SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering, NeurIPS 2024, the interface is the design
  • Wang et al., OpenHands: An Open Platform for AI Software Developers as Generalist Agents, ICLR 2025, an open platform to experiment on

AI-assisted development: the human evidence

  • Peng et al., The Impact of AI on Developer Productivity: Evidence from GitHub Copilot, arXiv 2023, the controlled experiment everyone cites
  • Tufano et al., Using Pre-Trained Models to Boost Code Review Automation, ICSE 2022: LLMs entering the code-review loop
  • Barke et al., Grounded Copilot: How Programmers Interact with Code-Generating Models, OOPSLA 2023, acceleration vs. exploration; the best qualitative account
  • Vaithilingam et al., Expectation vs. Experience: Evaluating the Usability of Code Generation Tools, CHI EA 2022, people like the tools even when they do not help
  • Perry et al., Do Users Write More Insecure Code with AI Assistants?, CCS 2023
  • Pearce et al., Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributions, IEEE S&P 2022: the CWE-level security study; Project 2 topic B5 replicates its shape

Testing, analysis and assurance

  • Schäfer et al., An Empirical Evaluation of Using LLMs for Automated Unit Test Generation, TSE 2024
  • Lemieux et al., CODAMOSA: Escaping Coverage Plateaus in Test Generation with Pre-trained LLMs, ICSE 2023
  • Deng et al., Large Language Models are Zero-Shot Fuzzers (TitanFuzz), ISSTA 2023
  • Xia & Zhang, Keep the Conversation Going: Fixing 162 out of 337 Bugs for $0.42 Each Using ChatGPT, ISSTA 2024, repair as a conversation, with the cost reported
  • Xia, Wei & Zhang, Automated Program Repair in the Era of Large Pre-trained Language Models, ICSE 2023: the systematic look at LLMs as repair tools
  • Kang et al., Large Language Models are Few-shot Testers: Exploring LLM-based General Bug Reproduction, ICSE 2023 (LIBRO): from bug report to failing test

Engineering systems that contain a model

  • Amershi et al., Software Engineering for Machine Learning: A Case Study, ICSE-SEIP 2019, the nine-stage workflow and what makes ML different
  • Sculley et al., Hidden Technical Debt in Machine Learning Systems, NeurIPS 2015: still the sharpest warning in the field
  • Mitchell et al., Model Cards for Model Reporting, FAT* 2019. Project 1's model card follows this
  • Gebru et al., Datasheets for Datasets, CACM 2021, and Project 1's data sheet follows this

Doing empirical SE research well

  • Ralph et al., ACM SIGSOFT Empirical Standards for Software Engineering Research, read the standard matching your study type before writing the proposal; reviewers will
  • Wohlin et al., Experimentation in Software Engineering (Springer): the reference book for study design and threats to validity

The AI side: enough to understand the above

  • Vaswani et al., Attention Is All You Need, NeurIPS 2017: the architecture
  • Brown et al., Language Models are Few-Shot Learners (GPT-3), NeurIPS 2020: scale and in-context learning
  • Hoffmann et al., Training Compute-Optimal Large Language Models (Chinchilla), NeurIPS 2022: why your Project 1 model gets ~20 tokens per parameter
  • Hu et al., LoRA: Low-Rank Adaptation of Large Language Models, ICLR 2022
  • Ouyang et al., Training Language Models to Follow Instructions with Human Feedback (InstructGPT), NeurIPS 2022
  • Rafailov et al., Direct Preference Optimization, NeurIPS 2023: the simplification that stuck
  • Wei et al., Chain-of-Thought Prompting Elicits Reasoning in Large Language Models, NeurIPS 2022
  • Lewis et al., Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks, NeurIPS 2020
  • Yao et al., ReAct: Synergizing Reasoning and Acting in Language Models, ICLR 2023, the loop every agent runs
  • DeepSeek-AI, DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning, 2025

How to find more

  1. Chase citations. Take one paper above, read what it cites and what cites it (Google Scholar's "cited by"). Two hops usually reaches the frontier.
  2. Skim recent proceedings: ICSE, FSE, ASE, MSR, EASE, AIware. The last two years are enough for this course.
  3. Mine the lectures. Every lecture closes on open questions, and any slide making a specific claim shows its source at the foot.