What do studies say about Agile?

Agile Studies
This is going to be a different posting.  This one will be organic and added to as I find other good bits of information.  I need something like this to make it easier to go over an inventory of studies.  I need this to solve my "searching for studies" impediment which comes up periodically.

Help me keep this up to date.  If you have an Agile paper you want cited, comment about it. 
Refactoring:
 Understanding the Economics of Refactoring
Abstract: A novel method for estimating the expected maintenance savings given a refactoring plan. This work is motivated by the increased adoption of refactoring practices as part of new agile methodologies and the lack of any prescriptive theory on when to refactor.
Lance's Summary:

• best to refactor code that is going to be changed a lot in the future, so pay attention to where your rate of change is and keep that code clean
• a guideline is that your refactoring pays for itself after six times you needed to revisit the code and change it

Code Red: The Business Impact of Code Quality
– A Quantitative Study of 39 Proprietary Production Codebases

Abstract: Code quality remains an abstract concept that fails to get traction at the business level. Consequently, software companies keep trading code quality for time-to-market and new features. The resulting technical debt is estimated to waste up to 42% of developers’ time. At the same time, there is a global shortage of software developers, meaning that developer productivity is key to software businesses. Our overall mission is to make code quality a business concern, not just a technical aspect. Our first goal is to understand how code quality impacts 1) the number of reported defects, 2) the time to resolve issues, and 3) the predictability of resolving issues on time. We analyze 39 proprietary production codebases from a variety of domains using the CodeScene tool based on a combination of source code analysis, version-control mining, and issue informa- tion from Jira. By analyzing activity in 30,737 files, we find that low quality code contains 15 times more defects than high quality code. Furthermore, resolving issues in low quality code takes on average 124% more time in development. Finally, we report that issue reso- lutions in low quality code involve higher uncertainty manifested as 9 times longer maximum cycle times. This study provides evi- dence that code quality cannot be dismissed as a technical concern. With 15 times fewer defects, twice the development speed, and substantially more predictable issue resolution times, the business advantage of high quality code should be unmistakably clear.

Lance’s summary: I like where this paper is going but I’m not sure how to measure the ROI of coaching teams without:

-  measuring the code base in pre-sales

- creating a model that expresses likelihood that the code that’s being refactored is “the right code to refactor.”

STATIC CODE ANALYSIS, Alexandru G. Bardas, 2010

Abstract: This 9 page paper published in Romanian Economics Business Review mentions economic reasons for software development teams to use static code analyses as giving a 17%-27% reduction in ownership cost due to problems being discovered during the code creation rather than escaping into production. 

Test Driven Development:

TDD Improves Quality
Abstract: A paper first published in the Empirical Software Engineering journal reports: "TDD seems to be applicable in various domains and can significantly reduce the defect density of developed software without significant productivity reduction of the development team." The study compared 4 projects, at Microsoft and IBM that used TDD with similar projects that did not use TDD.

Does Test-Driven Development Really Improve Software Design Quality?
David S. Janzen, California Polytechnic State University, San Luis Obispo Hossein Saiedian, University of Kansas
Lance's summary: TDD allows simpler design == less lines of code to do the same thing
Summary: Software developers are known for adopting new technologies and practices Software developers are known for adopting new technologies and practices

Test-Driven Development and Software Quality
Lance's summary: a blog that comments on the IEEE paper "Does Test-Driven Development Really Improve Software Design Quality."

On the Effectiveness of Unit Test Automation at Microsoft

Lance's summary: Yes, unit testing alone has measurable economic benefits, but the benefits are less than TDD.

Pair Programing:
A good summary about the practice along with a little satire at wikipedia.

The Costs and Benefits of Pair Programming
Alistair Cockburn & Laurie Williams

Lance's summary: Pair programing creates a lot of value over the total product lifecycle (higher code quality, lower defects, reduces team bottlenecks, and increases work satisfaction) at the cost of 15% more development time.

ABSTRACT
Pair or collaborative programming is where two programmers develop software side by side at one computer. Using interviews and controlled experiments, the authors investigated the costs and benefits of pair programming. They found that for a development-time cost of about 15%, pair programming improves design quality, reduces defects, reduces staffing risk, enhances technical skills, improves team communications and is considered more enjoyable at statistically significant levels.


Evaluating Pair Programming with Respect to System Complexity and Programmer Expertise
Erik Arisholm, Member, IEEE, Hans Gallis, Tore Dyba ̊, Member, IEEE Computer Society, and Dag I.K. Sjøberg, Member, IEEE

Abstract—A total of 295 junior, intermediate, and senior professional Java consultants (99 individuals and 98 pairs) from 29 international consultancy companies in Norway, Sweden, and the UK were hired for one day to participate in a controlled experiment on pair programming. The subjects used professional Java tools to perform several change tasks on two alternative Java systems with different degrees of complexity. The results of this experiment do not support the hypotheses that pair programming in general reduces the time required to solve the tasks correctly or increases the proportion of correct solutions. On the other hand, there is a significant 84 percent increase in effort to perform the tasks correctly. However, on the more complex system, the pair programmers had a 48 percent increase in the proportion of correct solutions but no significant differences in the time taken to solve the tasks correctly. For the simpler system, there was a 20 percent decrease in time taken but no significant differences in correctness. However, the moderating effect of system complexity depends on the programmer expertise of the subjects. The observed benefits of pair programming in terms of correctness on the complex system apply mainly to juniors, whereas the reductions in duration to perform the tasks correctly on the simple system apply mainly to intermediates and seniors. It is possible that the benefits of pair programming will exceed the results obtained in this experiment for larger, more complex tasks and if the pair programmers have a chance to work together over a longer period of time.

All I Really Need to Know I Learned in Kindergarten
By Robert Fulghum (Fulghum 1988)
Lance's Comment: Not really a study, but important information for people interested in pairing studies.  This paper contains a lot of the behaviors needed for successful pair programming.

Abstract:
Share everything.
Play fair.
Don’t hit people.
Put things back where you found them.
Clean up your own mess.
Don’t take things that aren’t yours.
Say you’re sorry when you hurt somebody.
Wash your hands before you eat.
Flush.
Warm cookies and cold milk are good for you.
Live a balanced life – learn some and think some and draw and paint and sing and dance and play and work every day some.
Take a nap every afternoon.
When you go out into the world, watch out for traffic, hold hands and stick together.
Be aware of wonder.

Pair Programming Testimonials
Abstract:
If you would like to share your pairing experiences or read about other people's, this is the place. Describe the good and bad of your experiences pairing to help others in a similar situation - just a paragraph or two would be great . . . what worked well, what didn't work so well, what kinds of things did you 'attack' together, tips and tricks, etc. A great example of a testimonial is by Ron Jeffries in the July/August 2000 IEEE Software page 25.

Economic Cases for using Agile software development techniques: 

NIST Planning Report 02-3 The Economic Impacts of Inadequate Infrastructure for Software Testing.

Abstract:

This study takes a look at several business in several industries (auto manufacturing, financial services, ...). One of the conclusions is about the cost of rework (cost of change curve) is exponential. (See Table 1-5)