Why Extreme Programming is Difficult to Scale, and the Essential Elements of Scaled Agile

Implementing Extreme Programming is difficult not on the learning but the teaching side. On "how to teach novices to learn to program," Extreme Programming has a practice solidified in its methodology: pair programming. The "pairing" thing, in fact, implicitly assumes that the ratio of novices to skilled programmers cannot be higher than 1:1. Otherwise, there will be a situation where no novices are paired.

The ratio of the number of novices to skilled players in the team, or the leverage ratio of the team, is a key factor affecting the implementation of extreme programming.

Extreme Programming, which originated in the United States, implies an assumption about the team structure: the number of novices in the team is not more than the number of skilled players (or, in other words, the leverage ratio is not higher than 1). This assumption is generally true in the IT industry in North America, Western Europe, and Australia. A friend in Germany said that the leverage ratio of their team is as low as 1:3 - that is, the number of skilled players is three times the number of novices. Under such a team structure, pair programming can undoubtedly be very effective in delivering reliable working methods to novices and maintaining the basic skills of the team.

However, a "leverage ratio not higher than 1" is a luxury in China's IT industry. There are too many software development teams in China, and the leverage ratio is 3:1, 5:1, or even 8:1. Under such a team structure, skilled programmers cannot pair up with all novice programmers, resulting in a lack of guidance and supervision for novice programmers. Although they try to regulate the work of novices through training, processes, documents, etc., the lack of basic skills still leads to the novices' inability to complete their due actions with qualified quality and efficiency. The infeasibility of pair programming in a high leverage environment is why extreme programming is difficult to promote on a large scale in China.

Many teams try to solve this challenge is to letting novices only engage in a specific segment of software development, for example, just coding. Someone in the front will do detailed design for them, and someone in the back will do testing for them. I'm afraid that's not right. Novices can't obtain the basic skills required for complete software delivery by repeatedly practicing and engaging in the work of a specific segment. The results caused by this method are not uncommon: even after 1-2 years of coding, many programmers still cannot translate the requirements into appropriate software design and deliver software products that meet the quality requirements. The reason is simple: they do not practice software design and quality assurance in their work. This division of labor by segment does not help improve the basic skill level of the whole team. The team that does this is bound to be locked in a state of low efficiency and low quality.

When Royce used this diagram in his paper, he was trying to say that such a division of labor model was not feasible.

Image Source: Figure 1

Compared with skilled programmers, novice programmers are not incapable of understanding requirements, designing software, and ensuring code quality. They cannot perform these actions accurately on a larger scale due to a lack of practice. According to my observation, for the requirements that can be completed within 4 hours, only a little training is required, and the novice can solve it well, and the quality meets the requirements. It can be seen that novices are fully competent for the end-to-end delivery task of software.

However, for the demand granularity of 2-3 days of expected workload (which is the appropriate granularity for user stories in general), novices cannot effectively split it. More specifically, novices cannot change the requirements of a larger scale into smaller scale development tasks by performing the following operations:

• Put forward diagnostic problems and define the scope and boundary of requirements;
• Conduct analytical thinking and list clear requirements and acceptance conditions;
• Use conceptual thinking to map requirements with appropriate object models.

It can be seen that what newbies lack most is not the development ability of end-to-end delivery - this ability, we have verified, can be possessed after a very short period of training. The key ability that newbies lack and are difficult to acquire quickly is the cognitive ability to clarify, analyze and dismantle more complex problems. Division of labor in a highly leveraged environment must be defined based on the shortage of this critical capability.

Therefore, we identified the essential elements of scaled Extreme Programming (and any form of scaled agile) in a highly leveraged environment:

• All programmers, whether proficient or novice, carry out end-to-end requirements delivery;
• Reduce the cognitive requirements of novices through more detailed demand splitting than classic user stories;
• Based on the development task with less than 4 hours of workload, the novice can be quickly trained to deliver basic skills through deliberate practice.