Sunday, August 01, 2004

The Role of Conversation in Software Engineering (2/3)

In the programming literature, conversation is considered a double-edged sword. One the one hand, programming includes plenty of ?tacit? knowledge, which is easy to transfer in face-to-face discussion, but difficult to transfer with documentation. On the other hand, programmers are more effective in quiet and peaceful work spaces.

First of all, many programming tools have features, which you may not find alone. The filename completion in Windows 2000 is a good example: many people learn about it from peers, rather than from discussion groups or documentation. Also many people learn to use a debugger only after someone shows them how debugging works, even if it is in prominent place in the menus of their text editors. In addition, there are many platform-specific, tool-specific and software-specific skills which you typically learn from peers. When a programmer starts working with a new platform without guidance, it may take considerable time to sort these out.

Peter Naur's essay ?Programming As Theory Building? claims that one essential part of programming is building so-called ?theory of the program?, which is ?a theory of how certain affairs of the world will be handled by, or supported by, a computer program?.

Naur gives a good example:

"The compiler had been developed by group A for language L and computer X. Now another group B wanted to write a compiler for language L + M, a modest extension of L, for computer Y. The group B decided that the compiler for L developed by the group A would be a good starting point, and made a support agreement with group A. This support included full documentation, annotated program text, additional written design discussion and personal advice. The arrangement was effective and B managed to develop the compiler they wanted.

In the present context the important issue is the personal advice from group A on how to implement the extension M to the language. During the design phase group B made suggestions on how the extensions should be implemented and submitted them to group A for review. In several major cases it turned out that the solutions suggested by group B were found by group A to make no use of the facilities which were not only inherent in the structure of the existing compiler but were discussed at length in its documentation, and to be based instead on additions to that structure in form of patches that effectively destroyed its power and simplicity. The members of group A were able to spot these cases instantly and could propose simple and effective solutions, framed entirely within the existing structure. This is an example of how full program text and additional documentation is insufficient in conveying to even the highly motivated group B the deeper insight into the design, that theory which is immediately present to the members of group A.

Later, the compiler developed by group B was taken over by other programmers of the same organization, without guidance from group A. Information obtained by a member of group A about the compiler resulting from the further modification of it after about 10 years made it clear that at that later stage the original powerful structure was still visible, but made entirely ineffective by amorphous additions of many different kinds. Thus, again, the program text and its documentation has proved insufficient as a carrier of some of the most important design ideas.?

After this, Naur starts telling the characteristic features of this insight. First of all, the insight is formed when the programmer writes the program, or modifies the program under guidance from someone who has this insight. Secondly, it is not ?What there is? type of knowledge, but ?How to? ?knowledge: How certain behaviour can be achieved while preserving the essential features of the program. My experience is that documentation is almost always ?what there is? type of knowledge: lists of modules, the responsibilities of each module, lists of requirements which the system fulfills etc.

Thirdly, the insight is essentially tacit: It can't be fully documented. Transferring the insight requires interactive discussion. Fourthly, iss characteristic feature is accountability: A person with the insight is able to answer background questions about the program, like the division of responsibility between classes, the reasons for choice of parameters for functions etc.

Fifthly, for a person without the insight, it is difficult to modify the program. Naur says that a program is ?alive? if there are programmers with the insight available to make modifications to the program. A program is ?dead? if the insight has been lost from the organization. A dead program can be run and used, but modification is problematic: Program resurrection is risky and unproductive.

The negative aspect of conversation is simpler. Joel sums it up (behind the link, the actual essay starts at the middle of the text). Basically, when a programmer is interrupted, it takes 15 minutes for him to get back to full concentration. There is also statistical data which shows that a programmer with a well-defined task can accomplish it more quickly in peaceful and quiet environment with no interruptions.

"Agile Software Development" draws some conclusions from this. Programmers in the same project should be placed near each others so that they can overhear each others' discussions. This way, they are more likely to have similar view about the theory of the program. Geographically distributed teams are bad but when they can't be avoided, special attention should be paid to communication. Two unrelated working groups should not be placed next to each other on landscape offices, since this creates "noise" talk which only disturbs.

(Note to casual readers: The other parts of this series about face-to-face conversation have nothing to do with software.)

No comments: