The Mythical Man-Month – Extracts III

Source: The Mythical Man-Month

Conceptual Integrity

I will contend that conceptual integrity is the most important consideration in system design. It is better to have a system omit certain anomalous features and improvements, but to reflect one set of design ideas, than to have one that contains many good but independent and uncoordinated ideas. Unfortunately most programming systems reflect big conceptual disunity. Usually this arises not from a serial succession of master designers, but from the separation of design into many tasks done by many men.

Taking into account that the purpose of a programming system is to make a computer easy to use, the ratio of function to conceptual complexity is the ultimate test of system design. Neither function alone nor simplicity alone defines a good design.

Simplicity and straightforwardness proceed from conceptual integrity. Every part must reflect the same philosophies and the same balancing of desiderata. Every part must even use the same techniques in syntax and analogous notions in semantics. Ease of use, then, dictates unity of design, conceptual integrity.

How to get conceptual integrity ?

Conceptual integrity dictates that the design must proceed from one mind, or from a very small number of agreeing resonant minds. Schedule pressures, however, dictate that system building needs many hands. Two techniques are available for resolving this dilemma:

• The first is a careful division of labor between architecture and implementation.
• The second is the new way of structuring programming implementation teams such as surgical team.

Separation of labor between architecture and implementation.

It is a very powerful way of getting conceptual integrity on very large projects.
By the architecture of a system, I mean the complete and detailed specification of the user interface. For a computer this is the programming manual. For a compiler it is the language manual. For a control program it is the manuals for the language or languages used to invoke its functions. For the entire system it is the union of the manuals the user must consult to do his entire job.

The architect of a system, like the architect of a building, is the user's agent. It is his job to bring professional and technical knowledge to bear in the unalloyed interest of the user, as opposed to the interests of the salesman, the fabricator, etc.

Architecture must be carefully distinguished from implementation. As Blaauw has said, "Where architecture tells what happens, implementation tells how it is made to happen."

Objections to this technique

This is criticized with the following questions:

Are not the architects a new aristocracy, an intellectual elite, set up to tell the poor dumb implementers what to do?

The answer to this must be yes and no.
Yes, in the sense that there must be few architects, their product must endure longer than that of an implementer, and the architect sits at the focus of forces which he must ultimately resolve in the user's interest. If a system is to have conceptual integrity, someone must control the concepts. That is an aristocracy that needs no apology.

No, because the setting of external specifications is not more creative work than the designing of implementations. It is just different creative work. The design of an implementation, given an architecture, requires and allows as much design creativity, as many new ideas, and as much technical brilliance as the design of the external specifications. Indeed, the cost-performance ratio of the product will depend most heavily on the implementer, just as ease of use depends most heavily on the architect.

Has not all the creative work been sequestered for this elite, leaving the implementers as cogs in the machine?

Similarly, I observe that the external provision of an architecture enhances, not cramps, the creative style of an implementing group. They focus at once on the part of the problem no one has addressed, and inventions begin to flow. In an unconstrained implementing group, most thought and debate goes into architectural decisions, and implementation proper gets short shrift.

Won't one get a better product by getting the good ideas from all the team, following a democratic philosophy, rather than by restricting the development of specifications to a few?

I will certainly not contend that only the architects will have good architectural ideas. Often the fresh concept does come from an implementer or from a user. However, all my own experience convinces me, and I have tried to show, that the conceptual integrity of a system determines its ease of use. Good features and ideas that do not integrate with a system's basic concepts are best left out. If there appear many such important but incompatible ideas, one scraps the whole system and starts again on an integrated system with different basic concepts.

The Mythical Man-Month – Extracts II

Source: The Mythical Man-Month

How to take forward really big projects

Small sharp team concept is not a viable solution because it is too slow. So we have to find other options.

The dilemma

For efficiency and conceptual integrity, one prefers a few good minds doing design and construction.
For large systems one wants a way to bring considerable manpower to bear, so that the product can make a timely appearance.
So how can these two needs be reconciled?

Harlan Mills's Proposal

Mills proposes that each segment of a large job be tackled by a team, but that the team be organized like a surgical team and not a hog-butchering team.

Surgical team

One does the cutting and the others give him every support that will enhance his effectiveness and productivity.

Hog-butchering team

Each member cutting away on the problem.

Surgical Team - Roles and work division

The following are the roles of 10 people that contribute in well differentiated and specialized roles on a programming team built on the surgical model:

The surgeon

Mills calls him a chief programmer. He needs great talent, ten years experience, and considerable systems and application knowledge, whether in applied mathematics, business data handling, or whatever.

His main responsibilities are:

• Defines the functional and performance specifications .
• Designs, codes  and tests the program.
• Writes its documentation.

He needs to have effective access to a computing system which not only runs his tests but also stores the various versions of his programs, allows easy file updating, and provides text editing for his documentation.

The copilot

He is the alter ego of the surgeon, able to do any part of the job, but is less experienced.
His main responsibilities are:

• Shares in the design as a thinker, discussant, and evaluator. The surgeon tries ideas on him, but is not bound by his advice.
• Knows all the code intimately.
• Researches alternative design strategies.

He may even write code, but he is not responsible for any part of the code.

The administrator.

The surgeon is boss, and he must have the last word on personnel, raises, space, and so on, but he must spend almost none of his time on these matters. Thus he needs a professional administrator who handles money, people, space, and machines, and who interfaces with the administrative machinery of the rest of the organization. He can serve two teams.

The editor.

The surgeon is responsible for generating the documentation; for maximum clarity he must write it. This is true of both external and internal descriptions.

The editor, however, takes the draft or dictated manuscript produced by the surgeon and criticizes it, reworks it, provides it with references and bibliography, nurses it through several versions, and oversees the mechanics of production.

Two secretaries.

The administrator and the editor will each need a secretary; the administrator's secretary will handle project correspondence and non-product files.

The program clerk.

He is responsible for maintaining all the technical records of the team in a programming-product library.
The clerk is trained as a secretary and has responsibility for both machine-readable and human-readable files.

All computer input goes to the clerk, who logs and keys it if required. The output listings go back to him to be filed and indexed. The most recent runs of any model are kept in a status notebook; all previous ones are filed in a chronological archive.

The specialized function of the program clerk relieves programmers of clerical chores, systematizes and ensures proper performance of those oft-neglected chores, and enhances the team's most valuable asset—its work-product.

The toolsmith.

File-editing, text-editing, and interactive debugging services are now readily available, so that a team will rarely need its own machine and machine-operating crew.

But these services must be available with unquestionably satisfactory response and reliability; and the surgeon must be sole judge of the adequacy of the service available to him. He needs a toolsmith, responsible for ensuring this adequacy of the basic service and for constructing, maintaining, and upgrading special tools—mostly interactive computer services—needed by his team.

His job is to see to the tools needed or wanted by his surgeon, without regard to any other team's needs. The tool-builder will often construct specialized utilities, catalogued procedures, macro libraries.

The tester.

The surgeon will need a bank of suitable test cases for testing pieces of his work as he writes it, and then for testing the whole thing.  The tester is therefore both an adversary who devises system test cases from the functional specs, and an assistant who devises test data for the day-by-day debugging. He would also plan testing sequences and set up the scaffolding required for component tests.

The language lawyer.

By the time Algol came along, people began to recognize that most computer installations have one or two people who delight in mastery of the intricacies of a programming language. And these experts turn out to be very useful and very widely consulted. The talent here is rather different from that of the surgeon, who is primarily a system designer and who thinks representations. The language lawyer can a neat and efficient way to use the language to do difficult, obscure, or tricky things.

Often he will need to do small studies (two or three days) on good technique. One language lawyer can service two or three surgeons.

How It Works

The team just defined meets the desiderata in several ways.
Ten people, seven of them professionals, are at work on the problem, but the system is the product of one mind—or at most two, acting uno animo.

Notice in particular the differences between a team of two programmers conventionally organized and the surgeon-copilot team.

• First, in the conventional team the partners divide the work, and each is responsible for design and implementation of part of the work.
  In the surgical team, the surgeon and copilot are each cognizant of all of the design and all of the code. This saves the labor of allocating space, disk accesses, etc. It also ensures the conceptual integrity of the work.
• Second, in the conventional team the partners are equal, and the inevitable differences of judgment must be talked out or compromised.Since the work and resources are divided, the differences in judgment are confined to overall strategy and interfacing, but they are compounded by differences of interest—e.g., whose space will be used for a buffer. In the surgical team, there are no differences of interest, and differences of judgment are settled by the surgeon unilaterally.

These two differences—lack of division of the problem and the superior-subordinate relationship—make it possible for the surgical team to act uno animo. Yet the specialization of function of the remainder of the team is the key to its efficiency, for it permits a radically simpler communication pattern among the members.

The Mythical Man-Month – Extracts I

Source: The Mythical Man-Month

Identifying the joys and woes inherent to the system programming.

The Joys of the Craft

Following we can find some answers to the questions:  Why is programming fun? What delights may its practitioner expect as his reward?

• The sheer joy of making things.
• The pleasure of making things that are useful to other people.
• The fascination of fashioning complex puzzle-like objects of interlocking moving parts and watching them work in subtle cycles, playing out the consequences of principles built in from the beginning.
• The joy of always learning, which springs from the nonrepeating nature of the task.
• The delight of working in such a tractable medium. The programmer, like the poet, works only slightly removed from pure thought-stuff.

The Woes of the Craft

• One must perform perfectly.
• Other people set one's objectives, provide one's resources, and furnish one's information. One rarely controls the circumstances of his work, or even its goal.
• Designing grand concepts is fun; finding nitty little bugs is just work. With any creative activity come dreary hours of tedious, painstaking labor, and programming is no exception.
• One finds that debugging has a linear convergence, or worse, where one somehow expects a quadratic sort of approach to the end. So testing drags on and on, the last difficult bugs taking more time to find than the first.
• The product over which one has labored so long appears to be obsolete upon (or before) completion.
  

Why more software projects have gone awry for lack of calendar time?

• Our techniques of estimating are poorly developed.
• Our estimating techniques fallaciously confuse effort with progress.
• Because we are uncertain of our estimates.
• Schedule progress is poorly monitored. Techniques proven and routine in other engineering disciplines are considered radical innovations in software engineering.
• When schedule slippage is recognized, the natural (and traditional) response is to add manpower. Like dousing a fire with gasoline, this makes matters worse, much worse.

About creative activity

Dorothy Sayers, in her excellent book, The Mind of the Maker, divides creative activity into three stages:

The idea,

A book, then, or a computer, or a program comes into existence first as an ideal construct, built outside time and space, but complete in the mind of the author.

The implementation,

It is realized in time and space, by pen, ink, and paper, or by wire, silicon, and ferrite.

The interaction.

The creation is complete when someone reads the book, uses the computer, or runs the program, thereby interacting with the mind of the maker.

Demythologizing of the man-month.

The number of months of a project depends upon its sequential constraints. The maximum number of men depends upon the number of independent subtasks. From these two quantities one can derive schedules using fewer men and more months. (The only risk is product obsolescence.)

Ideas into Words – Extracts III

Source: Ideas into Words

Refining Your Draft

Before you start refining, do whatever will freshen your view of the manuscript. At a minimum, take a break and print out the manuscript.  After your break, proceed as if you had never seen the manuscript before. The idea is to approximate an outsider’s clear view of the piece as it stands. Next do the following things:

Read at cruising speed  and jot down your reactions

Read at cruising speed, like any other reader, but jot down your reactions in the border. Note that word—your reactions, not fixes. Keep moving, reserving your attention for the text and your own reactions. You want to notice every slightest flicker of boredom, impatience, confusion, put-off-ness, or pleasure. Do you have an impulse to skim? To jump ahead? To laugh? Are you working hard? Is your mind wandering? Make a quick note and keep moving. Write barely enough that you’ll know what you meant, along these lines:

• Waiting for story to start. What’s this about? . . .
• Bored . . .
• Woke up here, comp. lab busy at midnight a good touch
• LOL [laughed out loud] . . .
• Skimming, impatient . . .

Read the text out loud, or at least murmur it to yourself, lips moving, in order to spotlight any awkward patches.
Noting positive reactions is a must, and not only to preserve morale. Most of us tend to think of editing as “fixing” what is off. We forget the other half of the job, and maybe the more important half—retaining and strengthening what is good. The better to retain it, mark it.

Check the structure

In editing, your initial concern should be structural.  Aim to strengthen and balance the whole. Sweep through from beginning to end, again and again, solving the problems that your reactions pinpoint—first the big ones, then small ones.

About the opener

• Do you actually have an opener? Or were you merely clearing your throat? Initial reactions like “Bored” and “What’s this about?” are ominous.
• Does the opener still match the story as it turned out to be? Does the piece deliver on its promise?

About the closer

• Do you actually have a closer? Between fatigue and a desire to be done, you may have simply stopped without telling the reader good-bye.

Check your marks and examples

• Take a look at the passages you marked as any variant of “boring.” Do you want or need the material?
• Is the passage boring only because it is unclear? Most things seem boring when we don’t understand them.
• Do your examples demonstrate what you say they do? Bad examples sometimes survive from before you had total command of the subject, or because you found them charming.

Check the shape

• How’s the shape? As a whole, does the piece flow? Is it beginning to seem inevitable, as if the segments could never have been in any other order?
• Only with all big pieces in place should you go ahead to polish your writing, a process not unlike that of a plastic surgeon treating an aging movie star: you work all over.
• Take into account that :
  - Leading edge of a paragraph must be used to direct or redirect the reader’s attention.
  - Last part of a paragraph must be used to  emphasize something, the place that gives the reader her final impression (and perhaps a millisecond longer of brain time). Last place gives you a way to spotlight particular words and ideas that are critical to later understanding or that have important resonance.
  - Middle of the sentence must be used to de-emphasize something.

Follow the basic rules

• Replace passive verbs with active ones,
• Take out the garbage words—at least, most of them. By “garbage words,” I mean puny all-purpose modifiers such as very, really, rather, sort of, kind of, somewhat, quite, absolutely, extremely, and on and on.
• Take out redundant qualifiers
• When in doubt, throw it out.
• Your subconscious is your friend. If your subconscious made you do something, ask yourself why

Ideas into Words – Extracts II

Source : Ideas into Words

Creating your first draft

Following you can find some advices to create your first draft

As you write, keep your eye on the ball

I borrowed the sporty image in this mixaphor, hackneyed though it is, because in sports we all know it’s true (which is how it got hackneyed). It is hard enough to hit a tennis ball streaking toward you at 118 miles an hour. It cannot be done by a person who is preoccupied with losing, or his appearance, or anything else.

Your initial effort needs to be more or less continuous

Meaning day after day, as in all the arts. An artist friend often quotes her painting teacher on that subject: “You must. . . go . . . to the studio,” her teacher would say, slowly and with emphasis. “Once you are there, you might spend all morning sweeping the floor. That doesn’t matter. What matters is that you must . . . go . . . to the studio.”Yes, master, I hear you. “What am I really trying to say?” is a near-magic question.

Write out loud, mumbling or whispering to yourself as you write

Because reading is processed in the speech centers of the brain, any sentence or paragraph that is hard to speak will be hard to read, period. Not a lot harder, of course—but 1 percent improvements have a way of adding up, and this particular habit may be a 2 to 5 percenter.

Polish your prose late in the process rather than early

The more you work on a piece, the deeper it burrows into your neural pathways, and the harder you will struggle to see it freshly. The more effort you invest, the more every word will seem precious near  impossible to change.

Consider starting a bone heap

A place at the end of the manuscript for discarded sentences and paragraphs that you might yet want—dead examples, for example, or an aside that grew so big it disrupted the train of thought. The trouble with these items is that one gets attached to them, having invested the labor to create them. Hence the value of the bone heap: Knowing you can always retrieve that little gem, you’ll find it easy to be ruthless. An example is not quite working? Out!

Write with your notes and references open.

As a creative person, no matter how well you understand the subject, you need the constraints of genuine facts and quotes. Otherwise, you are likely to improve the stories and ideas past recognition. Use your notes. As a boss of mine used to say, “I don’t have time to take shortcuts.”

Make sure you put in all your raisins (i.e., fun facts, great quotes, and interesting comparisons).

Have you ever eaten a bread pudding that had too many raisins? I can’t imagine such a thing, and so it is with writing. You may not be able to turn a brilliant phrase yourself, but if you can recognize brilliant material when you see it, you can come close to a brilliant effect.

Take chances.

A draft is only a draft—by definition, the right place to experiment. Try writing lushly, or speaking more directly to the readers, or whatever you want to try. You will find the edge of the cliff, the place where you’ve gone too far, only by going over. Then once you’ve found the lip, you can stay two paces back.

Write using active verbs

Just as you were taught in high school English. Sentence by sentence, focus on action (which does what to what) rather than “procedures are” or “the data show that.”

Explain as needed

Not sooner and not later, not more and not less. If the article’s structure is right, the subject will unfurl like a morning glory, example/case and explanations inextricably mingled. Avoid any long patches of bald theory.

Keep the reader with you

Joined at the hip, by putting up a little slalom flag every time your train of thought takes a swerve or detour. A word or phrase will do it, of which our language has hundreds

Ideas into Words – Extracts I

Source: Ideas into Words

Main Advice

Whenever you get stuck while writing, stop struggling. Close your eyes, visualize a specific, living, breathing reader, and say to yourself, “What am I really trying to say?” Whatever the answer, write it down. Polish later if it’s needed, but you may be surprised at how trivial the polishing can be.

Getting Started

The first things that you have to do are:

Think about the readers.

Readers come in clusters. There is never only one, though one will be central. When you write, you will address that key reader directly, thereby rousing your social skills. The other readers will listen in and benefit from the occasional aside (or joke, or whatever) that you tuck in for their benefit.
From that viewpoint, your goal in writing is to capture and serve as many different readers as possible, yet stay focused on the core concern shared by the subgroups. You directly address the key reader, offering 100 percent of what that person needs. Then you throw the others a bone whenever one comes to hand.
As you start to think through a piece, imagine yourself as each reader in turn:

• Who are these people?
• What does each one need and expect from you?
• What will each group want to know?

If you meet one particular reader completely, will that do most of the job for the others? Yes, that’s the primary one, the reader. Knowing the reader early on will help you decide how to approach your article, and later it will help you choose vocabulary, examples, and analogies.

Think about the subject matter and mark your material for use in writing.

With your reader(s) held in mind, review all your notes and printed matter so that all is fresh in your mind, seen as a whole. If you are writing a brief news item, such a review may take fifteen minutes. For a major feature, it may take several days.

Write a head and subhead

Good ones, not perfunctory. The process will force you to get precise about both topic and approach.
As a unit, the heads have two jobs:

• To lure the readers in and
• To constitute a fair billing.

Consider pheromones, the chemical signals with which animals (including us) attract mates—moth pheromone does nothing for rutting bucks and vice versa. In the same way, the allure of your headline should speak specifically to the right readers, the cluster of people you are talking to.

Make a plan

Following the advice once given by Alexandre Dumas père for three-act plays:

• The beginning (first act) should be clear, clear, clear;
• The middle (second act) should be interesting, interesting, interesting;
• And the end should be short, short, short.

Your written plan may be very simple, especially for something short:

• Head and subhead,
• Idea for the opener,
• Idea for the closer,
• Plus a list of three to five major points you want to make in between.

Structure your piece in such a way that, when your train of thought comes to an end, its caboose just happens—of course not, but it should feel that way, natural and inevitable— to be a good place to leave the reader. That place might be a scene, a new insight, a question, or simply a final image that encapsulates the major idea. Often, the conclusion enlarges the picture, and it may well bear on the reader’s eternal question, why anyone should care. 

If the grand finale of your article is clear to you but the structure is not, try a more “logical” approach:  Start your plan from the end. Whatever your grand finale, back up. Ask yourself, what does the reader need to know to really get this final part? Good. That material belongs in the penultimate section. And so on. Just keep backing up toward your opener till you get there (or perhaps to a better one yet).A shape should then be apparent—or at any rate present, even if you don’t yet see it.

The Clean Coder–Extracts

Source: The Clean Coder: A Code of Conduct for Professional Programmers

About software projects

The fundamental assumption underlying all software projects is that software is easy to change.If you violate this assumption by creating inflexible structures, then you undercut the economic model that the entire industry is based on.

Professionals spend time caring for their profession.

You should plan on working 60 hours per week. The first 40 are for your employer. The remaining 20 are for you. During this remaining 20 hours you should be reading, practicing, learning, and otherwise enhancing your career.

Presumably you became a software developer because you are passionate about software and your desire to be a professional is motivated by that passion. During that 20 hours you should be doing those things that reinforce that passion. Those 20 hours should be fun!

Professionals practice.

True professionals work hard to keep their skills sharp and ready. It is not enough to simply do your daily job and call that practice. Doing your daily job is performance, not practice. Practice is when you specifically exercise your skills outside of the performance of your job for the sole purpose of refining and enhancing those skills.

Team Player

We’ve all heard how important it is to be a “team player.” Being a team player means playing your position as well as you possibly can, and helping out your teammates when they get into a jam. A team-player communicates frequently, keeps an eye out for his or her teammates, and executes his or her own responsibilities as well as possible.

Test Automation Pyramid

The kinds of tests that a professional development organization needs are:

Unit Tests

At the bottom of the pyramid are the unit tests. These tests are written by programmers, for programmers, in the programming language of the system.
The intent of these tests is to specify the system at the lowest level. Developers write these tests before writing production code as a way to specify what they are about to write. They are executed as part of Continuous Integration to ensure that the intent of the programmers’ is upheld.

Component Tests

Generally they are written against individual components of the system. The components of the system encapsulate the business rules, so the tests for those components are the acceptance tests for those business rules. It passes input data into the component and gathers output data from it. It tests that the output matches the input. Any other system components are decoupled from the test using appropriate mocking and test-doubling techniques.

Component tests cover roughly half the system. They are directed more towards happy-path situations and very obvious corner, boundary, and alternate-path cases. The vast majority of unhappy-path cases are covered by unit tests and are meaningless at the level of component tests

Integration Tests

These tests only have meaning for larger systems that have many components. These tests assemble groups of components and test how well they communicate with each other. The other components of the
system are decoupled as usual with appropriate mocks and test-doubles.

Integration tests are choreography tests. They do not test business rules. Rather, they test how well the assembly of components dances together. They are plumbing tests that make sure that the components are properly connected and can clearly communicate with each other.

System Tests

These are automated tests that execute against the entire integrated system. They are the ultimate integration tests. They do not test business rules directly. Rather, they test that the system has been wired together correctly and its parts interoperate according to plan. We would expect to see throughput and performance tests in this suite.

Manual Exploratory Tests

This is where humans put their hands on the keyboards and their eyes on the screens. These tests are not automated, nor are they scripted. The intent of these tests is to explore the system for unexpected behaviors while confirming expected behaviors. Toward that end we need human brains, with human creativity, working to investigate and explore the system. Creating a written test plan for this kind of testing defeats the purpose.