Planning

People ask me about Agile and other project planning methods in high tech today. What I tell them is that all tools can be used well or poorly. It takes leadership, forethought, common sense and perseverance to plan and follow through on a project.

Many managers do incomplete jobs at planning, or they do not manage the plan once it is developed, or they do not handle change appropriately. Each of these mistakes will affect predictability. I have seen some organizations that embrace some of these techniques as a facade for "it will get done when it gets done". The problem is that your CFO, VP of Sales and customers cannot plan appropriately in that kind of environment. Your company needs predictability in order to succeed.

I am a lot less interested in the form you use to plan than the functionality. All of these techniques can be applied using almost any method. Here are some basics:

• Do not commit to a date until you have preliminary specifications and detailed schedules
• Make sure every stakeholder physically signs off at each phase
• Adopt a train model
• Don't make top-down dates
• Complex projects require very detailed planning that will change
• Manage each slip with urgency and importance
• Deal with major changes as if you were starting anew

I have talked many times about the proposal and definition phases of a project. You cannot have predictability without specifications. Those specifications must be good enough to ensure you hare heading in the correct architectural and product direction. They must also be good enough to create schedules. The schedules must prove that you can make the target date. You must work with the information you have at the time and understand that it will change. The schedules should be manageable (< 2 week tasks, measurable deliverables, etc.).

Don't assume that verbal or incidental communication is enough to get buy-in from your company. Get a signoff sheet for each phase in writing. Make sure development, QA, docs, architects, sales(customers), support, and the CEO are all in the loop. It is awful to find out after you have spent money on developing something that you have done the wrong thing.

Feature driven releases often slip. If a major feature slips and the release slips, then there is pressure to add more features since it will be a while until the next release. Adding late features can cause more integration work and potentially more slip or worse yet cause quality problems. Separate features out from releases. Make releases time based with a shorter cycle (3 months). With a time based release, the features that are ready can go out. Big features can be developed independently. You can respond to customer needs more quickly. The train model requires adequate automated testing to be successful in order to reduce the overhead of more frequent releases.

Top down dates don't work. Schedules must be bottoms up. Managers must help there teams make aggressive but accomplishable schedules. Managers must help make trade-offs around function, quality and resources and have substantive business discussion to achieve the right balance. Don't commit until everyone on your team can look you in the eye and commit. This is a pay-me-now or pay-me-later issue. It is more than a burn-out issue. It is again about predictability.

Complex projects need detailed schedules. Even if they go out two years. Schedule re-scheduling events for long projects. If you use Agile, you can translate detailed schedules into sprints but you still need the homework upfront. You need the exercises to get predictability. The schedule is really a step-by-step design. It makes your developers think. It allows your team to identify many dependencies and resource issues upfront. It will change -- get over it. There is no free lunch. You cannot have predictability with coarse grain tasks. You cannot have predictability without having your best guess at the details upfront.

You need to manage the plan. Most slips start from day one of the schedule. Require your team to have at most two tasks open per person at one time as they accumulate risk (this may require scaffolding). Require that each slip have a mitigation plan on a weekly basis. Do not shove all delays into a bucket to be dealt with later. Your team needs to adjust quickly to issues.

Finally, if you have major changes including features additions or architectural surprises, you need to go back into a planning phase. Never assume you can just add it on. Do the resource planning, specifications and schedules. Do new signoffs. Anything less will affect predictability and cause slips.

We all face these issues. True leadership in planning is the foundation for success for your team.

More later ...

How late should you allow changes in a release?

This entry is about the very end of a release. I have talked about the whole end-game in a previous posting but this period warrants more discussion. There are always more bugs to fix. Sometimes this pressure comes from engineering trying to perfect a release and sometimes from product management trying to help out one more customers or avoid a service call. Either way I suggest you avoid a lot of changes down the end of a release.

There should always be a quiet period before a release where only bugs that would stop shipment and cause an immediate patch of a production release should get fixed. Clearly the length of this period will vary with the size of release or the maturity of a company. But the period should not be less time than a full run of testing (which should be set to take 7 or less days).

In this period, any bugfix is as likely to cause more problems as fix problems. People like to minimize the impact of changes in this period but my experience does not support that position. I have seen slips due to innocuous changes.

If you have not met your bug goals or you have not completed testing do not enter this period because you will need to fix bugs that are not appropriate for this period. Do not assume you can fix a lot of bugs and keep the release stable.

So what can you do to manage this period:

• Show leadership
• Plan a patch release shortly after your release as a vehicle to handle critical non-showstopper bugs.
• Get involved in the bug reviews yourself
• Do not start new testing that has not been performed before in the cycle

Managing this period well will improve your company's and group's predictability which will in turn enhance your chances for success.

More later ...

Effective execution

Here is an interesting checklist that can help you determine how effectively and how efficiently your organization is executing. The answer can obviously scale by size the size of your organization. This list is not in any order and is by no means complete but it is a good start for any sized organization.

Are test developers separate from test executers?

Do code developers develop automated unit tests?

Is functional test greater than 70% complete against absolute best?

Is integration test greater than 70% complete against absolute best?

Are there clear owners for each release/project?

Do you have proposal phase?

Do you do a proposal phase actual signoff?

Do you have a definition phase?

Do you do a definition phase actual signoff?

Are there preliminary product requirements in the proposal phase?

Are there rough engineering estimates in the proposal phase?

Is there a target release date in the proposal phase?

Are there complete product requirements in the definition phase?

Is there preliminary engineering (functional/design) specifications in the definition phase?

Is a criteria document (beyond no critical bugs) defined in the definition phase?

Are there committed detailed schedules in definition phase?

Is the more than 90% of testing automated?

Can you do a full test cycle in less than 7 days (except duration)?

Can and do code developers run an automated test suite before checkin?

Is there a 12 hour representative test (full function, representative integration)?

Do you require full functional/design specifications before checkin?

Do you require full unit test pass before code complete?

Do you have predictable releases?

Do you require code reviews before checkin?

Do you do nightly builds and pass your 12 hour test?

Do you allow adequate time in integration achieve 90% of the criteria document before release?

Do changes in the release plan require signoff and revised schedules?

Do you have at least 75% confidence in all of your schedules?

The more of these questions you can answer "yes" to, the more effective your team will be.

More later ...

Work Harder, Work Smarter, Skip Steps ...

Everywhere I have worked people ask me to have my teams work harder or work smarter. What does this mean and how can you do it?

Unfortunately, we as engineers sometimes translate the orders to work harder or smarter into skipping steps. I have been guilty of it myself. You find a piece of a project that needs to be re-architected in the middle or the end of development and you can face it or pray for a miracle. We have all been there.

However, each time we ignore the correct steps, we pay for it in the end. Either we don't make the dates, burn people out and lose them or incomplete or buggy products to the field.

You should employ smart people and you should employ smart processes. Take the time to do requirements documents, specifications and code reviews. Take the time to get sign-off from all of the stakeholders. Take the time to do it again if things change.

Don't get me wrong, you should challenge your teams to get it right the first and you should challenge them to meet their commitments. You should challenge everyone to act as a team and pitch in when things go wrong. Just don't throw out the baby with the bath water. The fun part of software is coding something and seeing it work. The rest of the stuff I mentioned above is like brushing your teeth or paying taxes. Only your leadership will insure the hard parts get done.

The smartest way to work is to take the time to do good engineering. The effort you and your team will need to employ will differ from project to project and company to company. Take the time to figure out what you need and then help your teams accomplish their tasks and hold them accountable.

More later ...

How do you handle the end-game of a release?

So you did everything right. You defined your product and acceptance criteria well. You focused on the task at hand. You followed good processes for design and development. Now you have hit code complete. How do you get from here to a released product? How do you manage your resources and schedule?

Everyone has their own names for various phases. So that you understand what I am referring to, let's start with some definitions (the following definitions explain what you should have at the end of each phase):

• Development: design, design reviews, coding, code reviews, test code development and test code reviews are complete. The features basically work and the tests have been run but bugs may exist. You have met the Alpha entry acceptance criteria.
• Alpha: All criteria for General availability are met. Ready for field test. Beta documentation is ready.
• Beta: All criteria are met and the product has been field tested.

At the end of Beta, the product is ready for General Availability. Some companies have a Limited Availability phase before General Availability to augment their field test. Many customers will not field test Beta. A Limited Availability Release provides a production product with expectations that the product needs more field exposure.

The aforementioned phases occur regardless of what you plan or what you call them. Some people release final product to the field that I would call Alpha. The product still must go through the rest of the phases to achieve the stability expect in a released product.

Do not enter one phase until you meet the acceptance criteria for the previous phase. If you do, you will just be fooling yourself and slip later on. It is near impossible to make up time during these integration phases.

How do you size the time required for Alpha and Beta? A number of factors should be considered including:

• The number of features added to the product
• The amount of foundational change in a release
• The length of a QA cycle
• Efficiency of the team at addressing issues
• How complete the product is from the previous phase

This paragraph provides some guidelines for scheduling Alpha and Beta. If you have a product that can be robustly tested within a 7 day period and you introduce 10-30 moderate-sized features, then the Alpha time would be from 4-6 weeks and the Beta time would be from 6-8 weeks. For a larger product of 30-100 features, an Alpha cycle of 12 weeks and a Beta cycle of 24 weeks is more appropriate. Clearly the above factors will modify those times.

From here it is a matter of managing the test cycles, field test cycles and the bug list.

You must make sure that your test team has viable test candidates. Try to not sequentialize test (i.e. don't delay doing one test because another did not work). Make sure the test information is clear and concise. Report progress to the whole team. You should consider any bug stopping the test team from doing their testing, as the highest priority bug to fix.

Someone must drive field test. Start it as early as possible (even during Alpha). Make sure you are using your own products where possible. There is no substitute for real world testing. If you cannot accomplish this, you may have to do a limited availability release (see above). Make sure there is enough time to get feedback from the field test and incorporate changes. Send someone to customer sites to get them up and running quickly. Define what you need them to accomplish. Select your Field test sites well. Avoid using pre-sales customers as field test sites.

The final thing is the bug list. You need to agree upfront about what your criteria is. Do not wait until the end to negotiate this. Do not use Alpha or Beta time to fix bugs from the previous releases as they should have been scheduled during the Development phase. All releases have bugs. When you classify bugs, the only piece of information that is important is when the bug needs to be fixed by:

• Development complete
• Alpha Complete
• Beta Complete
• First Patch or Bugfix Release

Your product team including product management, engineering and support should help determine when or if a bug must be fixed.

You need to track the trend for your bugs. The three critical pieces of information are:

• How many bugs can your team fix per day
• How many total bugs are in the system
• How many new bugs your team is finding each day

With this information, you can determine whether you will make the schedule. You can also set goals about where you need to be during each week of Alpha or Beta. The product team can determine the cost/benefit trade-off for fixing each bug. If you cannot make the schedule, then you should ask your team to work nights and weekends to help try and get back on schedule. This is typical and should be expected for the integration phases of a release. If the trend continues, you will need to adjust the schedule to accommodate improving quality and you should examine your development practices so that you can do better on the next release. If you are experiencing a lot of bugs or as you get close to a milestone, conduct daily meetings with your direct reports or full staff as needed.

Once your release is stable enough to reach Beta, you need restrict the type of bugs you allow into a release. You must assume that any bugfix is as likely to introduce a new problem as it is to fix an existing one. You must also provide more restrictions and scrutiny on changes you allow your developers to check-in. More rigorous code reviews and management sign-off is common.

Finally when you reach a point when you have your golden bits that you wish to release, you must have rules about what to do between the time you ok that release and when it gets into customers hands. You will always find one more thing you can fix. So create a rule that says any bug found after the release is signed-off must go into the first patch or bugfix release (unless you would have stopped shipment for the bug in an already released product).

It is easy to slip releases in Alpha and Beta phases. There are always exceptions to the above process but you should remain focused, watch things closely and adapt.

More later ...

How do you develop predictable schedules?

Let's go back to basics. I believe predictability is the main reason to have schedules in the first place (see my earlier post on schedules). The only way to get predictable schedules is to pick good tasks. So how do you pick good tasks?

Here are the basics on how to do it:

• Pick tasks of reasonable size (10 days maximum)
• Make sure the result is measurable
• Legitimize design time, test development time, integration time, etc.

If the duration for your tasks are too long, then you will not know your task is running late until it is too late. Slips in long tasks cause slips in releases. The only way to mitigate this risk is creating smaller tasks. This may require more scaffolding to test intermediate results. It may require longer schedules. If your goal is predictability, you need to tell your team that predictability is a priority for you and it is ok to identify shorter tasks, even if it results in a longer schedule. Longer schedules alone will not provide predictability. You need smaller deliverables.

Each task should have a measurable deliverable. Without a measurable deliverable, the schedule and the task are meaningless with respect to predictability. It is easy to create a task whose description is "Coded new APIs" but at the end what do you have? All you have is that code. Does it tell you that you are finished? Does it tell you it works? What if someone finishes the coding but it takes twice as long to get working? Identifying a separate integration phase does not help either because you cannot tell how far along you are. Instead I suggest a task description like "coded, developed tests, code reviewed and checked in 5 APIs". If the original goal had 15 APIs, then I would create three of these tasks (or whatever fits in 10 days). This sends a clear message that the task isn't done until it includes all of its pieces and is measured.

Finally, make sure that tasks include explicit time for design, code review, etc. It is easy to either slip because of these items or worse skip them. They are not the most fun tasks for engineers, so make them count. Without these items, it is easy to slip a schedule late in the game because of a lapse in up-front hygiene.

This is hard and your teams will need both practice and help to succeed, but it will payoff in success and rewards.

More later ...