Derail?

So what is the cause? Again, the miracle really didn’t occur.
In agile transformations, we have noted that one pebble can derail the process. Think of a big round rock at the top of a hill. The rock is the transformation. The hill is the organization. It takes an amount of effort to begin to move the rock down the hill to start with, but one you over come the transition from potential to kinetic and get it rolling, usually momentum will take over and the process will continue to flow. However, to be successful, we must continually sweep the path like the sweepers on a curling team. What we find is that if our rolling rock of change hits even the littlest pebble remaining on the path it will deflect off the path and the transformation will at best continue forward but with an unknown trajectory.

Acknowledging that it is far easier to derail the transformation than it is to instantiate and maintain, what can we do to increase and maintain our probability of success?

Change Agent

I have been a change agent and a process consultant and I have stood before well-dressed executives and stated clearly and with conviction: If you achieve this goal state, all forms of frankincense and myrrh will come to you. And they have believed me.

The cost to paint the As-Is was low – it is a paint by number. The cost to portray the To-Be vision was higher – we consultants are paid for our insight and experience. The expense to connect these dots is exorbitant, not because it is where the money would be made, but because it also required that certain guarantees could be stated: principally that the transformation could happen. Business Process Reengineering always seemed to happen, but unfortunately the To-Be never seemed to achieve its promise to anyone other than the consultancy. They made their margin and then some, but few clients ever really reached their desired outcomes.

Itsa Miracle!

We so desperately like to focus on the As Is – To Be transformation that we minimize the piece in the middle, the actual transformation. The current situation is manifest through its problems and through its inability to allow the organizations to achieve the goals set out for them by their management. It is often easy and straight forward to speak the To-Be – the transformed state. We look at our problems and imagine them solved, simply and directly, and there we are. Indeed, many a consultancy has prided themselves on the speed through which they can identify the exact resultant state to which the client should aspire and change. Unfortunately, the rub is that, when asked, the aforementioned miracle rarely happens.

Ch-ch-changes

What if everything you assumed about how and why organizations worked was wrong?

What if everything you knew about how and why organizations transformed was also wrong?

Hybrid matrix approach can take the best of both flat and hierarchical and perhaps leave out the worst

This approach can produce a self sustaining nurturing organization that is both self healing and strongly mentoring supporting both rebuilding and strong growth environments.

The agile hybrid is established by the creation of a matrix organization where there is a flat discipline-based groups that are loosely structured on “guilds”. Instead of a manager for the group, you have a Guild Master who is responsible for the technical competence of the individuals in the group (specifically their hiring and maturation as technical gurus). The masters ‘review’ all of the individuals in their team, even though there may be well over 10. They do this by outsourcing the responsibility of 360 degree reviews to the projects that the individuals worked on. In an agile organization, this becomes an ideal model because you match the review cycle to the release cycle and do the reviews every quarter.

Projects in this type of organization become the verticals and they have full time masters of their own – project managers (scrum masters with delivery responsibility) and product owners. As part of the project chartering activity, a team is derived and then shopped to the guilds for fulfillment. When there are conflicts in desired talents or capacities, a portfolio manager will be needed to adjudicate disputes. The product and product managers are responsible for the day to day operations of the project and though the people assigned are not really part of the project, they report to the projects for the duration of their assignments. There is no multi-tasking between project as this dilutes the effort even worse than trying to get organizational managers to do real work.

This third model allows a minimization of overhead in the organization and a maximization of skill-based throughput. There is still a degree of finesse necessary to manage and maintain the organization. One interesting point is that the guild masters are typically the most technically advanced people in the org and yet they do not actively work on any project. Their success is the creation of journeymen who are every bit as good as they are and who can do the work with the level of expertise that reflects the guild master, yet giving the master more reach by extending his capacities. If the master’s 2nds are not operating in the same order of magnitude, the guild masters are not doing their job and the organization will never be able to scale.

More on Guilds

OK, great. Whatever. Even in BF Skinner’s Walden Two, neither society nor society are this open. This is a great ideal, but it dissolves the corporate veil. The organization exists to serve it’s constituents and in most organizations, guidelines, guardrails, and limits are placed. OK. So how do we reign this in and still be true to the vision?

A flat organization is managed as a guild based organization. Guilds are mean cruel and archaic structures. They rose to power in the medieval era and are alternatively seen as the epitome of Adam Smith and Karl Marx. It is argued that guilds stifled trade, inhibited innovation, and distorted the distribution of wealth; or completely the opposite. In fact, all craft guilds pursued pious goals. In addition to the regulations governing their crafts, guilds were both benevolent and religious societies governed by strict rules for mutual aid, arbitration and the procuring of spiritual benefits*.

Even granted the separation of Church and State and there is much that we can take from the guilds. There is also much we can leave. There are also some things that rise from the operation of guilds that is worth noting, but only so that we can recognize bad behaviors and nip them in the bud. One such behavior is the way guild masters both poached promising upstarts and colluded against others masters in the same guild**. So, leaving out organized religion and collusion and some other practices that were frankly reproachable, what do we actually have left that is not so left of center that we end up with technical communes (though these may actually not be a bad idea – think Semler***)?

The guilds saw achievement of mastery in their craft as a means of achieving piety. In the Medieval eras the piety was to a deity. Though much has been written in western literature about the Christian based guilds, the Eastern religions used the same construct and organization. If we translate the piety into reverence to a product or technology vision instead, we immediately have a solid relationship to our businesses.



* Gary Richardson, “Craft Guilds and Christianity” and Sylvia Thrupp, “The Merchant Class of Medieval London”

** From Craft and Christianity, p155:
As long as the masters maintained a solid front, all of them profited from the pact. Journeymen were forced to accept lower wages. Journeymen had few other options. But, masters had trouble maintaining the pact. Collusion was in the collective interest of all masters but not in the personal interest of an individual master. One of the masters – perhaps bit smarter or greedier than the others – would realize he could cheat his colleagues just as he could cheat his subordinates. After the other masters had lowered wages, he could offer remuneration or working conditions slightly more attractive than the collusive level and hire the most talented journeymen. His shop would be productive and profitable. Other masters would observe his success, and then try to outbid him for the worthiest workers. Bidding would escalate until the cabal broke down and wages returned to the market clearing level. In other words, the masters could not carry out their self-serving plot unless they could (a) determine who was adhering to the pact and who was not and (b) reward the former or punish the later, so that all masters had an incentive to maintain the cartel.

*** Ricardo Semler – See his books Maverick or The 7 Day Weekend.

Flat organizations tend to be more skilled and more able to foster innovation

But flat Organizations rely heavily on srong self-motivated and self-regulating individuals – basically a primadonna culture.

If an organization has one level – i.e. it has a flat organizational model, individuals use capability to distinguish themselves. They become more adept at focusing on developing their skills and advertising their capabilities, giving them (at least the successful ones) more clear stature in the company and industry. For companies that require innovation and the circulation of new ideas in their product development operations, this enhances their competitiveness in the market. Additionally, since one’s prestige and stature in the organization becomes driven by their skills, capabilities, and the advertisement of this both internally and externally; these organizations tend to be able to recruit considerable top-notch talent.

Equally, flat organizations are harder to oversee and more complex to maneuver through. They require a broader approach to management as well as a lighter touch so as to not stifle creativity. In a flat organization, the only way to keep a control on direction and activity is through mutual respect and alignment to a common vision. Otherwise, pockets will develop that may have fascinating results, but dissipated strength and local optima will develop. If these are neither harvested nor eliminated, precious resources will be spent without furthering achievement towards the organizational objective. In a research based organization, this is ideal as it allows the individuals (or teams) to explore venues through to their natural conclusion. DEKA’s production of the Segway is a great example of this

Management in a flat organization needs to be done with thorough peer review and mentorship. Everyone, regardless of their seniority should either choose or be assigned a mentor. As individuals gain capability, they may choose their way through continuous learning with different masters – stay with each mentor long enough to gain what they feel as their appropriate level of awareness in a field. As people excel in one direction or another, it is the role of the mentor to encourage the individual to stay with the discipline and continue their journey. Frequently, people will make poor decisions in other’s eyes, but as soon as fences are constructed, hypocrisy becomes established.

Hierarchical organizations are recursive and lead to command and control behaviors

This is certainly not my first choice for a transforming agile company.

In a hierarchical organization, small groups tend to be calved off the larger organization until there is an entire tree structure of pods of people with like backgrounds and objectives. Modern management theory suggests that one person can adequately control/manage no more than 10 individuals, so for larger organizations, trees are built such that each has about 10 people and an manager, who is themselves an individual in someone else’s group.

These pods are typically like talents because the approach to management has evolved to be a high touch interface where the manager has frequent interaction with the direct reports or both status and planning of both their activities and their growth (we must note that growth usually takes a back seat unless the individuals do it on their own. Given the high touch, it is equally unrealistic for the organization to assume that the individual will participate much in actual work. Granted this, we tend to note about a 25% drop in productivity for every level of hierarchy that a manager has to manage – A first line manager will have a maximum throughput of 75%, second line: 56%, and so on. After this, it is really silly to expect that a manager will really be able to be productive at anything other than management or control.

Based upon this, it is clear that the flatter the organization and the stronger your self organization and management capability is the more productive you will be as an organization. When you couple the fact that managers tend to make more than their staff, and 3rd line and above do not do productive work, becomes clear that you lose economical edge to the flatter org as well.

Organizational Changes: from Command for Control to Agile

Most organizations tend to adopt a standard hierarchical organization structure. This is one where there is a CEO who has an executive leadership team reporting to him, each individual on that team themselves have a duplicate of this structure for their teams and so on down to the bottom. In addition, there is typically another set of structures for project within programs within product lines, all recursively reflecting the structure. Regardless of the goals and the objectives of the organizations, or the products that they build, the goal of most individuals in these organizational pyramids is to rise to the top so that they can jump off into the next level. Imagine, if you will, any large organization being an inverse Mario Brothers dungeon – with perils increasing as you jump up, from level to level rather than the reverse (but the same theme music blaring in your ears as you run!).

People in the organization are subsumed by this goal – that of reaching the pinnacle, assuming that the princess is there somewhere – rather than achieving the goal of product or project release. Basically, their goals and the corporation’s goals are not aligned. Those that have wanted to innovate or create or even release have left to go other places where it is easier. Those who are interested in accession have remained. I find it mildly humorous the number of principals in the organization who will state that their desire is to become a manager, even though they dislike dealing with people and would prefer to contribute more “wholesomely”.

Clearly, an alternative is to eliminate the prize at the top of the pyramid. That is hard, because America and its capitalism is all about the rewards of the powerful. But, once we acknowledge that this is what we are all about, we can basically help people find power in niches in which they both excel and that align with the overt goals of the organization. Reward people for contributing rather than only managing. Create an organization where power is represented through disciplinary leaders rather than controlling directors.

OK, so let’s recognize the differences between organizational structure options and then define a hybrid that works here.

Where to go next

The objective would be to eliminate the friction between the organizational layers so as to remove as many of the pebbles as possible. The two changes I would make first and foremost would be aligning the organization reporting structure to the agile values of the division with the elimination of the command and control hierarchy that we currently rely on and second the establishment of an agile portfolio management structure to emphasize the concepts of transparency and eliminate the perceived need for the current task forces with their attempt to “drive” projects to completion by the perennial death marches.

Post Sisyphus

Another analogy worth thinking about here is that of a rock (a really large one) rolling down hill. If that rock is our agile transformation, its momentum is what we are most interested in. At the beginning, it was standing still, stuck in the dirt on the top of the hill. It took a considerable force to get it unstuck and moving (in the right direction); however, once it was, common wisdom suggests that it is not easily stopped. Unfortunately, the reality of the situation is that it actually takes fairly little to derail the change. Imagine, as our rock is rolling down, picking up more and more speed (and therefore momentum), it hits a little pebble. Depending upon the momentum we have at the time, the three possibilities are:

1. Our rock stops, lodged on the pebble
2. We roll right over it as if it was not there
   
3. We hit the pebble and go bounding off course
   

What happens in each individual case is completely dependent upon the momentum the transformation has achieved at the time hit hits the bump – too little and we stop altogether; more than enough (the process is well institutionalized) and we pay no notice to it; everywhere in between and the unpredictable (and unpleasant) happens.

Most organizations undertake transformation as result of unacceptable performance. Rarely can you justify perturbing a machine that is adequately delivering ‘stuff’, so it follows that for anyone undertaking a transformation there are enough issues that arise so that their rocks appear to be rolling through a bed of gravel. My strenuous recommendation at this point is to first push for organizational alignment between the layers as that tends to reduce or eliminate most of the immediate irritants and will allow us to purge the resentment and re-establish trust from individuals and create teams. From there, regaining momentum on the transformation will be much simpler and indeed internalization of the agile methods at both the operational and the organizational levels will dramatically increase throughput of the teams and collaboration between the teams in all of your ‘current flows’

Phase and Harmony

In most organizations, the teams themselves are functioning relatively well as agile entities. They have the overall concepts and practices understood and internalized. Every iteration, they go through a number of motions following the patterns they have been taught. Unfortunately, they also carry a degree of cynicism that permeates their activities, some more than others. Mostly, this cynicism rises from the desire to bring about changes we (the individuals on the teams) feel need to happen in order to optimize our efforts. I have seen indication that attempts had been made initially but since the desired actions were felt to be outside of the individual spheres of influence, the they have given up trying to achieve the change and, worse yet, believing that the organization’s desire for change is only lip-service.

Basically, the teams have tentatively pushed out of their flow and into that of the enterprise above them and some of those irritating turbulent eddies have formed. There seems to be little attempt to understand the irritants in the next higher flow and so a defeatism has developed and fostered the cynicism. The result here is that the teams go through the motions with the agile approach, but no one is really embracing it as a way of doing work. Evidence of this is the total disinterest in improving their execution of the art, the value of transparency and the unwillingness to seek goal alignment between products and organization. Instead a resentment has been born between one group and the next as well between one layer and the next. From this dysfunction, mistrust becomes more and more entrenched and it becomes impossible to reach phase and harmony, forcing the teams to always operate well below their potential.

To add to this, the resentment tends to be redirected towards the change undertaken and redirected towards agile methods themselves. Since titles and responsibilities tend to be changed without specific care given to the roles themselves, middle management feels as if much of their power and ability to affect direction has been lost. Rather than retrenching and establishing leadership from mutual respect, most of the folks again tend to rely on position and title to direct the actions of their teams and people. Clearly a left over command and control trait, but it is one that seems to be more well respected by senior management (the next flow up) than the collaborative structures that we try to base the methodology on.

Your Agile transformation is stuck

An associate of mine also in the transformation field noted that agile transformations are extremely fragile. Truth be told, all transformations are fragile, but agile ones appear more so because we celebrate collaboration and transparency and so everyone’s contribution becomes more obvious and the collective’s progress more clearly visible – or not.

Any transformation relies on establishing sufficient momentum under the new processes so that the level of effort needed by the teams to keep them moving becomes minimal and sustainable. Once organizations get over the initial hump of learning, they should become fueled by the benefits: energy put in to use the new approach is dwarfed by the value received from using it. At least this is what we hope and expect will happen.

Occasionally, the change agent gets it all wrong and the local culture will not sustain the new methodology. More frequently the methodology used at one level of the organization is enough out of phase with the governance of the organization and the resulting discord does not allow the new methodology to ‘adhere’. A good analogy here are the stratified layers in any moving currents, such as layers of water or air. Wholly within the current the flow is smooth, but where the layers touch, there is a fair amount of turbulence. Indeed, experimenting with water or air where the different currents are colored shows that whenever a random stray from one flow gets ’loose,’ eddies form infectious pockets of turbulence that swell, fester, and eventually dissipate – but only after the currents in that area are well disturbed.

The Path Less Traveled

by Robert Frost – another part time NHerite, just like me!

Two roads diverged in a yellow wood,
And sorry I could not travel both
And be one traveler, long I stood
And looked down one as far as I could
To where it bent in the undergrowth.

Then took the other, as just as fair,
And having perhaps the better claim,
Because it was grassy and wanted wear;
Though as for that the passing there
Had worn them really about the same.

And both that morning equally lay
In leaves no step had trodden black.
Oh, I kept the first for another day!
Yet knowing how way leads on to way,
I doubted if I should ever come back.

I shall be telling this with a sigh
Somewhere ages and ages hence:
Two roads diverged in a wood, and I --
I took the one less traveled by,
And that has made all the difference.

In Tom DeMarco’s book “Deadline” he tells the tale of a kidnapped Program Manager, and interesting tale, but a tale none the less. In it, the protagonist is given the opportunity to run a project multiple times simultaneously. Nowhere but in a novel do you get to travel both paths. Indeed, it is hard except in a poem to take the path less traveled because of the nuances and the mores of the establishment.

Agile used to be the path less traveled. Many individuals and groups can attest to the difficulties in breaking from established patterns sufficiently to try a new methodology and others can tell you how easy it is/was to ground a transformation in mid-flight. Never the less, ultimately it will be the agile path that makes all the difference.

What does Done Mean?

What does Done mean?

Yesterday in the parking lot two related questions were asked and I offered to give an answer that would relate them. The questions were What does “Done” mean? and What constitutes potentially shippable code?

First, let me address the easy one: What does “Done” mean?

Well, the literalist in me wants to point out that “done” is merely the past participle of the verb to do, and as such indicates the completed action of the active verb. Dictionary.com defines “done” nicely as accomplished. When we consider “done” in relation to our development efforts, there is really only one way to define it: No milestone has been accomplished until the product owner acknowledges it as such.

Since the Product Owner is responsible for the creation of any particular story, only they can determine whether or not their intent has been accomplished. Procedurally, we drive to eliminate a bulk of the subjectivity by defining acceptance criteria while we write the initial story. This approach is consistent with TDDevelopment as it concretely defines the finish line for the team before they start.

Potentially Shippable refers to a state of the system from whence it is actually presentable for General Availability (GA). Clearly there are a bunch of details that need to be completed in order to move stories from “done” as accepted by product owners to GA and accessible to external customers. Examples of these details are the compilation of printer ready pdfs from completed user documents – clearly this needs to be done once all the documentation updates are “done”, but before the product can be considered shippable.

Often, teams utilize one or more hardening sprints during which these final details are completed and while system level performance testing and training can be undertaken. Additionally, during these hardening sprints, the teams are able to address lingering issues such as defects that have not been completed or even take on a couple additional stories, as long as all changes are staged prior to being integrated into the release candidate.

So, granting these definitions, teams frequently have difficulty in applying the abstracts concretely to their projects. Specifically, what are we talking about?

1) High level stories are written by the PMs. These are narratives that walk the reader through a scenario that is obviously testable: when a scenario is run, a predetermine result can be perceived. Depending upon the complexity of this story it may be broken down into more manageable chunks, each with their own results based acceptance criteria. When the work is verified, it is verified at the lowest level – typically a task of 1-3 days duration. When the work is validated, it is validated at the highest level – typically the larger story’s scenario. The theory here is that if all the low level pieces are verified and the story can be validated at the abstract level, then by induction all is complete.

2) Similarly if we are going to verify at the lowest level, we must ensure that we have all the component packages identified at the task level so that the work it self can be inclusive. If we need end-user documentation for a particular high-level story, then there should be stories and tasks associated that cover these activities. If we have an end-user, we need to incorporate useability stories and tasks. Ideally, a check-list should be created to help the teams ensure they have included tasks which cover the entire spectrum of activities that are necessary. There is a caveat however; the spectrum of activities must correspond directly with the acceptance criteria put forward by the PM when the story was initially written. If they don’t feel a certain aspect needs to be incorporate, then don’t. On the other hand, if the team really feels it does, they should initiate a dialog with PM to determine whether they have discovered an omission, purposeful or not.

Thus, Done means the stories are verified by QA against their acceptance criteria and there are no lapses. Potentially Shippable means that all high level stories are validated by the PMs and demonstrable to the organization as complete.

Tying Release Planning to the Critical Chain

Another conversation with my friend Tom Looy caused me to start thinking about project networks and how they relate to Release Planning. A project network is a construct of tasks or work products (which are groups of tasks) that link together to form a mathematical graph representing a logistical map of the project. Ok, what does this really mean and how do MMFs and IFMs work in to this (as you know they must or I would not have brought them up!)

Well, first a bit of background – a project network is used to identify a project’s Critical Chain. As initially defined by Eli Goldratt, a project’s critical chain is its manifestation of the Rope from the Theory of Constraints’ Drum-Buffer-Rope concept. It is what is tying all the operations centers together with their dependencies and precedences such that the “right thing” will pop out the back when the project is over.

In most undertakings where TOC or CCPM are utilized, the “right thing” is pretty well defined. In many of the projects where agile methods are used, the “right thing” is generally understood, but not defined. However, there are a couple of valuable things we can take from the project network, if we can figure out how to integrate it into our concept of Agile Planning. First, we get the concept of buffers for our schedules. The buffers provide us with the ability to project out our actual releases early on in the planning and give us some flexibility in statistically committing to firm dates. Second, we get the actual critical chain. The critical chain is valuable because it focuses us on everything we need to do to achieve the objective we are setting out with, it also offers us a view of all the dependencies that need to be completed so as to allow us to achieve our real goal. Though this sounds reasonable, it is somewhat striking to suggest because the only way to establish the critical chain is to identify all the tasks we believe we need to accomplish, which is rather antithetical in agile approaches.

True though, when we create a project network what we are actually doing is creating a future reality tree for the project: if the project were completed successfully, we would have …; therefore, in order have that, we need to have … and so on back to “today we start the project by doing …”. Now certainly this does not mean that we need to identify every single action we are going to take. When CCPMers talk of identifying tasks they typically do not mean the same low level tasks we refer to: i.e. those building blocks of stories or features that tend to be chunks of work no more than 1-3 days long. What they refer to are basic components of the project that must be completed to support the implementation of the next level of the tree.

The exercise of creating the tree is more than a planning exercise – it is designed to ensure the important dependents and precedences are identified and understood so that the complete book of work can be defined, the critical chain specified and the constraint(s) illuminated. Once done, ROIs and value props can be determined and project viability understood.

Clearly this is more ‘ahead’ work than we do in an agile project, but consider if you will, the opportunity that we create if we were to take a leaf from this book: During release planning, as part of my chartering exercise, let’s say we create a high level project network. In doing so, I identify a lot more detail about the entire project than I might other wise do. However, I also produce a dependency chart and identify a critical path such that should the customer desire a change later on, we can reference it back to the network and understand what it impacts, and if so, how far back in the tree we need to start accommodating it before it adversely affects the critical chain.

All of a sudden, we can do both cost/value derivations on features as well as put in place an incremental funding model for both features and the project. If I know that “A” is in my set of MMFs, then it is probably also in the critical chain. If my customer asks to reprioritize, deprioritize it or even eliminate it, we can quickly and clearly see how the value proposition for the entire project changes. Equally, if other features are added to or removed from either the project or the set of MMFs we can understand what the impact may be. Once we start seeing the project network as a future reality tree, then the project itself becomes less abstract and more concrete both as an entity and as part of the holistic value stream of the organization.

MMFs and the IFM

There is a concept called minimal marketable features (MMF) popularized by the book Software by Numbers that looks at project/product viability in terms of return on investment. The thinking goes, at a very high level, implement those features that have a high value payback first, get them into production quickly, and they will act as an incremental funding model* for the rest of the program. The difficulty with the model, however, is with getting to critical mass.

For products that are not in new spaces, particularly if they are replacing systems already installed, there must be enough features present to not necessitate running the two systems (old and new) in parallel. For new product, there must be enough features present to make a compelling sales story. In both of these situations, this number is the set of MMF and truth be told it is always smaller than anyone is comfortable admitting to.

Think about MS Word. Hey no! That is unfair, everyone picks on Microsoft and this blog is a Google hosted blog. Let’s pick on Google! Blogspot is a great equalizer. It has allowed every Tom, Dick, and Matt to become published bloggers wherever they want in the world wide blogsphere. That alone has great value to some of us. The html editor here on Blogspot does not have all the features I use on my word processor of choice (MS Word). I can not do foot notes easily, for one. However, I can easily publish a blog that other people can read and where I can store ideas until later without fear of losing them. I can get other folk’s comments on them and do some rudimentary collaborating. These are the highest value features to me. I am actually glad that they did not wait until they had footnotes working before they made this much of the system available to me. And, as you can see, I have found a perfectly viable way of doing foot notes without any additional special editing tools.

If you needed to create a list of the MMFs for a blog editing and posting tool, most of us would have put down a word processing capability. What we really meant was text editing. Using the MMF model, the team could implement text processing capability, gotten the site live, then come back later and upgraded the system to a word processor (or not, maybe it is not actually ever needed).

The one drawback to MMFs that causes them not to be used as much as they should is exactly what they were designed to support. Laying out the MMFs both encourages and enforces some rigor about the value proposition of your features. Often times this is hard to make objective. Think about Blogspot again. If their goal was to corner the market on bloggers, what would be the most important feature? It actually is not any of the functional features of the site, but rather the ease and cost of entry into the entire concept. In the early days of blogging, you had to both buy blog software and sign up with a hosting entity upon which you could blog. This took money and commitment. Now, on blogspot, it takes seconds to create a blog and thusly the ease is high and the cost is zero. There are now literally millions of blogs out there that get started and never updated because it is free and simple to initiate.

I wonder, though, did anyone sit down and define the value to the organization of this feature? Monetization of Blogspot is similar to the rest of Google – ad revenue. If there are Billions of blogs that are never updated and consequently never looked at, their ad revenue is going to be negligible, but they still take disk space and infrastructure. Thus, ease of entry allows them to capture the market, but it may also have a negative impact on the organizations costs. These intangibles are much harder to equate to specific numbers and so quite challenging to integrate into the MMF model.

In the early days, before Google bought Blogspot, you can imagine the strategy sessions they might have had**:
“We need to take a commanding percentage of the consumer blog market. If we do, we will further our cause for blogs for the proletariat and creation of a world wide stage for every closet politician and every person of strong moral position.”
“True to that! Therefore the most important feature must be ease of initiation closely followed by a simple approach to rubbernecking on other people’s blogs so that inherent curiosity will keep people on the Blogspot site for longer and longer visits!”
“Guys, wait!”
“We will also need to keep it simple so that the common man can use it without a lot of training or documentation.”
“Without doubt, cause I hate writing documentation!”
“Guys, wait, listen!”
“What….”
“Yeah, what?”
“If everyone signs up and everyone uses this, how are we going to create enough space for them? Where are we going to put them? I only have this one server here….”
Silence.
More Silence.
“I know, if we get it up and running fast enough and take enough of the market quickly enough, someone big will become worried and will buy us to make sure they are in the market before someone else buys us. It can be their problem!”
“Great. Ease of use and Low cost of entry! Get as many people to sign up as fast as possible and demonstrate potential and hope Scotty here can hold her together long enough!”







* I love footnotes. I wish the blog software would support them though! The increment funding model is basically an accountant friendly approach to undertaking a project whose stakeholders know how important it is, but can not necessarily express it with discrete numbers. At a basic level, it says “spot me some cash and I will show you some value. As we progress towards our final product, I am happy to take little bits of funding at a time”. It is hard to do long term planning in this way, but the idea is that you will win your CFO’s heart (and pocket book with the healthy returns) and he will release the grip he has on the purse. This is a common model for startup funding.

** This is all fiction. I have never talked to the Blogspot guys, I am only imaging what might have happened in their executive offices.

Release Planning

While using agile methods, we advocate defining a vision for the project that will align everyone and help them focus on the right underlying concepts and point them towards a shared product vision. This product vision is championed by the product owner and defines the “right thing”. When done right, we don’t need to define the vision in great detail, because everyone on the team understands* what is being targeted. The high level plan is then: develop towards the goal – undertaking only what is needed to realize the goal state, but remaining flexible enough to tune our progress as we better understand convergence towards the PO’s vision.

The draw back to this comes in the form of a question I hear far too often: How will I know when we will be done? Planning a release is critical for most organizations. There are always many costs and impacts that will are borne by tangential or ancillary groups within and without the organizations actually doing the development. These must be both planned and budgeted and can’t be taken lightly. The failure of many projects can be traced purely to the inability to integrate them into an operational environment, which can’t be done without planning and scheduling.

So, how do we ensure success? Clearly, it is simple if we timebox the release. When we get close to the end of the timebox, we move the sprints from releaseable to release and ensure that whatever we have done is hardened enough to go into production. However, how do we do it when there is a minimum number of features that need to be complete in order to create a viable release? That is a great question!




* Quick test for your team – ask any two people for their 30 second understanding of the vision / goal for the project. If the two elevator pitches agree, your team is spot on and your PO has done their job well; if not …. my guess is that you will also have problems achieving “done” on the project.

on Mura, Muri, and Muda

(specifically, from Jim Womack’s e-letter )

My first reaction is that he is pretty much right on, particularly since his name is Womack.
Second reaction is to suggest that there would have been a much simpler way to come to the same conclusion, 20 years ago as well. (much simpler way)

Remember our favorite equation: P=T-OE (from Goldratt's Theory of Constraints in the Goal). You can drive OE to its limit (0), but that causes P to reach a max at T. If we assume that the effort is spent driving OE to 0, then we probably do not have cycles to increase T at the same time. Therefore, P's max value becomes T.

Comparing to the 3Ms, driving OE to its limit is the same as eliminating waste. Most of what everyone sees as "waste" falls in the OE category and so, it makes sense. Eliminating waste therefore logically stagnates profit at what ever level the organization optimally can achieve given whatever its throughput currently is. To release frozen capacity for profit (my new byline, I think), ignore the Muda, focus on increasing the Throughput. Mura and Muri, in their combination, are the constraints of the system. True, you can't optimize the organization by focusing on the T alone, you must address unnecessary OE, but focus first on the other two and once they are converging where you want them to, you will have plenty of time to focus on Muda.

And besides, any parent knows, muda always happens last.

April Fool's ........ Not!

In an interesting flip, where as we always look at the world from a US centric point of view (IT or otherwise), with the fall of the value of the dollar around the world, there are many places with higher costs of living than the US. Check this out: Harvy-Nash Survey (use the link to download the survey - the tell-tale graph is on page 24). In case the link goes stale or you don't want to bother downloading the whole document, what it says is:

Figure 29. Top ten offshore destinations

1. India
2. China
3. USA
4. Malaysia
5. Brazil
6. Philippines
7. Vietnam
8. Poland
9. Romania
10. South Africa

India dominates the offshore league table for another year, capturing over half the market for offshore services of the respondents to the Harvey Nash survey. China and the USA are now very evenly matched in the eyes of the survey respondents but South Africa drops from fourth favoured destination last year to tenth this year. Canada drops just outside the top ten to 11th after occupying the fifth spot last year. Vietnam continues to perform strongly, maintaining 7th spot amongst much larger and more established outsourcing peers.

We may be the next decade’s Vietnam!

To Co-locate or Not to Co-locate

There has been a second argument pushed forward by IT leaders to the H1B Visa shortage: If you can not increase levels of visas so that we can bring in workers qualified to help move us forward, then we will find ways to distribute the work to the workers’ homelands. From a Homeland Security position, the Bush administration has the champagne (real imported stuff, no domestic trash for them) popping as yet another victory in their triumphant march towards homogeneity. However, what they are missing was the industries conclusion, and indeed their actions. Once we are successful at distributing work, we can do so continuously until no work really resides in the US.

There are many companies that pride themselves for accepting 1 in 100 applicants (or even higher ratios). Most companies no longer will pay for relocation and so they cast local nets. The 1 they hire may be the best of the 100 resumes they got, but consider the difference between that company and the one that casts a global net and picks the 1 best candidate and can actually integrate them productively without moving them. Which company will have a better team?

True in an agile environment (and any other), productivity is significantly better when the team is co-located. However, now is our time to examine as to whether I can, at a lower overall cost, have a better team working in a distributed way with a lower productivity but still be more cost effective than someone else’s team that is collocated in the US.

American Programmer

Yourdan wrote two books in the 90's that dealt with the state of the IT industry at the time. The first, The Decline and Fall of the American Programmer, was pretty pivotal in my interest in Process Improvement. His basic premise was that unless american programmers spent more of their time renewing their skills and staying at the forefront of the state of the art, they were going to find their jobs outsourced to software factories in other countries.

One tremendous effect this book had on me was a poignant statement that Yourdan made towards the end of the book: most american "programmers" only had execution manuals in their office (programming books) if they had any books at all and actually rarely read anything beyond language manuals. It was that comment that made me look around in my office (I had an office back then) as well as that of the rest of my team and realize he was right. I immediately began compiling a library and reading it voraciously.

His second book was the Rise and Resurrection of the American Programmer. Its message was
that we seemed to have organically heeded his call and rallied to the call. Our productivity was back up and though the drive to outsource was still there, it was mostly focused at the lower end of the spectrum - the help desks and the call centers. The book ended on a happier note.

Well, he could have rewritten them again in the naughts. As you all realize, we are a global economy and highly leveraging offshore skills, talents, and capabilities. There are reports all the time about how the education and skill levels outside the US meet or exceed those in the US. The reality is different though. There are good people everywhere in the world. There are also great people and not so good people (from both the skill and intent point of view). There are companies around the world that actively seek out the best and the brightest regardless of their location and then create means of distributed collaboration.

Regardless, there are also companies that merely seek adequate capabilities but at the lowest cost. I had a company tell me recently that they had found a way to get qualified teams of people all around the world bid by reverse auction for work they wanted to outsource. Their coup d'etat was that for a particular book of java work, they managed to find a team in Vietnam that would do the work for $2.5/hr*. This is an extreme case but many organizations now are outsourcing work because they can find rates so low that even with inherent communications and skill inefficiencies, the rates are still below what the costs would be onshore.

Yourdan's point was that if we don't make ourselves skill-wise relevant, we would be outsourced. Now, we are seeing skills taking a back seat to volume. If organizations continue to accept "good enough" (one of Yourdan's watch word phrases that, once we began to practice mid-90's, we were able to climb back in to relevance in the US) at continuously lower levels of "good", there will be no way we in the US can compete.





* When I asked whether if they would consider accepting the bid and paying $3/hr just to boost their standard of living, I was met with blank stares and an incredulous "why?". This is a different situation, one of abdicating global responsibility perhaps, but not the point I am making here.

How do we establish a new truth?

Truths are strange ducks. You hope they will be self evident, but more frequently, they are not. The biggest issue tends to be in that a truth to one person is not necessarily so held by another. Much of it centers around faith, and much of the proliferation of the truths center around trust in the one asking you to hold some concept as the truth.

So, perhaps the critical step in the process of establishing a new truth is being able to trust those that are recommending the ideas behind the to-be truths. Trust is also an interesting concept. In relationship to truths and trust, the venerable Einstein has been quoted saying “whoever is careless with the truth in small matters can not be trusted with important matters”. Stephen Covey Jr., in his book The Speed of Trust states that the principal path to Trust is through integrity and in turn integrity is seen as not only telling the truth, but leaving the right impression about that truth. In other words, you can be truthful and honest, but twist it for your own purposes. Think about the sales process in most consulting or product organizations. Often, the sales people believe they are truthful, but skate in the fuzzy boundary between fabric and fabrication.

If we are going to instantiate a new truth we, the purveyors, must retain our integrity and meet the expectations that we set for ourselves. It is inappropriate to set the bar so high that there is only a small probability of achieving is; but equally, we can not set it so low that any methodology would perform as well. Let’s us strive to be aggressive but possible, meet the expectations we set, and make them clearly above what other methodologies can accomplish. This is the way we will establish a new truth.

self evident truths of agile project management

Often I get asked how you ensure that your project is done one time when you use an Agile approach. Upon deeper probing, what I find they are really asking is how do you protect against a run-away project. There has developed a common misconception in the industry that, just like agile engineering techniques, agile project management is a cowboy approach. The view is that with continuous iterations and “releaseable” code, when do you stop and, indeed, how do you stop all that requirements-creep?

To me, what this is, is a cry for help. Clearly, the project manager is not managing the project (or perhaps even the people). Regardless of what methodology is being used, the project manager is that person who has been given the role with delivery responsibility for the project. In many of the canonical depictions of agile methods tend to tread lightly on the role responsible for project delivery: often it is implied that the team itself takes on this responsibility. Well, that in itself is fine. This makes the team the defacto PM and collectively responsible for delivery.

In Agile methods, we commonly assure customers that we are geared up to meet their expectations, but there is a non-diminishing perception that once we are given control, we don’t give back product on their schedule. Further, there is a palpable and growing fear that the development teams will estimate poorly and the product release will string out continuously to “Next Iteration” – the agile developer’s version of manana. However, if the project team is collectively the responsible for delivery – as a PM collaborative, they should be holding each other to as delivery standard such that the Release is as time boxed as the iterations and that, if the customer directs, the release happens on schedule, but with less content if necessary.

If we can establish a new commonly held truth that agile teams deliver within time boxes and that the time boxes are indeed that: time boxes, then much of the fear of transformation will, in my estimation ebb away.

Question #10

How do you get companies to follow scrum? In a CMMI implementation, there is a SEPG that is responsible for both institutionalizing and verifying the processes. This becomes a two prong effort – one to review and revise the process and the other to educate the people. Organizationally, management enforces that they must use this process across all projects, though individual teams can ask for waivers on parts of the process. The SEPG also is responsible for negotiating and granting these waivers.

Usually, any agile method is a collaborative effort. My favorite approach towards enabling an organization is to take a team of curious and energized individuals with a cross functional distribution and level their education with respect to the state of the art in agile methods. Collaboratively, we would pick a point to start from, a point of departure for the enablement - so to speak. Then with Sr. Mgmt’s input, we would pick a pilot project (or two) and begin a process implementation with the team. The pilot should be important and visible enough in the organization so that other individuals and teams would actually take notice of both success and failure.

Across the implementation, we would over socialize and over communicate so that the curious across the organization would watch and become interested. By keeping attention high, the approach would be to infect them with the desire to try something different and then as our successes become visible, encourage them to try it on their own. The expectation would be that the methodology would catch on and spread virally. It is also expected that support networks would be encouraged and grown so that when new teams needed help (whether they knew it or not) there would be both a place they could go for answers and a place from which guidance could come when it was not requested (the collaborative element). This is the truest way for an organization to follow scrum.

Once a critical mass of projects is attained, there needs to be various mechanisms put in place to support and maintain the process and separately to oversee and guide projects to their completion. This is best served by a PMO. Not a classical one, but an agile one. In a classical sense, the PMO is responsible for policing the execution of projects. In an agile sense, there is a more collaborative environment and indeed the function of the PMO is to help the PMs better execute their projects and meet their commitments. It is, shall we say, a retrospective opportunity for the company with respect to how it monetizes its undertakings.

Question #9

The book never talked about the business case of using this approach. For example, early implementation of functionality is early achievement of ROI. This needs to be shifted from a theoretical exercise to one that is compelling to the executive who will be paying for the project. I can see that one of their concerns will be loss of control.

There is a fairly sophisticated chartering activity I take projects through to get to the underlying assumptions and organizational goals. Once we lay those out so everyone understands them, the nucleus of the team creates a release plan with the business and develops the business case. This in turn is briefed to management for approval. Only with that approval does the project actually get initiated. The expectation I always set up for teams is that the critical 20% should be in the first release, because there will always be another project whose critical 20% has a higher priority that the second release of this project. Most projects will never have a second release and those few that do, will really be an entirely different project.

Once everyone understands this, we establish an iterative cadence to the development and go balls to the wall across the release pausing after each iteration only to 1) look back on how we did it and change the process if necessary and 2) ask the business if the assumptions made in the charter still hold and/or whether anything has changed at the business level such that we need to replan.

Question #8

I can see the CFO of a company having heartburn with this approach since it implies a never ending project and budget requirements. (they pay our bill)

THIS IS THE KEY!!!! You found it after reading only 1 book too!

But really, this here becomes the rub. The neverending story. I fight this all the time. This is why I start with programmatic release planning with probabilistic dates for content specific releases. Once you start down that path and use both burnups and burndowns, it is much easier to contain and control a team and a release. Most CFOs are not willing to suspend disbelief and have heard so much bad stories that it is indeed a hard sell – this is why we need to talk to the CEO instead. Take a look at this: CEO to CIO and back. It is from my other blog and rather than repeat the thoughts here, go there and take a gander. It was a fairly cogent way of expressing the thoughts.

Question #7

We need to have regular internal training for our SCRUM Masters so that we are always at least a little better than the clients we do business with.

Scrum masters do not get better with training – that is a misnomer that the world is still catching onto. Scrum masters get better with experience. When you stop having delivery responsibility it becomes really easy to excuse unsuccessful situations as the fault of other forces or individuals involved. However, in many cases, there will be times where the scrum master might choose a different path if they have responsibility for the delivery as well as the process. Those are the times and the areas that need to be investigated. Without the responsibility, you lose your edge (but inversely you increase your hourly rates – go figure!).


Our internal resources are always better than the client’s because we will have pulled them from the experienced part of the industry, rather than those folks who have merely taught the skills. I want people on my team who can say “something like this happened to me and this is what I did and why it worked or didn’t” I don’t want someone who will say “the book suggests that we do this, I can call author and ask for clarification if you want, but…” People whose only reference is the advice and direction of a book or teacher, no matter how good the book or the teacher, will be one dimensional. As my mom used to say, you catch more flies with honey than vinegar, but until you see a village consumed with e-coli, you don't really understand why you should catch those flies. Anti-patterns are hugely more valuable than patterns for teaching. I want my coaches to know the anti-patterns so that when the teams start falling into them it can be realized quickly and they can be hauled out before disaster strikes.

Question #6

The book seems to make the assumption that everything is development work and only the minimal documentation required to develop should be built. I would argue that this approach will be disastrous for maintenance. I’ve seen groups have to totally rewrite a new system because it wasn’t maintainable. Some documentation isn’t for the team. It is for the next team.

You are right. As I said, the problem with many organizations' attempts at Agile is that they religiously follow the books. In turn, the books simplify the approach so that it is easily understandable and easily implementable, though if you want to scale it, you must understand how Agile works at a deeper level (understand agile). Then, you must be able to abstract it up a level or two and if you don’t have an experienced coach, you must be willing to accept some trial and error in your path to effectiveness.

This being said, agile is all about doing the least amount of work necessary to achieve the acceptance criteria. If documentation needs to be done, regardless as to whether it is end-user, regulatory, or interim step documentation, it needs to be incorporated into the stories as acceptance criteria and no story will then be considered done if its requisite documentation is not complete.

Question #5

SCRUM talks about “sprints”. How about carrying the analogy forward and awarding ribbons at the end of each sprint. Blue if the target was achieved, Red if the target met 95% of expectations, etc. The point is that the set of ribbons that a team receives overtime become their scorecard of achievement. It is management’s job to ensure that they are successful (remove barriers, provide training, etc.) It also makes the team accountable to achieve results.

Agile tends to look at the completion of stories (or usecases or features or units of work) as a binary event. When the story is written, an acceptance criteria is defined that meets both the Verification requirements and the Validation requirements. This being said, during a sprint - or as it is more generally called, an iteration – things can be marked as done only if both QA and the Business agree that Verification and Validation have been satisfied.

At the beginning of the iteration, the team committed to completing a certain number of items. The burndown charts how they are doing and allows both the project team and outside observers to wonder out loud whether they are going to be able to meet their commitment or not and if not, what they can do to help facilitate meeting it. It begins giving you leading indication immediately and sponsors such conversations much earlier than with other methodologies.

Additionally, there is another graph called a burnup that tracks the teams progress against its release target. The two charts have to be used in concert to truly help control the project, as the team uses only the burndown, they can continually adjust the level of their commitment to the point where they are not really taking on much if any new work; however they are indeed meeting it each iteration and winning kudos from their management. In this situation, the Projected Release dates looms large only to find that most of the work is not done. By trending the number of stories “completed” after each iteration against the goal set during the project’s chartering process, the organization gets a much better picture of the actual productivity of the team.

There is another derived metric that has much controversy about it in the agile community and it is Velocity. I use velocity as the effectiveness of the team on the project. This is a fine distinction and needs to be made. People do a lot of stuff at work, most of which is important to the business at some level, even if it is otherwise classified as overhead. We encourage teams to not multi-task so that we can eliminate costly context switching times, but sometime it has to happen. Therefore, when we look at velocity, we are looking at the effectiveness of the team on this project. Low velocities are always cause for investigation and discussion, but it MUST be remembered that low velocity is often not the fault of the individuals, but rather the environment they have been cast into to execute the project.

Now, if you have the team estimate their work in a unit called Ideal Hours, then you can create a ratio between Ideal Hours and Actual Hours (duration), which gives you a measure of the effectiveness of the team. Ideal Hours are uninterrupted hours an individual or team can work on a task. Uninterrupted means no phone calls, no IMs, no email, no meetings, nothing that is not directly associated to the task at hand. When we train this, we talk about coming into work on the Saturday they were doing the upgrade to the Exchange Server and the PBX at the same time, so you got to sit in front of your workstation completely uninterrupted and hyper-focus, getting done far faster than you imagined possible….. This is the same concept that TOC/CCPM refers to a aggressive but possible estimates, or estimating at the 50th percentile for probability of completion. There is indeed an entire art to estimating, and the agile approach incorporates many approaches that actually produce a far better over estimate than any other methodology – we can talk about those separately.

Once you have the estimates in the Ideal Hours, the ratio is developed against durational hours of the team (these are all team measures) where you compile the total number of hours available to the team minus their preplanned vacation, training, or other non-work commitments. The Velocity measure is usually, for a highly functioning team, between 25% and 45% depending upon its size and whether or not it is distributed (collocation increases velocity). Velocity is also a function of the complexity of the project and the degree innovation plays in the ultimate results, so you would expect a highly distributed project with a high degree of technical difficulty and a fair amount of innovation to have a much lower velocity than a legacy conversion. The skilled coach or management team will understand these factors from the project’s charter and will acknowledge it, but the real value is as a base line factor. If the team varies substantially from expected velocity values, the work is potentially more complex than they thought or they are deviating from the desired expectations from the business, or everything is fine and great things are happening – regardless, a question should be surfaced and answered.

So, back to your point. All of that is correct. You do need to know what you are actually measuring though, before you start rewarding. But the ribbon idea is great - a blue ribbon sprint as versus a red ribbon sprint!

Question #4

One of the ways to help management feel like they are in control is through metrics. Schwaber's book suggests 4 charts or graphs to manage agile. We probably need more, and more focused on achieving business value. I think I should merge the “Implementing Goal Driven Metrics” class that I teach from the SEI into our implementation of the methodology.

I agree on the merger. I am not 100% convinced we need more than a small number of metrics to adequately control projects nor to drive transformation. Take a look at the my paper on Metrics for an Agile PMO. The Burndown chart is extremely powerful, as it tracks estimate to complete, and probability to complete as well as commitment and plan. It can also be broken down to display development, validation, and verification as three separate measures.

This is one area where the concept behind the balanced scorecard got it right. I believe that agile's Transparency practice helps us figure out how actions inter-relate and roll up through the various decision making levels within an organization (typically, ultimately to the Finance with profitability). I think as part of the methodology, we should recommended a balanced scorecard type metric program to align the entire organization (not just development or engineering) and allows them to put in some collected metrics at the bottom and demonstrate how they roll up to the Red/Yellow/Green/Blue indicators at each successively higher level. This would help folks implement methodology based on value proposition and indeed demonstrate that value prop explicitly.

Question #3

There needs to be a balance between control (old style management) and empowerment (agile). The balance may be a minimalist approach to the CMMI. Only put what is necessary to achieve the CMMI in a manner that enables empowerment. I think (especially for an organization getting started in this approach – or with a older workforce) that management needs to feel like they are in control (their performance payment (bonus) is probably dependent upon this). The CMMI can give them some level of control.

One of the first steps I take during a transformation is to understand what the compensation practices are at various levels of the organization. Bonuses and other compensation mechanisms are how professionals are ‘graded’. The age old saying “Tell me how you will measure me and I will tell you how I’ll behave”, is highly accurate. If your organization aligns its compensation plan so that its people are incentivized to try and maintain a transformation (or at least the transformative practices), then the probability of success goes up dramatically. If their compensation plan is orthogonal to the transformation, you can kiss the transformation goodbye.

There is a strong attribute of agile that stands out here, and that is the practice of transparency. By both teaching and encouraging teams to be transparent with their projects and programs, then they can keep their empowerment but the control issue tends to dissipate. By ensuring that organizations out side the project team have access to up to date status and information, then they can maintain the level of oversight that they feel is warranted with out having to resort to a fully command and control mode of operation.

Question #2

2. For some companies, there will be resistance to SCRUM. How close is it to lean project management?

Lean Project Management is a separate approach. Clearly Lean and Scrum are related, Lean has been adapted fairly successfully to SW by Mary Poppendieck (see attached), and though there is a fairly active consulting practice based around it, the methodology has not taken off as Scrum or Agile has, due to the fact that it is more complex to implement and requires more rigor and insight to initiate in any organization. The seven principles of Lean can be simply associated to both System Development (no surprise) and software development (with more abstraction). They are:

• Eliminate Waste
• Amplify Learning
• Decide as late as possible
• Deliver as fast as possible
• Empower the team
• Build integrity in
• See the whole

From their titles, their association to Agile is easy to see; however, once you start trying to create the actual linkages, abstraction really becomes your friend. I like Lean a lot. Mary and I have had many a conversation on how Theory of Constraints ties in with Lean – I think both of us have enlightened the other and though Agile seems to have become the vernacular, TOC and Lean I think are far more powerful in producing substantive improvements to the value chain. Where I have been striving for the past several years is an integrated methodology that combines all three so that the resultant approach is simple to understand, intuitive to execute, and productive as all get out!

As an aside, Capital One initiated their transformation with a desire to reduce time to market of their projects and started with Lean as a methodological framework within which they used Agile and Scrum practices as the implemented procedures

10 Questions generated from Schwaber's Agile Project Management with Scrum

These questions or points were raised by a friend of mine who is an agile novice but has a lifetime of Process Improvement experience and is one of the few CMMI HM Assessors in the country. I find them quite interesting. The responses are mine.


Question 1: Agile makes a lot of assumptions about worker’s behavior. This may be true for some organizations. When it is an older or culturally ingrained organization or has lots of older workers, the SCRUM approach will probably be a challenge to implement.

Many culturally ingrained organizations immediately realize that this methodology is not for them and never attempt the transformation. A subset of them, typically the most interesting subset – such as the Travelers, Sabre, The Gap – realize that they are becoming irrelevant in their industry and must change something. The decision is usually made on the Business side or in Sr. Mgmt. Some how, someone latches onto Agile and decides it is the right path. Rarely does anyone consider whether or not it is a cultural fit for their groups.

In situations like this, corporations that attempt to do a transformation on their own usually find so much active and passive resistance that they are not experienced in over coming that they fail quickly. In organizations that bring in consultants to help with the transformation, I have still seen a fairly low rate of success. I attribute this to the fact that though the consultants can address and over come the active resistance, they are expensive and the passive resistance can and does outlast them. Once the consultants are gone, there is a pretty high probability that the culturally ingrained approach will resurface and the transformation will wither, sometimes leaving the company unable to develop at all.

The Agile PMO as a means of reinforcing transformation

http://www.agilejournal.com/articles/articles/the-agile-pmo.html

Agile Journal - Feb 2008

Transparency

One of the things that Margaret Wheatley says in her book “Leadership and the New Science” strikes to the heart of why we preach transparency. The statement:

Everybody needs information to do their work. We are so needy of this resource that if we can’t get the real thing, we make it up.†

On projects, we always used to say – no surprises good or bad. It was a mantra that we used at the PMO level to help project managers feel comfortable coming to us for both reporting and for requesting. The general idea was that if you surfaced a risk, someone who’d been there before could help you work it before it became an issue. Mostly people saw surfacing as an admission of incompetence – and no one liked that – so you can imagine how often we got to help. The unfortunate side effect of this was you never saw real risks, just real problems. At each review, the project managers had plenty of risks and an occasional issue to expose to the group, but on closer inspection, these always turn out to be either trivial issues or fabricated risks. The real ones stayed swimming voraciously like trolling barracuda in the murky shallows by the program’s banks.

Often, the PMs are not overtly hiding anything, they just don’t have the data to really know their own status. Stepping up to an individual and asking them what percent of their task is complete is at best and estimate of an estimate (how much have I really accomplished of what I thought I needed to do). The lack of precision is enough to give the PM a sufficient level of discomfort to the certainty and so causes them to rightly be unable to confidently report progress – in other words, since they don’t have the information in the form they need, they make it up. The quest then, is to find a way such that progress is essentially black and white – to eliminate the ambiguity and ensure that the PMs have the information they need to do their work.

By forcing the programs and projects to be fully transparent and show progress in tangible terms (hey, how about that burndown or burnup chart?), the PMO spends less time wading cautiously through the weeds and more time actually helping. If all the signs are positive, the review can be over in moments. If the signs are not, the program manager becomes responsible for explaining what the data implies and what they have done about the anomalies. Then, when necessary, the help can begin the PMs stop sleeping with the fishes.





† Thanks to Tom Looy (http://conversationswithandrew.blogspot.com/) for this.

A voice for discontinuity!

There has been a growing discussion of late for what is being called the “continuous scrum” or the Kanban approach. Some have even talked about how if 4 week sprints are good, and 2 better, then maybe we keep driving down to the limit and use 0 week iterations – essentially, establish a continuous model. The concept behind this is thrilling and the results I have seen suggest teams can notch up their productivity and increase their throughput as well. For a high-performance team, this is like a nitro boost into their engine. They become the reality every truly agile product manager dreams of!

Unfortunately, not every team is a high performance team and not every product manager is truly agile. The risk that needs to be assessed is what happens to teams that don’t quite reach the plateau of High-Performance or are asked to operate in an environment that is anything less than fully agile enabled?

First, a high performance team is a self-optimizing team. In terms of the agile maturity model, it would be a level 5 maturity team – one that can evaluate and self-correct. When moving to a continuous cycle (Zero length Iterations), there is a substantive risk that a team without the ability to self-optimize could, put the project into a PIO -like negative feedback cycle that would cause irreparable damage to the goals and objectives of the effort. Clearly, seasoned pilots and highly experienced coaches are able to put the ship right with an aggressive but calm damping approach, but we often do not have these types of people available.

In a continuous cycle, there are no “fire-stops”. Once a negative feedback cycle becomes instantiated, the team must first recognize the signs and then be able to fight the degeneration themselves; hopefully in time to save the project. In an iterative process, every iteration review produces a fire stop – an opportunity for the team to check point with their customer and their organization. One activity I encourage teams I coach to go through during each of these iteration reviews is to bring out the assumptions documented during the project chartering activity and to ask the Product Owner whether or not any of the assumptions have changed, as it can substantially change priorities and goals if they have.

Bottom line is I do not advocate stepping into a continuous cycle until everyone is convinced that the maturity level of the team is such that this risk is mitigable. Even in those situations, I strongly believe that there should be an operative and equally mature Agile PMO in place in the organization, to act as a check and balance for the corporation. Once a team reaches the level where a continuous cycle is achievable, it would be extremely easy for them to get in a hyper-productive groove and loose sight of corporate goals and objectives which may change outside of the peripheral vision of the team.

successful transformation

I am a firm believer that successful organizational transformation needs two things: sufficient momentum generated from the top and a supporting structure within the organization to reinforce the concepts and propel the artists forward. Topside momentum must be established by Sr. Mgmt with substantial vested interest in seeing the organization transform and meet their goals.

This doesn’t happen if the changes are isolated to IT. IT is a service bureau. It is a Cost Center, there to create the tools by which the rest of the business can create value. No business ever made more money now and into the future by having a more effective automated purchasing system, BUT an automated purchasing system has allowed businesses to more effectively turn their AP into Cash (providing substantive value).

If we teach and preach at the “C” level, then we address high level issues and concerns and create business value there. At that level, there is more of an understanding of value and therefore a higher willingness to pay on a value-priced schedule, rather than for an hourly lowest-cost provider. I have been working with “C”s and VPs along this line almost exclusively for the past couple of years and have been quite successful – basically assessing what their real issues are (some RCA) and then stating what I can help them with (if anything – referring them elsewhere if necessary), and then tailoring solutions to fit within their cultural constraints. It is a rare luxury where I am able to also address the second point – tuning their organizational structure to reinforce the transformation.

With respect to efforts I have coached in the past; Segway was successful because we were able to tune the organization. Borland was less successful, but since we were able to tune the compensation system to align the teams with achieving what the company needed to have done – we effectively reduced the transformative process from years to year.

How to set up an Agile PMO to best support the transformation

The instantiation of an Agile centric PMO at the outset of an agile transformation can help bolster the depth and breadth of the changes. Many practitioners reiterate that agile is a very experiential methodology and that there is a fair need to suspend disbelieve while trying the first agile project. Accepting these sentiments, it makes sense that the creation of a support mechanism simultaneously with the initiation of the transformation would be of great value.

In similar veins, we also state that the best way to inculcate the new methodology is to have one or more experienced mentors on the team, constantly able to bolster individuals when doubts arise. If this is appropriate for the team members, why not for the project managers too? By establishing a PMO made up of a mentor and some or all of the project managers working on agile projects, both a support network and an oversight board are created. Their responsibility remains to provide check and balance to the agile. The recommendation is to choose a period that is at least as large if not larger than the maximum iteration length amongst the teams participating (monthly, perhaps). The rationale for this is to be able to review the previous iteration and any issues that were present there and make recommendations on handling them in the future. By setting up the periodicity this way, there will always be at least 1 entire iteration to review and projections on the next to consider.

Establishing a peer group gives the PMs an informal network from which to seek help outside of the PMO reviews while helping them all mature and learn from each other’s experiences. Once the peer group gets large enough, the actual PMO review responsibility can be rotated through the group with the mentor and several different PMs being responsible each time.

The actual content of the review becomes simply reviewing the trendlines for the Burndown and Burnup charts and asking for explanations on anomalous behavior on either. If the charts imply confidence of meeting the business’ expectations, then the review is short and sweet. If either adequate explanations can not be put forward or leading indicators belie satisfactory conclusions, then a longer discussion is entered into in order to help the PM identify interventions that will change the course of the project. It remains the PMs responsibility to steer the team, but as with any retrospective, independent view points often help make the path more obvious.

So, what's the problem?

It is becoming more and more recognized that iterative development, specifically those using agile methods, offer a significant advantage to those companies trying to achieve both higher throughput in their project organizations and better alignment with the business organizations. Historically, these types of methodological shifts have been initiated by either the business or director levels within the development organization. Many times, these individuals can initiate an entire organizational transformation with the graces of senior management by referring to successful case studies documented in industry literature . Overcoming the immediate concern that Agile is merely cowboy development with barely more controls than the loose code of conduct rules that go with the cowboy activities, IT has begun to embrace the methodology principally because it has indeed been able to deliver on its hype.

The collection of these documented success stories are swelling and are driving Agile into the mainstream of software development. However, in the past several years, we have also begun to see an interesting phenomenon whereby successful transformations begin to devolve fairly rapidly, even to the point where projects begin to slide and even fail. We are also seeing a number of companies contracting in services to re-enable their agile transformation with the hope of recapturing their early successes and perhaps making the transformation stickier this time around.

Thinking back to 9th grade brings to mind the old Newtonian adage: things in motion tend to stay in motion unless acted upon by an outside force. Although it is certainly possible that in any of these situations where the transformation has deteriorated there may be active resistance to the change that acts as the outside force, but typically a process change is not deemed successful until there is consensus amongst the team members. This suggests that active resistance would have been weeded out prior to declaring the methodological shift successful. If we throw in to this mix another adage, this one from my early university aspirations, I think we can begin to understand what the crux of the issue is. From thermodynamics, we understand that entropy suggests that systems deteriorate into disorder if there is no continuous containment forces. Thought of like this begins to offer some clarity to the forces at work (or not). Clearly there are outside forces that are causing the transformation to slow or even stop – if we believe that at least one of these forces is entropy, which makes sense, can it be reversed?

Continuous containment is an odd term to use with a process or even a methodology, so let’s take some liberties here and refer to is as encouragement or positive reinforcement. Do the agile projects in your organization get positive reinforcement, negative reinforcement, or neither? And, what form does this reinforcement take, if it exists? In most Development and IT organizations, there is a responsibility that belongs to the Program Management Office (PMO) to provide checks and balances on each project. They offer an independent review of budget and schedule for senior management as well as a periodic critique for the project managers. It is this critique that is the reinforcement, the containment force, for the transformation.

Earned Value Behaviors

In companies that use traditional ‘plan-driven’ methodologies to execute their projects there is usually a check and balance system put in place with the intention of providing independent review of project status and progress. Called the Program Management Office (PMO), this group becomes the most influential force in the organization with respect to both the execution of programs and the approach by which they are undertaken. In the classical sense of you-are-what-you-measure, when the PMO requests that the Project Manager come to the review prepared to discuss budget, progress, and work performed to date – the essential variables for earned value calculations – PMs begin to focus their work and their teams on these measures. These critical aspects thus get passed on down to the team and so everyone begins to worry about where they are as versus where they anticipated being. Interesting as well, this course of measurement only represents lagging indicators, telling the team how well they have done, not leading indicators, suggesting where they will end up. Ultimately, the result of this is we can tell the business how well or poor we performed to date, but we can not accurately tell them how things will turn out.

This lack of leading indicators for the project force us to continuously rely upon the original plan for guidance as to where we want to end up, progress-wise. As the project itself wears on, deviation from the plan tends to increase and the use of the original plan as a barometer becomes more and more invalid. Stepping aside from this deficiency to forecast, projects tend to get graded on how close to the plan their progress to date lies. In an Earned Value environment, the significant measure is BCWP or budgeted cost of work performed. As it states, this is a measure of how close to the planned cost you were able to execute the work completed. Note several things, however:

1. The project is seen as successful as long as it is meeting the plan’s budget, regardless as to whether or not the plan is viable
2. Since the measurements of task completion and their costs are asynchronous, partially completed tasks tend to be viewed as proportionally complete to their elapsed time to date (in other words, a task is 70% done if its team has billed 70% of the task’s estimated effort).

These two realizations produce a set of behaviors in the Project Manager and subsequently in the team. Basically, since team all wants to be seen as successful (even more so than they want the project to succeed), there is a desire to meet the plan and to be able to correlate work completed to budgeted costs. As long as these objectives are somewhat ambiguous, then clearly the teams can claim success – producing another behavior, one of obfuscation of completion status.

All in all, the age old complaint is that the teams and the team’s management end up finding ways to ‘game’ the system and even though the PMO goes through these monthly evaluations to assess progress, even they know it is a sham. Only once delivery milestones are met or missed does the business really understand the entirety of the situation. This ultimately ends up as to why we continuously hear of projects being canceled after significant funds have already been spent with no clear path to success .

Metrics, Part 1

The objective of lean is to eliminate wastes, as waste is seen as the barrier to efficiency and in turn, loss of efficiency reduces throughput and hence profits. Certainly, profit, throughput, and efficiency are terms at least one abstraction beyond what many software development teams are trained to understand. Even the ranks of Project Managers are thin in their understanding how corporate and division profits vacillate given the vagaries of the throughput of their projects. Honestly, even efficiency is a measure that is both more abstract and subjective than most of us are able to clearly understand as a factor of project success.

Therefore, rather than take a top down view at this problem, we can swing around and use Shingo’s bottom up perspective and look simply at the 7 wastes. If we accept his successes at Toyota as validation of Lean, then we can accept that the elimination of these wastes will yield substantive improvements on the bottom line. Induction then states that objective measurements against these wastes become a reasonable view on how to assess progress against the goal of improvement.

Simply stated, if we measure how we are doing in each of Shingo’s 7 Waste categories, we will have sufficient metrics to track our projects both on their own (the lower the waste, the better the project team is able to perform) and against corporate standards.

I do want to note that when comparing a single project against a corporate standard or norm, the team and the process improvement staff must all be of consensus that dramatic aberration (that outside of the statistically acceptable variation) is neither really bad or really good, but rather indication that immediate detailed review must be undertaken. When a project starts reporting data way above the norms most organizations understand intuitively that something is amiss, usually they focus on insufficiencies or inabilities of the team. More likely, variation above the norms means that the team is either (or both) not sufficiently trained in the approach or the domain, or the project objective is not well understood (i.e. it is substantially more complex than the team initially understood). This type of aberration becomes an early indicator that the assumptions made in the project charter are no longer valid. Equally, when a project starts reporting way below the norms, then review is still necessary. Mostly, this type of aberration occurs when the team has over simplified the requirements and the project runs substantial risk of not meeting client expectations.

Structure for an Agile Organization

Part 1 - Values

When we look at an organization that is successfully agile, we see one that has been able to shift tactics as their business needs change and continually move their hyperbola of focus to where ever is first strategically aligned and second producing the most value. To do this well, they must have a set of values that they have internalized to their core – that both their corporate execs and their rank and file all believe and follow. Often, actually stating these values is hard work as they are not easy to define succinctly and the effort to develop them from the organizational leaders is seen as too new-agey to be seen as a serious endeavor by modern captains of industry.

Unfortunately, most of the time when I urge organizations to develop value statements, they either leap immediately to their mission statements and hold them up as values or co-opt someone else’s values as their own. Both paths lead the organization to peril because the mission is usually more of a vapid marketing statement than actual values that anyone truly holds and other people’s values are just that – other people’s.

Since most organizations I work with are focusing on agile methods, they tend to co-opt the values from the Agile Manifesto which, though still someone else’s, are not a bad place to start. When they do that, what I try to do with them is deconstruct the value statements and tailoring them to the organization at hand. For instance, the first value – Individuals and interactions over processes and tools – is a commonly co-opted one; however, not one that is usually internalized. The way I transliterate this value is that the company must genuinely feel that their people are individuals and not resources and that the most valuable tool is not some computer program but personal interaction. This too borders on touchy-feely and is usually met with stares and eventually the statement that it can not scale. I try to counter with the addition that with proper processes, it does not need to scale, but by then, I know I have lost them. I do not know why modern business feels they can not be modern and efficient without dehumanizing their internals.

Higher Goals

At first review, my higher goal is to teach.
There are several things I teach different peoples, some I do better than others. Business wise, I seem to principally focus on teaching organizations how to first begin to understand what their real objectives are and then how to achieve them. Sometimes, the host organization never wants to recognize their real objectives and instead prefer to focus on simpler, more straight forward goals - like making more money now and into the future.

I used to think that that goal, as stated, was the quintessential objective of any organization - if all activities were aligned correctly, then this is what would be achieved. I have more recently come to believe that there is a more crucial path that must be followed. I believe that there is a social consciousness that the corporate entity must have in order to just be viable and then it needs to adopt a holistically optimized fiscal policy in order to really succeed. Basically, a company whose executives are hell bent on making as much money as possible for themselves and/or their shareholders on the shoulders of a second class within the organizations (the workers as versus the executives) is doomed to failure, as it drives towards local optimal. Ricardo Semler has had some interesting ideas that he has put into practice and succeeded with. He was a contemporary with Goldratt's original work, and though I can not find any indication that they knew each other or of each other, but I find early Goldratt and early Semler rather complementary.

Bottom line is that I am really tired of corporations - large or small - taking advantage of their workers. We should train and support and mature them - like a medieval guild. It does not have to be compassionate, but if it does not take care of its workers, then they will not be able to continue producing work that provides value to the company and its clients. Think of it as a symbiotic relationship more than anything else! If my higher goal is to teach, I would like my highest goal to be to teach this message, and perhaps show the world, one industry at a time, how we can make our society a better place by symbiotically respecting and supporting each other. The final finesse would be to put it into a worldly perspective such that this whole process fits within the greater flow of the river of life, such that our actions and results seek to reach a holistic global optima, fitting in with nature rather than replacing it.