Coordination Strategies: Beyond Scrum | Part 3

In the previous post, we discuss different strategies that are imperative in successful implementation of Scrum process model in GSD context. In this post, we take this discussion further by introducing few strategies that are originated from other disciplines than software development; however, they can have positive impact on LFDS.

Queuing Theory: Increase Parallelism by Eliminating Queues¶

Queuing Theory [1] provides an insight on the release cycle in the large-scale product development. In today’s competitive market, in order to survive, it has become essential to minimize the release cycle of the new features. Hence, careful consideration of Work-in-Progress (WIP) queues with long-batch of task should be taken into account in large-scale Scrum, as they increase the average cycle times and eventually, reduce value delivery. C. Larman and B. Vodde propose the following approaches to manage WIP queues in large-scale Scrum context [1] :

  1. Eliminate the queues by changing the system – removing the bottlenecks and the other factors that create the queues.

  2. Introducing the feature teams  that implement customer features from end-to-end and reducing the dependencies, can vanish these queues, as the feature teams begin a highly parallel development cycles.

  3. Reduce variability e.g. ambiguity in user stories [2], estimated vs. actual effort variance and so on. Variability is considered as one of the three sources of waste in lean thinking [3].

Incorporate Lean Thinking in LFDS¶

Lean thinking is a proven system that works for the large-scale development. Toyota and other companies show such evidences. The main two principles of lean are following:

  1. Continuous improvement or kaizen,
  2. Respect for people [3].

The lean principle always strives for self and organizational improvement and promotes an atmosphere of continuous learning and embracing changes, which is complementary to the Scrum principle as well. Reducing waste from every phase of the development cycle is one of the primary motto of lean principle. Hence, adopting and embracing these principles in the software development sub-culture of the organization is bound to have a positive impact on the efficiency of LFDS.

References

[1] C. Larman and B. Vodde, Scaling Lean & Agile Development: Thinking and Organizational Tools for Large-Scale Scrum: Addison-Wesley Professional, 2008.

[2] M. Cohn, Succeeding with Agile: Software Development Using Scrum: Addison-Wesley Professional, 2009.

[3] P. Middleton, “Lean Software Development: Two Case Studies,” Software Quality Control, vol. 9, pp. 241-252, 2001.

Revisions

[R-1: 11-03-2013] Updated formatting to make this post more consistent with current CSS.

Coordination Strategies: On Successful Implementation of Global Software Development using Large-scale Fully Distributed Scrum| Part 2

In the previous post, we discussed the motivation behind employing various coordination strategies while using LFDS in GSD context. In this post, we discuss several intra- and inter-team coordination strategies to successfully implement LFDS to alleviate the impediments incurred by the GSD. Two primary aspects – “scaling” and “distribution” of Scrum are the focus of these strategies.

Strategy 1: LFDS Team Formation¶

In LFDS, in order to achieve high performance gain, despite of the challenges of GSD, team formation plays an important role. From the literature related to the GSD and Scrum [7, 8-11, 12], it is evident that Scrum feature teams can alleviate the prevailing distance of GSD by its inherent characteristics of being cross-functional (cross-discipline), self-organizing dedicated and long-lived. Jim McCarthy describes the feature team as follows: “Feature teams are about empowerment, accountability, identity, consensus and balance… ”[13]. In order to form such a team, we present following strategies.

Conway’s Law: Avoid Component Teams¶

Old approach in team formation is to divide the teams by component of the systems where each team implements and maintains its own component(s). Long ago, Mel Conway has observed the effect of such team formation and has stated as follows:

“…Any organization that designs a system will inevitably produce a design whose structure is a copy of the organization”[14].

The realized disadvantages are code smells such as, code duplication, unnecessarily increased system/code complexity and above all the coordination complexity in the large groups. In organizational perspective the most notable risks of such a team formation are following–

  • Resource dependency
  • Lack of cross-functionality
  • Difficulty of scaling up or down the team
  • Single point of specialization success or failure
  • Sequential life cycle development
  • Lack of knowledge sharing which incurs a learning debt
  • Lower productivity and increased handoff and delay due to the physical dispersion and so on.

Hence, in LFDS, we should avoid such a team formation by promoting feature teams that focuses on end-to-end customer features.Following diagram Fig. 1 illustrates a strategy that reduces the risk of forming component team in large-scale software development by engaging all the feature teams in end-to-end feature development.

clip_image002
Fig.1. Task distribution in LFDS

The detected primary organizational benefit of this approach is the shifting of the coordination challenge from upfront requirement analysis, planning and inter-team management towards the coordination at the shared source code level[7]. Moreover, this approach complements with all the key characteristics of feature teams mentioned earlier while promoting and necessitating the clean code through consistent code refactoring [1], continuous integration [2] and code coverage [3] through unit and integration testing.

Acknowledge the contextual differences¶

So far, we discuss the scalability aspects of the LFDS team formation. However, the team members of a feature team are distributed across the globe in the LFDS model, which initially reduces the teams’ efficiency and velocity[6] due to the geographical dispersion, differences in culture, time-zone, and standards. An interesting survey is done by G. Hofstede, who surveyed IBM employees in over 50 countries to identify major cultural differences [15]. Only way to alleviate this distance and increase team velocity is to acknowledge and take these differences into account during initial product planning phase. Personal interaction and collaboration is an excellent way of getting familiarizes with the cultural differences of difference development sites. We notice in different literature [8, 9] that at the initial phase, the team members start working in a local site, which in turns, make them familiarize with each other’s contextual and personal similarities and differences. After working certain period together to achieve efficient communication framework by inter-personal interactions, higher productivity through team work, shared ownership and more importantly, building the trust, team members are distributed in multiple sites and scaled up, as necessary. As a result of this face-to-face setting, they can efficiently communicate and coordinate afterwards [24] and the organization can achieve feature teams with high performance and hyper-productivity, as observed in the several case-studies [8, 9].

Promoting Software Development Sub-culture¶

Promoting and synchronizing the teams with a shared vision requires the establishment of software development sub-culture such as, Definition-of-done [6]. Professor E. Carmel has addressed that in software professionals world-wide belong to software development sub-culture which is stronger than their national culture [4, 5]. Furthermore, the sustainability and improving of such standards can be effectively keep it up by establishing the community of similar-professionals (e.g. architects, database developer, UI specialists) from different teams periodically and promoting their collaboration. This way, whole organization can reach to a same standard, complementing to the process that works.

Strategy 2: Product Owners in LFDS¶

In Scrum, the Product-owners get involved in lots of inward-facing tasks- sprint planning, daily scrum [6] and outward-facing tasks- talking to users to capture their needs, answering questions from the team, communicate with the other stakeholders, developing product pricing and road-map and so on.

In a large-scale software development with multiple teams, product owner role is too difficult for one person to manage. So, we must find a way to scale it in the LFDS model. Ideally, one product owner should be responsible for one team, but no more than two teams. This is the most that one product owner can work with. Fig. 2 illustrates the product owner’s hierarchy in a large-scale product development with multiple teams where the product owners collaborate and coordinate among each other as a team.

clip_image003
Fig. 2.Scaling Product Owners [15]

Additionally, the layers depicted above can be added or removed depending on the scale of the product. As a consequence of this strategy, individual product owner can concentrate and resolve issue related to one or at most two feature teams, which enhance the inter-teams coordination through efficient communication and impediments resolution.

Strategy 3: Manage Prioritized Product Backlog¶

There should be one to one mapping between a product and product backlog. Main goal is to keep all the backlog items prioritized on the same scale. If the teams in the LFDSwork on items from the multiple product backlogs, then the backlogs items must be prioritize against one another. Fig. 3 depicts a Content Management software product backlog with two different teams – Content Management Team and Content Delivery Team.

clip_image004
Fig.3. A prioritized product backlog with different views

Both teams share a common product backlog, but have a different view on the product backlog. They work on it based on the priority. It is imperative to note that few items are viewed by both team which are items recognized as dependency items that both teams are responsible of or the items that either of the team can work on [15]. Notable results of such a prioritized product backlog are following.

─ Conceptual integration of different components in a single integrated product.

─ Dependency identification, earlier in the development phase.

─ Efficient inter-team sprint planning[6] and release planning of the product.

Strategy 4: Dependency Management: Proactively Manage Dependency¶

Scrum promotes self-organization in a proactive manner [6]. Hence, at the end of the day, resource utilization can reach to the maximum level in this framework in the GSD context [8]. But, in the multi-site LFDS, inter-team dependency management requires careful consideration in the sprint planning. Typically there are few strategies that Scrum teams can accommodate in their process.

Rolling Look Ahead Planning¶

As Cohn describe from his experience in [15], rolling look ahead planning can help significantly in identifying dependencies among the team. It requires of planning ahead at least two sprints to identify and acknowledge the dependencies. However, only the detail task break down should be down for the current sprint, whereas, for the upcoming two sprints, all the sprint backlog items should be identified so that the dependencies are identified and communicated with the other teams.

Share Members among the Teams¶

It is a useful strategy for the synchronizing the knowledge of particular features among the teams and identification of the inter-team dependencies [15]. Moreover, this model also promotes cross-functionality and shared ownership on the integrated product. Fig. 4 illustrates the sharing team members among different teams. However, this strategy does not contradicts with the strategy of feature teams, as this transition of the members are periodical with the intention of building better knowledge repository horizontal to the product’s components stack as depicted in Fig. 1.

clip_image005
Fig.4. Sharing team members among the feature teams

Strategy 5: Communication that Connects¶

Communication is one of the key impediments of LFDS model as stated in earlier. In order to compensate communication richness and coordinate effective communication framework, LFDS promotes all kind of formal and informal means of communication that works. Though in the Scrum framework, there are specific communication protocols in place (e.g. Sprint Planning, Retrospective, Daily Stand-ups, Scrum-of-scrums[6]) for intra- and inter-team communication, in order to do it effectively in GSD, several best practices should be adapted, such as – preference of phone communication, video conferencing over email and instant messaging. Furthermore, “Seeing is believing”; hence, as mentioned in [8-10], in order to alleviate the distance, team rooms with constant video feed from all the sites involved in one feature team can be established using cheap, simple , ubiquitous technologies (e.g. Skype webcam and so on). As there is no hard-and-fast rule, the main idea is to promote communication that connects.

Last but not the least, infrastructure is another key inevitable issue in GSD context; hence, it is not addressed as a separate strategy. However, it is noteworthy to mention that, success of  LFDS depends, to a large extent, on the efficient infrastructure setup facilitating shared code access, continuous integration and efficient communication and tracking tools.

In the next post , we discuss more on  the coordination strategies that are beyond the scope of traditional Scrum process and are borrowed from other sources. One thing is common in them that they fits very well in LFDS + GSD context to reduce noise and alleviate efficiency of this process model. Thanks for you visiting the blog.

References

[1] http://en.wikipedia.org/wiki/Code_refactoring

[2] http://martinfowler.com/articles/continuousIntegration.html

[3] http://en.wikipedia.org/wiki/Code_coverage

[4] E. Carmel, Global software teams: collaborating across borders and time zones: Prentice Hall PTR, 1999.

[5] E. Carmel and R. Agarwal, “Tactical approaches for alleviating distance in global software development,” IEEE Software, vol. 1(2), pp. 22-29, 2001.

[6] K. Schwaber, Agile Project Management With Scrum: Microsoft Press, 2004.

[7] C. Larman and B. Vodde, Scaling Lean \& Agile Development: Thinking and Organizational Tools for Large-Scale Scrum: Addison-Wesley Professional, 2008.

[8] J. Sutherland, et al., “Fully Distributed Scrum: Linear Scalability of Production between San Francisco and India,” presented at the Proceedings of the 2009 Agile Conference, 2009.

[9] J. Sutherland, et al., “Fully Distributed Scrum: The Secret Sauce for Hyperproductive Offshored Development Teams,” presented at the Proceedings of the Agile 2008, 2008.

[10] J. Sutherland, et al., “Distributed Scrum: Agile Project Management with Outsourced Development Teams,” presented at the Proceedings of the 40th Annual Hawaii International Conference on System Sciences, 2007.

[11] P. A. Beavers, “Managing a Large “Agile” Software Engineering Organization,” presented at the Proceedings of the AGILE 2007, 2007.

[12] M. Marchesi, et al., “Distributed Scrum in Research Project Management,” Agile Processes in Software Engineering and Extreme Programming, vol. 4536, pp. 240-244, 2007.

[13] J. McCarthy, Dynamics of Software Development: Pearson Education, 2002.

[14] M. Conway, “How do committees invent.,” Datamation, vol. 14(4), pp. 28-31, 1968.

[15] M. Cohn, Succeeding with Agile: Software Development Using Scrum: Addison-Wesley Professional, 2009.

Revisions

[R-1: 11-03-2013] Updated formatting to make this post more consistent with current CSS.

Coordination Strategies: On Successful Implementation of Global Software Development using Large-scale Fully Distributed Scrum | Part 1

Software engineering is evolving every day to address the ever increasing growth of system’s complexity and size along with the application domains and level of interactions with the other systems. In current era of globalization, with the intention of the increase in productivity and resource utilization, large-scale global software development (GSD) is a reality. However, it is known that the successful implementation of large-scale GSD using traditional distributed development process is quite difficult. The consistently increasing popularity of agile software development approaches is the result of their ideology for reducing complexity and focusing strongly on coordination, collaboration and communication. It makes the adaptation of agile methodology, such as- Scrum an obvious choice in the GSD context.

However, inherently, Scrum is only meant for local small teams and employing it to the large-scale GSD is a paradigm shift for this agile process. Therefore, this fact necessitate a pragmatic derivation of coordination strategies or best practices to successfully implement Scrum framework in the large-scale GSD context. In this post, we start we defining several concepts relevant for this topic. Afterwards, we provide the rational behind these strategies in successful implementation of GSD powered by LFDS.

Coordination

Coordination is a consciously organized relation between activities and forces[1], work tasks are divided over actors and the act of making different agents (i.e. people or things) work together[2]. It is widely acknowledged that Coordination in a large scale software development project is a persistently difficult problem to solve and in brief, following are the major identified factors in GSD interfering with the coordination of the software development[3].

  • Geographical distributed development sites,
  • Differences in time zones,  
  • Cross-cultural differences,  
  • Differences in the ICT infrastructure,  
  • Differences in the development standards.

Among different coordination mechanism in practice in GSD, most notable ones are listed as follows:

  1. Efficient communication framework supporting face-to-face, synchronous and asynchronous communication,
  2. Direct supervision,
  3. Standardization,
  4. Mechanism to minimize the coordination overhead such as, use of software architecture[2].
 In addition, we define integration is the manual process that combines the separate components as a working system. Composition, on the other hand, is the automated process of combining separate component as a working system.

Rational behind employing Coordination Strategies

One of the main concern of using the LFDS in a large-scale GSD stems from its very basic foundation of DASD which requires efficient communication framework and team work to achieve success. As a matter of fact, LFDS requires additional strategies in place to address the contextual differences caused by GSD. Following table illustrates the key differences between traditional Scrum and the Distributed Development model based on the realized impediments of GSD.

Realized Impediments Traditional Scrum Model(co-located) Traditional Distributed Development Model(global)
Communication Lower communication and coordination costs.

-Informal/Face-to-face

-Synchronous

-Many-to-Many

High coordination and communication costs.

-Formal

-Often asynchronous

-Tunneled

Coordination – Self-organization.

– Adaptive.

– Standardization.

– Plan-oriented.

Control – Lightweight.

– Cross-functional.

-Command and control.

-Clear separation of the roles.

Integration Higher productivity. Low productivity.
Interaction – Easier.

– Close proximity.

-Same time-zone, culture and work practices.

Difficult due to the contextual difference. It additionally influences the communication and coordination costs.
Conclusion Broad interface between the intra- and inter- team make the development cost relatively inexpensive. Tunnel­­­­­­­ed interface. Contextual differences increase the cost of the development.

Table 1. Difference between traditional Scrum and traditional Distributed Development model

Following are the additional observed problems in LFDS that are not inherent to traditional Scrum.

  • Loss of team-ness,
  • Loss of team ownership or us-them Problem,
  • Loss of communication richness.

To conclude, it is clear that in order to successfully implement LFDS, it requires additional adaptation of traditional Scrum model to address the paradigm shift of it in the GSD context and to bridge the gap between traditional Scrum and LFDS, i.e. Scrum with distribution. In the next posts 1 & 2, we elaborate on different essential coordination strategies that are, indeed imperative for successful implementation of LFDS.

Continue reading

Large-scale Fully Distributed Scrum

Scrum is an agile software development framework that also conforms to the similar success story of the performance gain and hyper-productivity [3-6,7]. J. Sutherland, et al., have defined Scrum as follows[1]-

Scrum is an Agile software development process designed to add energy, focus, clarity, and transparency to project teams developing software systems.

  C. Larman and B. Vodde describe Scrum as a framework for development in which the concrete details are being added by the relevant teams and evolved iteration by iteration[8]. Scrum is a very light-weight and flexible process. It is said that, Scrum is easy to understand but hard to use, because it brings light to the weakness, rather hiding those. Being flexible in nature, Scrum always promotes the process that works by encouraging transparency and visibility. Hence, adapting Scrum as necessary, while conforming to its basic principle is the key strength of this process model. Though, initially Scrum is meant for small co-located team of 5-10 persons, it can be scaled up and used in large-scale product development context. To do so, two additional concepts are incorporated along with the traditional Scrum principles: “large-scale” and “fully distributed” Scrum. In this section, we discuss these concepts, how we can accommodate those in the flexible framework of Scrum and their realized benefits in the GSD context.

Fully Distributed Scrum(FDS) is  a variant models(( for other models , see [1, 8]) of Scrum to support GSD where Scrum teams are cross-functional with members distributed across geographies[1]. FDS as a process leverages all the inherent qualities and complexity of Scrum and at the same time, it addresses an way of scaling Scrum to Large-scale Fully Distributed Scrum(LFDS)[8] by promoting GSD. LFDS is similar to FDS along with a set of additional strategies/rules that we have seen to work in large, multisite, multi-team software development with Scrum. Among various aspects of LFDS, the few realized advantages are as follows.

  • Adaptive changes. Realizing and handling the core impediments early in the development phase, eventually results in increase in productivity. Case study in [1] also depicts a similar situation.
  • Structured Communication Framework. Through Scrum, team members can communicate clearly (although they are distributed geographically), which is an essential aspect of GSD.
  • Shared ownership. It promotes better team work, higher velocity and intuitively, eliminates various soft issues in team orientation.

Above all, LFDS is identified as the force to change the organization by making the rocks visible[2]. C. Larman and B. Vodde state that adopting LFDS in a larger product group explores the weakness of the whole setup– in the structure , process, planning and so on, which in turns, necessitate relevant changes[2]. However, empirically it is well-known that, lowering the water to make the rocks visible is the easy part of the whole change process. Hard part is to remove the rocks which require careful redesign of different aspects of GSD process model. That is way, the coordination strategies are presented later in the post to deal with the challenges of the LFDS in the context of GSD.

Continue reading