What about PMs in Scrum-based development?

In scrum terminology there is no such thing as a project manager (PM). However, in addition to the product owner (PO) and the scrum master (SM) Henrik Kniberg in his excelent "from the trenches"-book talks about a "sponsor". Someone that provides for the teams long-term staffing and work environment as well as comitting to remove impediments outside the team. He describes himself, in his position as development line manager, as having this role.

In my experience many large organisations are totally project organised bringing together people from different parts of the organisation as well as external consultants to form a team. In such organisations the traditional PMs, if accepting to have an empowered team, is often ideal for this sponsor role.
That brings me to propose the following division of responsibility:

Product Owner: Responsible for what the team is to build. Merges requirements from different stakeholders and manages priorities.

Scrum Masters: Guides the team and PO in following the agreed process. Makes sure the surroundings understand how the team works and do not interfere. The SM role is usually not full time and is therefore filled by a member of the team.

Project Manager: Responsible for long-term staffing and external contacts in order to provide the team with the best possible environment to meet the needs of the organisation.

Ideas and books forming me as a Software Engineer

This September it was 15 years since I joined university to study Software Engineering, the official name was “Informatics”, but I think software engineering (or authoring and gardening as argued in another post) is better describing what I was interested in and, luckily, what I'm currently doing. Since then I've got some 10+ years of experience from working in the industry and a whole 15 years of continuous learning. As an exercise for myself I've set out to go through the major ideas I've picked up and used along with the people and books that collaboratively have formed me to the software engineer I am today. Hopefully you find this post at least a bit interesting and inspiring as well.


Object Orientation (OO)
In 1996, and the following years, OO was the new hot thing in software design, at least at my university. I learned to model domains (even though I didn't know that word by then) in objects using the OMT (Object Modeling Technique) notation by James Rumbaugh (later on one of the main contributers to UML). This way of abstracting the real world into objects with attributes, methods and relations very much appealed to me as it offered the tools to unambiguously document the analysis result by drawing a diagram. I did also believe my teachers when they talked about OO as the enabler of universal libraries of small reusable objects. Now I, and I'm sure they too, know this is never going to happen, but the OO-paradigm still brings structure for reuse within a system and hence is a great tool for honoring the DRY-principle. 
Ever since those first courses I started to think of OO as the way to design and build software systems. Later on I've come to see OO as the way of designing and building transaction based information systems where manipulation of state is the primary focus. Other types of systems might be better suited to use some other paradigm, but this type of systems for administering information is what I've been working with almost the entire time since I graduated. In university I learned the theory of OO, but we didn't actually turn any of our OO-designs into working software. We sure learned programming, but then it was to solve more traditional algorithmic problems. When leaving university I was all eager to learn how to do OO in real world projects. And boy, was I disappointed!
For several years I was employed in projects where I learned lots of things about programming and projects in general but nowhere was OO used to anything but data diagrams (now drawn in UML notation which I learned through Martin Fowler's “UML Distilled”), i.e. objects with attributes and relations but no behavior, that was later turned into database tables and data structures. All behavior was programmed into transaction scripts, either directly in the GUI components or as free standing functions or stored procedures. I was desperately looking for a real world example of true OO implementation because on my own I couldn't really figure out what it would look like in order to work properly. The first piece of the solution came when I got to read Craig Larman's “Applying UML and Patterns” . In this book he describes all aspects of OOA/OOD, how the software can be structured with boundary classes, controllers and entities, and shows by example code how it all comes together. Still, this isn't the approach intuitively encouraged by the structure in popular frameworks such as Java EE and Spring Framework. Out-of-the-box they rather suggests a static service structure operating on DTO:s, and that is, I think, the main reason why most systems are built with data and behavior separated and not using the true OO-paradigm.


Domain-Driven Design (DDD)
In a Jfokus 2008 tutorial I was first introduced to Domain-Driven Design. The three hour tutorial only scratched the surface of DDD but it was enough for me to understand that this was the description of how the full OO-paradigm is used to build real world systems. Soon after, I got to read Eric Evans' book “Domain-Driven Design - Tackling complexity at the heart of software”, which I have posted about previously, and since then I firmly believe that DDD is the approach to use for designing and implementing complex software systems.
DDD is not a method, nor is it simply a technique or an architectural pattern. I would rather call it an approach to software development, including analysis, design and implementation.
Perhaps what most of my colleagues first think of when DDD is mentioned is the design sketches I use to draw with the business domain at the center and technical integration packages all around. The domain being built on the conceptual building blocks, entities, value objects, repositories and services, as introduced by Eric Evans. But even though that part of the approach might be the easiest to pick up, and also important in building working software with DDD, I think what makes the biggest difference to me is the ubiquitous language. This practice of building a model with a shared language based on, but with more precise definitions than, the language spoken by the domain experts is the basis of my philosophy to always build the software so that structure and logic follows the business domain. I think that is the only way to guarantee a flexible design such that a small change to the business is always a small change to the software, never a big one. My experience is that the second you deviate from that principle, most often due to time pressure or pure laziness, you are asking for trouble further down the road. It might take a year or so, but it will come back and bite you!


Test Driven Development (TDD)
If DDD provides the approach on analysis, design and implementation, TDD is clearly what integrates that approach with quality awareness and assurance. TDD is my way to ensure a testable design and a correct implementation that is easy and secure to improve further. In a previous post, "Are you testinfected?", I told the story of how I got introduced to TDD, how I was skeptical at first and how I later on proved the usefulness to myself. Now I rarely, and definitely reluctantly, writes any code without first writing a failing test.


A Clean Coder producing Clean Code
Ever since the start of my career I’ve wanted to produce code that I’m proud of. However, the benchmarks for that assessment have changed over the years. Now I consider well tested and easy to read code being the standard to achieve. Since any piece of code is read much more often than it is written I think readability is prime quality measurement for code. Lately I’ve been reading both Clean Code and The Clean Coder by Robert C Martin. Both are excellent books. The first one giving good advice on how to make the code readable, the second really demanding reflection on what it means to be a professional software developer.


Agile with Scrum and Kanban
Another interest of mine is software development processes and techniques. At first I learnt how to deliver in a world of waterfalls, later I got to experience RUP. Despite all bad things said about RUP, I’ve always found the core messages on iterative, risk driven development to be right and frankly quite agile. However, agile approaches such as Scrum, Kanban and developmenttechniques gathered in XP take this a long way further. The major inspiration here has been Henrik Kniberg, first in his Jfocus tutorial in 2008 and through his two books on Scrum and Kanban.
Currently I’m working in a world of mixing and matching. I’m picking the best parts out of all those above to create a process and way of working that supports being as agile and lean as possible in an otherwise RUP-ified world. I think, at the current state it is somewhat of coming success.


“All problems are not worth solving”
That was one of the advice I got from an experienced colleague of mine many years ago when I was aspiring to go from being a sheer hacker to become a CapgeminiCertified Software Engineer. He probably gave me a bunch of other advice too, but this is the one I still remember. And I also think, for me personally, it might be the most valuable advice I’ve ever got. Since problem solving is my trade I happily go of designing solutions, then it is good to remember thinking for a minute or two about whether the problem really deserves being solved or if it is something we can as happily live with. At the bottom of this is that all resources are limited and needs to be applied where they make the best return on investment. On the other hand, we need to be careful not to spend too much time on deciding if a problem is worth solving, i.e. it is not always worth solving the prioritization problem if the solution offered is cheap enough.


The future
This pretty much sums up where I am today, but of course I will continue to evolve and in the near future I see the following ideas as the most interesting.
- Strategic design – How DDD concepts such as Bounded Context is applied at a larger scale to make strategic decisions on how to factor architectures and apply different design and development strategies. In most discussions “architecture” seems to mean frameworks, databases, application servers and technical layering. I think architectures would be much more interesting and usable if they concerned business domains.
- CQRS – I have only touched the surface of this approach to structure systems with separated command and query sides and event stores and messaging instead of relational database schemas and object/relational mappers. However, I think is an interesting approach and I definitely plan to dig in deeper.
- DCI  – Even though DDD has come to be my preferred approach I must admit the structure of the code base with behavioral code localized in every domain object has its draw backs. With requirements based on use cases or user stories, it is sometimes hard to answer the not so infrequent question “how is this UC implemented?”. DCI (Data, Context, Interaction) solves this problem through keeping the behavioral code in role objects implemented within a context representing a single UC. The code in the role objects are then injected into the data objects representing the domain. To me, it sounds like an interesting add-on to the ideas of DDD and in the future I like to continue exploring the benefits of this approach and the demands it poses on the implementation language.
- Functional programming – In order to build simpler programs that are safe to parallelize on multiple cores functional programming has gained new interest in the last years. Up until now I’ve lived only in the world of imperative programming and grasping the concepts and underlying mechanics of functional programming will pose a challenge. A challenge I’m happy to take on.
- Scala – Of all the new languages having emerged on the JVM I think Scala is the most interesting. It combines functional programming with additional powers in OO-programming without losing the benefits of static typing. All of which I think are interesting qualities. I think Scala might be the enabling programming language for many of the ideas I listed here for future exploration.

No man is an island

What’s the most common reason for failure in a software project, or perhaps in any project? You will get many different answers depending on who you ask. The designers might say “The requirements were not clear enough and the programmers didn't follow the architecture”. Testers on the other hand might say “The programmers never delivered what we were supposed to get when we were supposed to get it” and programmers tend to say “We got lots of ‘bug-reports’ that where really outside the scope”. The list can be long, but at the end of the day it all comes down to lack of communication.

Let’s face it, developing software is a complex and creative handiwork. That’s why we try to stay away from the over-planned, defined, way of waterfall and turn to iteration based approaches like RUP or Scrum. True iterations give us great feedback to base the next, (hopefully) better, version on. However, working with iterations is not enough. The saying “No man is an island” origins back to about year 1600 and is as true as ever when it comes to software projects. For many years we have set up the projects so that specialist testers form a test team, specialist designers form a design team, specialist programmers form implementation teams, and so on. This for sure brings advantages in having specialists learning from each other becoming even more specialised. But it also introduces a great deal of hand-overs between the teams, each missing some vital information. I believe this way of organising the development project is one of the key reasons why projects fail.

In the agile community we talk about “Cross-functional teams“, meaning that people with different specializations work together to achieve a common goal. In software development that means that a programmer sits next to a designer and a tester, working together on transforming a requirement into a piece of well designed, well tested code ready for production. The agile practices are all about maximizing communication, e.g. by locating people in the same room with access to broadband communication aids such as whiteboards.

If you are currently in a project where the test team and development team sit at different floors and the primary tool for communication is release notes, then you have a superb opportunity to increase the likeliness of project success just by re-locating so that developers and tester form collaborative teams working together on both design, test design, implementation and test execution.

The main pillars of Scrum according to me

In this post I'll share with you what I think are the most valuable pieces of Scrum, pieces I'm never willing to compromise over.

Iterative/incremental

Scrum is truly iterative mandating to produce a fully functional increment of the final product, including at least some business functionality, in each sprint. In addition each sprint ends with a sprint review and retrospective making the feedback loop explicit.

Transparency 

In Scrum everything is visible. Any misconception in the organisation, design or infrastructure is pulled into the light. Honesty is just the first name!

Continuous improvement

Fast feedback loops, in larger and larger circles, and transparency are a solid ground for continuous improvement both to the product under development and to the process itself. Always strive to do things faster and with higher quality than the last time around.

Self-managing teams 

Scrum is based on the belief that a team of professionals is most effective if left alone to organise, solve its problems and work towards the sprint goal instead of having management telling them what to do and how to do it. Managers are to take the role of facilitators/sponsors making sure the team has everything it needs to produce as much value as possible.

Definition of done 
When is a task done? In order to truly complete a piece of work and deliver a high quality increment it is absolutely vital to have a solid definition of done. Any work that should have been done to get the increment ready for production will not go away if left undone. Instead it will accumulate over the following iterations and add up exponentially to a large, undefined, piece of work requiring a stabilisation phase of unknown length to meet quality criteria.

These are my fundamental values which I try to stick to. What do you think? Do you have other/additional most pressures parts?

The only certification I'm proud of

It seams the popularity and importance of certifications goes up and down in the IT-industry. The only thing constant is the ongoing debate on whether certifications are valuable as a "receipt" on competence or a total scam. I think there is a value but the value vary greatly depending on the type of certification.

After a bit over ten years in the industry I've gathered a set of certifications. However, the value of them are in most cases very little over following a coarse or reading a book. I'm certified in both OOAD by IBM/Rational  and in Java programming by Sun/Oracle. I have also ample experience in both fields but I got the certifications prior to that, by just reading the books and completing a test. For my Scrum Master certification I didn't even have to complete a test, following a two day course was enough. This is the reason for this particular certification being so heavily criticised, and I agree, the "certification" does not bring any additional value. But the two day course was great for gaining fundamental knowledge on values and core practices and I have since practised implementing Scrum and other agile techniques and value systems successfully.

These types of certifications are, despite that it might involve hard work to complete them since they tend to test on so specific things, entry level in most cases and as such adding no more value than successfully completing an exam at a small course at university. But there are other types of certifications that are based on real world experience and proven capabilities evaluated by experienced software engineers. Within the Capgemini group we've had such certifications for different professions for a long time and more recently it has been aligned with the publicly known Open Group ITSC. This is a certification based on what I've done in real world assignments and I think it is also a valid quality guarantee for the knowledge and experience I can bring into a new project. This is the only certification I'm proud of!

True iterations deliver value

Everyone wants to be iterative and incremental these days. Iterations are indeed nothing new; they have been around inside and outside of RUP and other methodologies for a long time and they show up again in SCRUM (called Sprints) and XP. Many projects claims to be iterative, but are they really?

In my view you are truly iterative only if you for each iteration go through the whole cycle of analysis, design, implementation, integration, test and deploy to really produce a high quality increment of the final system. Many projects I’ve seen claims to be iterative but tends to focus only on analysis and design in the first iteration, hence implicitly converting the iterations to phases in a good old fashioned waterfall.

The only way you get to benefit the true value of iterative development is to run through the whole cycle and collect feedback from everyone involved. Then you know how to do even better in your next iteration. As a Software Engineer we should really push for true iterations in our projects since that is the only way we are sure to be involved and able to give and receive early feedback.

Use SCRUM to increase flexibility, efficiency and transparency in the delivery process

In this post I’ll give a short overview on why I think using SCRUM is a good way to increase flexibility, efficiency and transparency in the delivery process.

Flexibility
In SCRUM we work with short, iterative, releases (sprints) where the customer (product owner), at the beginning of the sprint, is allowed to decide what is most important right now and should be included in the next release. Hence "time-to-market" for a typical feature can be reduced to only the length of a sprint, typically 2 to 4 weeks.

Efficiency
SCRUM emphasises the delivery team as responsible for the delivery they have committed to within the sprint. The team is also empowered to make decisions on how to organise the work to achieve maximum efficiency now and in future sprints.

Communication on priorities, requirements and possible designs is short-circuited between the team and the product owner. Hence, additional management and detailed planning is eliminated and feedback cycles are sped up. Everyone involved is contributing directly to the delivery of high-quality code, following the lean principle of "eliminate waste".

Transparency
SCRUM introduces complete transparency to the delivery process. Expectations for the next sprint is set and agreed between the product owner and the team in the "sprint planning meeting" at the beginning of the sprint. Current status is tracked every day in "daily scrums" where all team members discuss what has been done and what should be done next. Anyone is invited to listen in on "daily scrums".

At the end of the sprint a "sprint demo" is organised where the team demos the features implemented in this sprint. Everyone is invited to the "sprint demo" to see and comment on what has been done and what needs to be done next.

The SCRUM team
A typical SCRUM team consists of 5-8 people including every competency needed to produce a production ready release in every sprint. Competency in the team includes:

- Analysis
- Design
- Implementation
- Test
- Configuration Management
- Scrum Master (responsible for the delivery process)