tag:blogger.com,1999:blog-54182589016369816902023-11-15T18:16:38.333+01:00SE Thinking - Thoughts from a Software EngineerMy thoughts on technology and process related to software engineeringJörgen Anderssonhttp://www.blogger.com/profile/10229551989831053791noreply@blogger.comBlogger33125tag:blogger.com,1999:blog-5418258901636981690.post-84475619965987431772023-01-19T13:40:00.004+01:002023-02-15T15:18:14.504+01:00Growing Development Efficiency<p></p><p class="MsoNormal"><span style="mso-ansi-language: EN-GB;">Over the last years we
have seen Developer Productivity Engineering (DPE) focus on speed in the
code-build-test cycle. Software Development Efficiency and DORA-/DevOps-metrics
broadens the perspective a bit and adds focus on deployment frequency, lead time
and quality/bug rates. Although these are all measurable, good and important, I
argue that these perspectives are far too narrow. To really grow development
efficiency in an organisation the whole organisation needs to be included.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-ansi-language: EN-GB;">To grow development
efficiency all aspects of building great things need to be considered. Great
developer tools, CI/CD-pipelines, automated testing strategies and good
production metrics are really important, but great tools only help you build
your thing efficiently. You also need to make sure you build the right thing,
achieving the intended impact, and building it right so that it can withstand
the test of time. In addition, you have to make sure your development teams and
surrounding organisation are at their best. Even the best development and delivery
tools can’t help you if you can’t get your prioritisations straight or create a
healthy environment to attract, grow and retain talent.<o:p></o:p></span></p>
<p class="MsoNormal">At <a href="https://www.pensionsmyndigheten.se/om-pensionsmyndigheten/jobba-hos-oss/it-jobb" target="_blank"><span style="background-color: white;"><span style="background-attachment: initial; background-clip: initial; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial;">Pensionsmyndigheten</span>
</span>(Swedish Pensions Agency)</a> we are taking a holistic approach to growing
development efficiency. While tools are important, they only contribute one
piece of the puzzle. Today I introduce the “Development (Organisation)
Efficiency” model we are using to define efficiency in a broader way. I also share
the simple tools by which we are working to grow our development efficiency by
building a shared mental model and a common language.<o:p></o:p></p>
<div class="separator" style="clear: both; text-align: center;"><div class="separator" style="clear: both; text-align: center;"><div class="separator" style="clear: both; text-align: center;"><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgi2FIym-Vquhqe0Jseoa60koXAW3XtUxuWj8XLf2GCNZaRRNrBhmlQUANrvClN9EqmRWaOXKmNgANchQDCaOSmfvU_nYiH7a0bbbeLMSeTjysUB44-xSZ0dFaGC5KizojDsvW5fLH_DD3EI9jLtB6WwDdHzbQ08lw8YUy7d_afMN7zgBPRuv-45zUz/s910/Development%20Efficiency%20model%20-%20ver%201_1.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="910" data-original-width="744" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgi2FIym-Vquhqe0Jseoa60koXAW3XtUxuWj8XLf2GCNZaRRNrBhmlQUANrvClN9EqmRWaOXKmNgANchQDCaOSmfvU_nYiH7a0bbbeLMSeTjysUB44-xSZ0dFaGC5KizojDsvW5fLH_DD3EI9jLtB6WwDdHzbQ08lw8YUy7d_afMN7zgBPRuv-45zUz/w328-h400/Development%20Efficiency%20model%20-%20ver%201_1.jpg" width="328" /></a></div></div></div></div><p class="MsoNormal">Please find the
material in <span style="background-attachment: initial; background-clip: initial; background-color: white; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; background: white;"><a href="https://se-thinking.blogspot.com/p/growing-developer-efficiency.html">this dedicated page</a></span> (or <a href="https://se-thinking.blogspot.com/p/utvecklingseffektivitet.html">this</a>, for a Swedish version) and don’t hesitate to comment or get in touch to share your
thoughts, make suggestions or ask clarifying questions.</p><p class="MsoNormal">You can also find more backgound and reasoning in the short (15 minutes) talk I gave at Jfokus 2023. You can find the recording <a href="https://www.youtube.com/watch?v=C2_aPB7sFYE" target="_blank">here</a>.</p><p></p>Jörgen Anderssonhttp://www.blogger.com/profile/10229551989831053791noreply@blogger.com0tag:blogger.com,1999:blog-5418258901636981690.post-85198063081509821822022-11-17T21:07:00.002+01:002022-11-19T18:29:41.168+01:00From six months release cadence to continuous deployment<p>Since it has been a while, almost ten years, since I blogged about what I'm doing at work I thought a bit of a summary would be appropriated. I don't intend to talk you through every task I've done since no one would be interested in that, but I thought perhaps it might be interesting to some of you to read about me joining the Swedish Pensions Agency (Pensionsmyndigheten, in Swedish) eight years ago and how we collectively since then managed to move from an enforced three, but in reality six, months release cadence for all systems to a continuous deployment scheme where individual teams and the part of the business they serve is in charge of how and when new and changed features are brought to production.</p><p>Disclaimer: This is my story, my perception of how it was, what happened and what it has become. Many of my current and former colleagues who are/were involved might have a slightly, or totally, different view. I'm writing this with a hope it will inspire others to discuss and question matters of organisation and software development processes and how they affect the daily joy and development efficiency of an IT-organisation.</p><h3 style="text-align: left;">How it was 8 years ago</h3><p style="text-align: left;">In 2014 I decided I was done with consulting, at least for now, and wanted to try out being an employee of the organisation where I spend my working days. I found a position as systems architect/developer within the development side of the IT-department at the Swedish Pensions Agency. The ad spoke about an agile and forward looking agency basing their development on Java and open-source tools I liked. At that time we were about 80 developers and testers maintaining and developing systems where some had been around a good ten years already, integrating with even older systems with their roots in the 80's or 90's.</p><p style="text-align: left;">We were all divided into three, about the same sized, silos without much direct communication, each managing a related set of systems. Communication and coordination was mainly handled through people being included in smaller or larger projects. Releases to production were mandated to happen every third month (for all systems synchronized), with a good set of manual end-to-end-testing leading up to it. Any release between those where to only include really critical bug fixes. And yes, we did a fair amount of those.</p><p style="text-align: left;">In my silo, "case management", we were further divided into people (most of us) working in the ever (it seemed) ongoing "program for case management" with a six month release cadence of new functionality, and the maintenance team (the rest of us). In the maintenance team we got a load of new functionality (and code) to maintain every six months and most of what we did was fixing bugs in someone else's code and then spend a disproportional amount of time merging the fix into branches representing the release due in a month's time, the one four months away and sometimes also the one due seven months into the future since development had already begun there.</p><p style="text-align: left;">We used daily meetings (standing!), but no one would call that "agile" today, and I don't think no one really did back then either. The "agile" in the ad I think was a more correct description of the state of mind or spirit within the organisation which came as a very positive surprise to me. There were already a great number of small experiments (or "pilots" as they were called) running in the organisation and whenever a new idea was presented by someone the answer was almost always to try it out as a "pilot". Moving to use Jira for internal organisation of work was one of those "pilots" that came to stuck.</p><h3 style="text-align: left;">What happened</h3><p style="text-align: left;">In beginning of 2015 we were all pretty fed up with the current situation. One of our neighboring silos, "external web", started up an official pilot to try out agile delivery with self organized, cross-functional teams and a six weeks planning horizon. At the same time they had started moving towards a brand new architecture with microservices and continuous deployment pipelines. Despite maintaining a large monolithic system with about half a million lines of code and no ambitions, nor opportunities to change the architecture, we still wanted to find another way of working also in the "case management" silo.</p><p style="text-align: left;">With support of first line management we organized a first workshop aiming to define what "awesome" would look like with respect to development organisation and delivery process within our silo. The result described cross-functional teams, each focused on one or two subdomains, responsible for both developing new features and maintenance tasks, working in a continuous delivery manner where changes were deployed to production weekly or more frequently when need arise. Many would say going to production weekly isn't continuous delivery. I think the important thing is to be <i>able</i> to do it more frequently, but actually doing it is a business decision and they thought, and still do, that once a week is great for the internal user base of case management clerks.</p><p style="text-align: left;">In cooperation with first line management we formed a small team to meet once or twice a week to discuss, develop and coordinate our abilities to meet this description of "awesome case management". We defined roles and teams. We engaged with parties outside our silo to find out boundaries for our new processes and to create ways to cooperate. We enhanced our technical abilities.</p><p style="text-align: left;">In order to do deployments to production every week, instead of just four times a year, we had to do those deployments during office hours, which meant they had to be zero downtime from a business perspective. We also had to make the whole process, from build and test to creation of release notes, automated. Luckily the system already had a heavy set of automated unit and system level tests which made us confident that if all those passed we were good to go to production. With a small team of dedicated engineers, helped by the fact that the whole agency at the same time moved sources to Git/Bitbucket, we built Jenkins pipelines, integrated with Jira and Bitbucket, to manage building artifacts, running tests and creation of release tickets, including information on all changes going into each release.</p><p style="text-align: left;">After working out the organisational, process related and technical issues with everyone involved we turned our silo into the new team structure and ways of working in march 2016. After a bit over a year of discussions and preparations we stopped moving people to work and stared moving work to people (teams). And we didn't look back. Of course it wasn't perfect, but with continuous improvement processes on several levels we were able to improve as needed. The area where we had the most problems was in prioritization and scheduling of larger activities, all coming from differently sized projects still run in parallel across the agency. This was a bit outside our reach, since it crossed several silos. We needed something more.</p><p style="text-align: left;">Luckily, with two silos having turned their organisation and way of working around in similar ways there were a pretty good case for the agency to continue forward and find new, more agile, ways for the whole development organisation. <a href="https://www.scaledagileframework.com/" target="_blank">SAFe</a> is a framework for prioritizing and coordinating work across many teams in an agile and lean way and since our new practices heavily resembled team and delivery practices in SAFe it came natural for the agency to decide to embark on that route. Of course there where a lot of education and discussions to be done, especially with upper management, but since the director general of the agency was a firm proponent decisions were made in due time and a first SAFe <a href="https://www.scaledagileframework.com/agile-release-train/" target="_blank">Agile Release Train</a> (ART) was started up as a pilot late 2019.</p><h3 style="text-align: left;">What it has become</h3><p style="text-align: left;">After about a year of trying out SAFe practices and organisation structures, as well as changing to full time remote work during the pandemic, the organisation decided we had gathered enough experience to have the whole development organisation go full-on SAFe. About one and a half year later we are running five ARTs and a few supporting groups for tooling and very specialized services. Each ART is organized around parts of the agency's rather broad business domain and employ both business and IT-specialists in their teams and management groups. SAFe's <a href="https://www.scaledagileframework.com/lean-portfolio-management/" target="_blank">lean portfolio management</a> helps us organize and prioritize all the large or cross-ART initiatives and the time of projects fighting over individuals for work on full- or part-time is forever over.</p><p style="text-align: left;">With the pandemic no longer directing how society works we have been able to employ both on-site <a href="https://www.scaledagileframework.com/pi-planning/" target="_blank">PI-planning</a> events and frequent "teamdays" at the office. All our gained experiences over the past two years have proven to us that SAFe practices works both in an on-site and in a fully remote setting. Our ARTs and teams now operates with a mix of working remote and being at the office. This gives everyone a great deal of personal freedom and also lays the foundation for team being able to function despite being split over offices in different parts of Sweden.</p><p style="text-align: left;">Also the infrastructure side has continued to develop. With a modern cloud platform and CI/CD-tooling, including a formalized microservice archetype, it's now possible to spin up new services in a few days and have them deployed to production whenever the business sees fit, several times a day if needed.</p><h3 style="text-align: left;">Conclusion</h3><p style="text-align: left;">It has been a really interesting, sometimes frustrating, but overall joyful journey for eight years. We have moved from projects throwing software "over the wall" to teams taking full responsibility for their systems and services all the way to production, from endless discussions over priorities and attempts to maximize utilization to pull-based planning and focus on throughput, from excessive merges and an in-reality six months release cadence to releasing new features as they are ready and we still want more. We are each day looking for ways to make our services better and our organisation a bit more efficient and capable. If you <a href="https://fosstodon.org/@se_thinking" target="_blank">stay tuned</a> chances are you may hear of some of that on this blog. </p><p style="text-align: left;">If you happen to be living in Sweden (and talking Swedish) and fancy come working with us, checkout our <a href="https://www.pensionsmyndigheten.se/om-pensionsmyndigheten/jobba-hos-oss/lediga-jobb" target="_blank">current job openings</a>. Over the years our IT-organisation has grown, and it continues to grow. Our society are in the middle of the digital transformation and IT is at the heart of our agency's business. I think we will see many interesting and challenging opportunities going forward.</p>Jörgen Anderssonhttp://www.blogger.com/profile/10229551989831053791noreply@blogger.com0tag:blogger.com,1999:blog-5418258901636981690.post-49622351213215278302022-11-05T15:58:00.000+01:002022-11-05T15:58:00.224+01:00Discovering the fediverse<p>Over the past week, while being on autumn leave, the resent events over at Twitter and the reaction to this by people I follow in the Java/Kotlin/JVM space have sparked a new interest in social media for me. I've been on Twitter since late 2010 but never tweeted much. Most of the time I've been there to keep up-to-date with new versions of software I'm using at work or at home and to find links to published conference talks recommended by people I follow. For long periods I haven't been logged in at all, just because it hasn't been interesting enough, I suppose. I also suppose it is partly on me since I haven't engaged enough, but I also think the advertisements and the algorithm deciding what tweets to push have made me feel not really at home. In addition, whenever going even slightly outside the Java/Kotlin/JVM bubble into tweets on the climate crisis or local politics, I immediately end up in the middle of the dumbening, anger and hate filled threads that social media in general seems to be full of today.</p><p>Last week I started to see tweets about people setting up accounts on something called "Mastodon". "Just in case", they said, with no more explanations. However, the starting point for me this week was <a href="https://martinfowler.com/articles/exploring-mastodon.html" target="_blank">Martin Fowler's writing on his own explorations into "Mastodon" and the "Fediverse"</a>. I'm not going to reiterate his writing here, instead I urge you to go for a read yourself. It sparked my interest and gave me some useful information going forward with my own discovery.</p><p>Another good source of information I found is <a href="https://axbom.com/mastodon-guide/" target="_blank">Per Axbom's "A Brief Mastodon Guide for Social Media Worriers"</a> and other posts linked from this one. It really opened up the previously hidden "Fediverse" to me, not hidden because it is in any way secret, just because I didn't have a clue about its existence. It turns out a network of community driven, connected (federated) servers running open-source software offering decentralized social media, including <a href="https://joinmastodon.org/" target="_blank">micro-blogging</a>, <a href="https://owncast.online/" target="_blank">live streaming</a>, <a href="https://joinpeertube.org/" target="_blank">video</a> and <a href="https://pixelfed.org/" target="_blank">photo</a> sharing, have been growing since about 2016 (as far is I understand). All using the <a href="https://en.wikipedia.org/wiki/ActivityPub" target="_blank">ActivityPub</a> protocol to exchange data as per their users needs, much like e-mail servers have been doing since Internet was born.</p><p>Even though the fediverse is large and diverse with many different types of social media as (surely not exhaustively) mentioned above I have myself so far only explored <a href="https://joinmastodon.org/" target="_blank">Mastodon</a> for micro-blogging. At the beginning of the week it was a network of about 3,100 community run servers hosting about 500,000 users. Less than a week later, as I write this, it has grown with another 200 servers and 180,000 users. It seems there was an upsurge in usage also in April when Elon Musk's buying of Twitter first became official and then now once again when his takeover is a fact. All this puts great pressure on both servers and administrators having to cope with more traffic and more moderating duties. Remember most of them are just volunteers hosting and moderating in their spare time. Some of the servers are really big and seem to attract most of the new users. However, since they are all federated together you can follow anyone in the whole fediverse regardless of which server you are at (That is not entirely true since server admins tend to exclude federation with servers known for spamming or other non ethical content. However, I consider that a good thing!). We should hope for more servers being started by communities, organisations and corporations wanting to be part of the larger community. It seems it would be better for everyone if the load is spread horizontally.</p><p>For us as new users it boils down to finding a server matching our interests and values and then set up an account or ask for an invite. I used the official moderated server list at <a href="https://joinmastodon.org/servers" target="_blank">joinmastodon.org/servers</a> to find a suitable one. But I'm sure there might be other ways to do it. My interest in programming and open-source led me to <a href="https://fosstodon.org/about/more" target="_blank">Fosstodon.org</a> which seems to have a really great set of server rules. When I had my first look at the server it was open to registration of new users, but about a day later when I felt ready to dip my toe and join in it had changed to "Request an invite". A bit disappointed but nevertheless determined to give it a try I filled in the form, including answering the question on why I wanted to join the server. To my surprise and happiness it was only about 30 minutes before I got my invite and could join the server for real. It turns out switching to "invite-mode" is a way for server admins to keep spam accounts created by bots from sneaking in with the stream of new people currently joining Mastodon.</p><p>Now I have filled my profile and thereby set up my new digital home at <a href="https://fosstodon.org/@se_thinking">fosstodon.org/@se_thinking</a> and started to find people I want to follow, both on Fosstodon itself and on other servers. One really nice thing about Mastodon is that in addition to curating your own home feed with people you follow and filters of things you don't want to see you can also follow and interact with the local stream of all "toots" (messages) from all users on your local server. With a themed server like Fosstodon where most users share a common interest this seems to give you two interesting streams. You can also follow the federated stream of toots from all servers followed by any user on your server. On Fosstodon, which has almost 30,000 users, there is a good chance we collectively are following at least one user on a great number of other servers which makes the federated stream massive. I haven't quite figured out how to use it or if I like it yet.</p><p>So, am I to quit Twitter now? For the moment I haven't cut anything, I've just added Mastodon to my collection of social media, but I'm pretty sure I'll spend more time on Mastodon and less in any of the others. Perhaps a close down of my Twitter account might happen in the future if I see it not adding any value.</p><p>Another interesting side effect of my activities this week is that I've rediscovered my old blog. And here I am, writing a new blog post for the first time in almost ten years. Is this a one time wonder? I don't know, but my newly found ambition is to take up blogging again. I think it would be fun to blog about what I'm currently doing at work, such as Software Architecture Visualizations using Simon Browns C4 model and Structurizr, Consumer Contract Testing with PACT and growing Development Efficiency within our IT-organization, among other things. Only time can tell if I will succeed but if you are interested in finding out <a href="https://fosstodon.org/@se_thinking" target="_blank">follow me on Mastodon</a> (or <a href="https://twitter.com/se_thinking" target="_blank">on Twitter</a>).</p><p><br /></p>Jörgen Anderssonhttp://www.blogger.com/profile/10229551989831053791noreply@blogger.com0tag:blogger.com,1999:blog-5418258901636981690.post-79490893755013719652013-07-13T11:44:00.001+02:002013-07-13T11:44:27.598+02:00What are the layers for?<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: Arial; font-size: 15px; line-height: 1.15; white-space: pre-wrap;">As I’ve described in<a href="http://se-thinking.blogspot.se/search/label/Software%20Architecture" target="_blank"> earlier posts</a> I like to use a <a href="http://alistair.cockburn.us/Hexagonal+architecture" target="_blank">Hexagonal</a> (or <a href="http://jeffreypalermo.com/blog/the-onion-architecture-part-1/" target="_blank">Onion</a>) architecture when building enterprise back-end systems. The reason for that is that it places the domain model in the center and surrounds it with whatever integration code is needed to make it work in the technical environment where the system is deployed. It also makes the domain model totally independent of anything outside itself, which makes it super easy to thoroughly test drive without any execution environment.</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">In a recent discussion (as in so many others) we pretty soon got down to the question why we need all the layers. The simple reason for that is that every layer has its distinct reason to exist, so I figured the best way to motivate them is to describe those reasons on a per layer basis. So here it goes.</span></div>
<b style="font-weight: normal;"><br /><span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"></span></b>
<br />
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b>Domain</b></span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">This is the center, the heart, of the system. The sub-title of <a href="http://se-thinking.blogspot.se/2011/07/software-engineering-essential-reading.html" target="_blank">Eric Evans’ wonderful book</a> is “Tackling complexity in the heart of software” and that is what <a href="http://dddcommunity.org/" target="_blank">Domain Driven Design</a> is all about. The book is full of good advice on how to do that, but from a layering perspective I think the number one technique is to separate the inherent complexity of the domain (i.e. the complexity of the business) from the accidental complexity that we introduce by using computers to solve the problem of the business (i.e. the complexity of using a database, messaging service or deploying as a web application). In short: do not mix code implementing business logic with code handling technical matters! Hence, the domain layer is where all the business logic, and nothing but the business logic, goes. Any service needed by the domain, e.g. a repository for storing entities, are defined as an interface, in business terms.</span></div>
<b style="font-weight: normal;"><br /><span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"></span></b>
<br />
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b>Infrastructure (part of the integration layer)</b></span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Surrounding the domain layer we have the code responsible for integrating to the “outside world”, the “anti-corruption layer” in Evans’ terms. I commonly call this layer “integration”, but it falls into different parts. One is the “infrastructure” or “persistence” and another is “application”.</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">“Infrastructure” or “persistence” commonly is about integrating with a database or messaging infrastructure but it could also be about integration with other systems. In short, this is the layer where any services defined as interfaces in the “domain” get their technical implementation. In this layer there should be nothing but technical concerns, and we must be careful not to let any business logic leak out to this layer.</span></div>
<b style="font-weight: normal;"><br /><span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"></span></b>
<br />
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b>Application (part of the integration layer)</b></span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">This is the part of the integration-layer where requests to the domain model come in from the “outside world”. Just like integrating with a database or another system is a matter of converting between different models (the purpose of the “anti-corruption layer” described by Evans) this part of the integration layer is also solely about conversion between the model of the incoming request and the domain model. This conversion can be split into three parts:</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">- </span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><i>Data conversion</i></span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">: Incoming data are converted to their counterparts in the domain model and later back to the data format of the return value.</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">- </span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><i>Functional conversion</i></span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">: The incoming service call is “converted” into one or several calls to the domain model. This often includes finding an entity from a repository, calling a business method on it and storing it back in the repository. But it could also translate into a single call to a domain service.</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">- </span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><i>Error handling</i></span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">: Internally the domain model most likely uses unchecked exceptions with specific meaning to the business. In the “outside world” other error handling mechanisms might be in use that requires conversion.</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Other than that the application services also tends to be responsible for starting and committing transactions.</span></div>
<b style="font-weight: normal;"><br /><span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"></span></b>
<br />
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><b>Wrap-up</b></span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: Arial; font-size: 15px; line-height: 1.15; white-space: pre-wrap;">In this post I’ve tried to show the very different responsibilities of each layer (application, domain and infrastructure) in my typical back-end systems architecture. This is an approach I’ve been using successfully for several years in a non-trivial system and although it has many properties similar to Hexagonal architecture, it doesn’t adhere to it in every bit. The reason for that is not in any way scientific, it is purely because I didn’t know about it at the time where I set out to design the architecture of the system I was working on to support the use of DDD (and then first and foremost separation of business and technical concerns) in the best possible way. Therefore the basis of the architecture described here, is the concepts of a Domain Model in a Bounded Context surrounded by an Anti-Corruption Layer as described by Eric Evans in his book.</span></div>
<div>
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span></div>
Jörgen Anderssonhttp://www.blogger.com/profile/10229551989831053791noreply@blogger.com2tag:blogger.com,1999:blog-5418258901636981690.post-68983853469323079912012-09-26T20:39:00.000+02:002012-09-26T20:39:14.871+02:00DDD on top of Services, DDD inside Services - A Domain Driven approach to SOA<div class="MsoNormal">
<span lang="EN-US">This post
is about Domain Driven Design (DDD) and Service Oriented Architecture (SOA) and
how the former can be used to build services for the latter. But it starts of
in a few observations about current state of the union.<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-US"><br /></span></div>
<div class="MsoNormal">
<span lang="EN-US"><b>The
"DDD needs a database" assumption</b><o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-US">I've come
to find it being a pretty common understanding that software systems using DDD
building blocks have to be backed by a database, and, even more specific,
having repository implementations using an ORM framework. This leads to the
assumption that domain entities, once added to the repository, are attached to
an EntityManager that is responsible for "auto-magically" saving
changed data as transactions are completed. I find these assumptions bringing
to many constraints on a domain driven system architecture without any need. In
fact, I can't find anything in the DDD approach, as put forward by Eric Evans,
that supports this understanding. Instead I see the domain model as a piece of
code completely free of any explicit or implicit dependencies on the
surroundings. For a couple of years I've been working using this approach and
in this post I will describe the architecture of such a system where repository
implementations rely on services published by other systems as well as storing
part of the data in a dedicated database.<o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span lang="EN-US"><b>SOA as
spaghetti on top of CRUD</b><o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-US">I've also
found Service Oriented Architectures (SOA) to be implemented as mostly CRUD
operations for fine grained data objects, or services with, even really
complex, business logic firmly placed in Transaction Scripts (TS). TS might be
good enough for simple things, but quite quickly the code turns into spaghetti
as complexity rises. Complex business logic is where DDD really shines, but
given the assumptions discussed above, many people don't see it as an option
when there is no underlying database, but a set of CRUD like services.<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-US">And there
we have another problem with many service implementations. When just using
simple DTOs as parameters and/or return values and presenting the client with a
CRUD-like service API, what support are we then giving the client-side
developer? A bunch of getters and setters that could be set in any combination!
How to know what (from a business point-of-view) makes up a valid object? And
how to know what are valid modifications? Such APIs force the client-side
developer to have intimate knowledge about the server-side implementation, and
changes to the API won't show up as compiler errors. When APIs that I depend on
change, I like the compiler to be able to notify me.<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-US">I think we
can do much better and offer the client-side developer much better support by
extending the service API with some DDD building blocks like Entities and Value
Objects. In this post I will explain how we did that, in effect implementing a
published domain.<o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span lang="EN-US"><b>Separation
of concerns - The independent domain model</b><o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-US">One of the
principles of DDD that I think are most important to apply is "Separation
of concerns". It is also part of Robert C Martin's<a href="http://en.wikipedia.org/wiki/SOLID_(object-oriented_design)" target="_blank"> SOLID</a> principles by the
name "<a href="http://en.wikipedia.org/wiki/Single_responsibility_principle" target="_blank">Single responsibility principle</a>". In short, every piece of code
should deal with one problem only and have only one reason to change. It is
also an important part of writing clean code since it increases clarity if a
piece of code only does one thing. In the DDD perspective I apply this
principle by separating code that models the business domain in order to solve
business problems from technical code that glues the application together or
handles communication with the outside world, such as databases or services
exposed by other systems. This leads to a system with a traditional layered
architecture with a slight twist: the domain model is kept in the center with
no outgoing dependencies whatsoever. The domain model is concerned only with
solving business problems, while the surrounding integration layer (or "anti-corruption
layer" in Evans' terms) is concerned only with technical issues. The
domain model is built using the DDD building blocks (entities, value objects,
repositories, services and factories), with repositories (and sometimes also
services and factories) represented as interfaces used by the domain, but
implemented in the integration layer. Thereby creating the dependency from the
infrastructure layer to the domain, and that's the twist! This is perhaps an
unusual architectural style, but it is not new. Similar models have previously
been described as “Hexagonal Architecture” and “Onion Architecture” among
others. <a href="http://blog.8thlight.com/uncle-bob/2012/08/13/the-clean-architecture.html" target="_blank">Robert C Martin has a nice summary of the history of this architecturalstyle on his blog.</a><o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-US">This
architectural style brings the following benefits:<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-US">- Easy to
read business logic, since it is not mixed with code for interaction with
databases, messaging services or other technical concerns. This caters for
fewer mistakes as the software evolves.<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-US">- Easy to
deploy. The domain code can be deployed in different runtime environments
without changes, since no dependencies exist. At one point we made great use of
this as we were moving to a new deployment platform while at the same time
developing new functionality for the business.<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-US">- Easy to
test business logic. All business logic can be tested by automation without any
need for a runtime container. Doesn't really have to argue why that is good...<o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span lang="EN-US"><b>Repository
on top of services</b><o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-US">Since the
domain is only concerned with business logic and the business doesn't care how
entities are stored, only the fact that they can be stored and at what step in
the business process that happens, all repositories in the domain are
represented as interfaces. It is now up to the integration layer to provide
implementations for those repositories. <o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-US">Regardless
whether entities are to be stored in a database, in files, or by calling a
remote service (In my case it was services exposed via a customer specific API
on top of stateless EJB, i.e. basically RMI, but it could be web services or
any other protocol as well), or a combination, a repository needs access to the
internal state of the entity in order to get (for storing) and set (for
re-constructing) values. Typically most of that state isn't public in the
domain model, so the repository implementation needs to gain access. Depending
on security settings, reflection might be an alternative. In most cases it is
not. Instead we made the members of the entity protected and created a specific
sub-class of the entity in the integration layer that we used both for reading
when state was not public, and for re-constructing entities on read requests.<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-US">With the
problem of gaining access to internal state solved it is pretty easy to write a
repository implementation that reads and writes most of the data over one or
several remote service calls and keeps
the rest in a few database tables in a dedicated database. This split storage
model is often a result from the fact that existing external services might not
fully support the needs of the domain model. Another reason might be
performance. In some cases we had to cache carefully selected data pieces in
our own database to ensure timely retrieval. The beauty of this approach is
that we can do all those tricks needed in the repository implementation without
having any part of it leak into the domain model code; the separation of
concern is total.<o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span lang="EN-US"><b>Published
domain and services with business meaning</b><o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-US">As
mentioned above, CRUD-oriented services that let a client store and retrieve
DTOs with getters and setter for every attribute push a great burden onto the
developer of the client code. If using DDD to implement the service internally,
we could do better by offering that domain knowledge to the client, packaged as
a published domain and services that carries business meaning. A CRUD-service
will never have a place outside the integration layer of the client, while the
signature of a service well crafted in business terms might be a good candidate
for a service API used directly in the domain model of the client. Of course in
the shape of an interface with a small implementation in the integration layer
to carry out the technical lifting of making a remote call. But the point is,
the service signature can be used verbatim and the integration layer can be
kept thin since the client domain doesn’t have to re-define what the service
means in business terms; hard earned domain knowledge is reused.<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-US">The same goes for the
business objects. If we publish those parts of the domain that can be used
outside our service implementation, we offer the client-side developer to
directly benefit from the domain knowledge we have gathered in building our
service. But what parts can be published? Well, if you think about it, the part
of a good API that is most useful is the limitations it imposes, i.e. the help
you get to avoid doing stupid things. By publishing a model of domain objects
as plain Java objects, with constructors and accessors ("getters",
but in business terms and not necessarily following the Java Beans convention),
we help the client-side developer to avoid constructing invalid objects and accidentally
changing state that, from a business perspective, should be immutable. With
only this much the client-side developer
gets much better support than with only the raw data format offered by our
stateless services. In addition it might also be appropriate to offer a thin
layer on top of the service calls that that exposes the services in terms of
these domain objects and takes care of transforming them to/from the raw format
used to go over the wire.</span><span lang="EN-US"><o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-US">Having a published
domain gives the client-side developer the choice to either just model
transformations of it into a more
suitable model for the client context or, if contexts are closely related, decide
to take on a conformist approach and extend the domain objects with additional
functionality. I've done both in different contexts and it is so much better
than having to experiment with DTOs to find out what are valid combinations of
attribute values.</span><span lang="EN-US"><o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span lang="EN-US"><b>Conclusion</b></span><span lang="EN-US"><o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-US">DDD is suitable for
implementing domains on top of external services. It is also suitable for the
implementation of such services, and if we carefully select parts of the domain
model to publish we offer great help to the client-side developer.</span><span lang="EN-US"><o:p></o:p></span></div>
<br />Jörgen Anderssonhttp://www.blogger.com/profile/10229551989831053791noreply@blogger.com1tag:blogger.com,1999:blog-5418258901636981690.post-36455964214119210812012-09-01T23:08:00.001+02:002012-09-01T23:09:39.323+02:00When is it appropriate to design a service?<br />
<div class="MsoNormal">
<span lang="EN-US">In a system
architecture using Domain Driven Design (DDD) you typically find a few typical
building blocks (stereotypes) - Entities, Value Objects, Repositories and
Domain Services – where the first two are stateful objects and the rest are
stateless services implemented either as infrastructure services outside the
domain (Repositories) or inside the domain containing only business logic
(Domain Services). A common question is “When is it appropriate to design a
service instead of placing the business logic in an entity or value object?”<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-US"><br /></span></div>
<div class="MsoNormal">
<span lang="EN-US">For
developers more used to building procedural designs, rather than object-oriented
ones, it seems to be more natural to place logic in stateless services and have
them operate on objects that are no more than data containers. This is what is
commonly referred to as an “<a href="http://martinfowler.com/bliki/AnemicDomainModel.html" target="_blank">anemic domain model</a>”. It is considered an
anti-pattern in the DDD community since it decouples data from behavior and
thereby produces a much less expressive and knowledge tense domain model.<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-US"><br /></span></div>
<div class="MsoNormal">
<span lang="EN-US">I'm not a
fan of stateless services in the domain model. Instead I try to favor bringing
business logic into the Entity or Value Object that holds the information
needed - in OO-terms it is called the "<a href="http://en.wikipedia.org/wiki/GRASP_(object-oriented_design)#Information_Expert" target="_blank">information expert</a>". In most
cases it isn’t that hard, especially if functionality is decomposed into short
methods, each allocated to the object representing the concept at heart of the
functionality.<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-US"><br /></span></div>
<div class="MsoNormal">
<span lang="EN-US">A specific
type of functionality that might be trickier to handle is entity creation. Who
is to be responsible? For sure, it can’t be the entity itself. In general my
experience is that there will be some sort of hierarchy between concepts. E.g.
a SalaryPeriod might be connected to an existing RegistrationPeriod, which also
contains submitted Timesheets. Then it is a good fit to have the
RegistrationPeriod handle creation of the SalaryPeriod. So in general it is almost
always possible to find a good place for rules regarding creation of an entity
in a parent concept. The same goes for functionality that has to operate over
several entities of a given type.<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-US"><br /></span></div>
<div class="MsoNormal">
<span lang="EN-US">However,
there might be cases where no suitable parent concept exists in the domain.
That is one of the cases where I find designing a domain service appropriate.
And there are others. Here is a small extract from a <a href="http://se-thinking.blogspot.se/2011/10/is-domain-driven-design-always.html" target="_blank">previous blog post of mine</a>
where I briefly describe another situation where I think domain services are a
good choice:<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-US"> "In general I think you could talk about
two types of systems, or parts of systems; those mostly concerned with changes
in object state and those mostly concerned with processing data streaming
through the system. </span>In the
first case entities, aggregates and repositories are a natural fit, in the
second I think transaction scripts (in DDD context called domain services,
since they do only concern domain logic, no infrastructure code [..]) are a
nice fit. When the most important feature is to crunch some data, perhaps
modify it and then route it further to some recipient (like another system or
some persistent store) I think it is the "processing pipeline", i.e.
the stateless service code, which should be emphasized. So in those cases the
internals of the data isn't very interesting and might be better left in some
simple DTO format."</div>
<div class="MsoNormal">
<span lang="EN-US"><br /></span></div>
<div class="MsoNormal">
<span lang="EN-US">To make the
list of appropriate service design complete I’ll end with adding a few lines on
external services. These services get injected into the domain. In the domain I
would have only an interface describing the service in terms of the domain. This
is the way to integrate with surrounding infrastructure or other systems. It is
the same pattern as with Repositories, in fact a Repository is just a specialized
service. <o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-US"><br /></span></div>
<div class="MsoNormal">
<span lang="EN-US">Another example
of an external service is when some part of the business logic, e.g. a
calculation is broken out into a separate service, implemented using another
programming language or paradigm. From a domain point-if-view it is as if we
get that calculation service from another system instead of doing it ourselves.
The reason might be performance or it might be that the logic already exists
and we want to continue using it instead of re-implementing. However, each time
this happens the maintenance burden is increased a bit, so I think it should be
done with careful consideration.<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-US"><br /></span></div>
<div class="MsoNormal">
<span lang="EN-US">To sum up,
favor business logic implementations in the domain objects, not in services. It
leads to a more elaborate and therefore more useful domain model which is
easier to keep consistent than logic spread over disparate services. I’m
convinced in the long run this approach makes maintenance of the system easier. <o:p></o:p></span></div>
Jörgen Anderssonhttp://www.blogger.com/profile/10229551989831053791noreply@blogger.com0tag:blogger.com,1999:blog-5418258901636981690.post-63872201239613207632012-08-08T22:13:00.000+02:002012-08-08T22:13:29.001+02:00How to handle reporting with Domain Driven Design?<br />
<div class="MsoNormal">
<span lang="EN-US">A pretty
common question regarding Domain Driven Design (DDD) is how to handle reporting
functionality in a system using a DDD-approach. As in most cases, the answer is
"it depends".</span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
If what we
are aiming for is easy to change reporting I would transfer data into a
BI-system and run reports from there. There is absolutely nothing to gain in
designing our own BI-tool. There are plenty of them in the market, and in
combination with some common data warehouse design patterns they do a good job
both extracting, storing and providing easy prepared and ad-hoc reporting. If
we do not need a fancy reporting interface just a separate relational database
schema would do for running some SQL-queries. The key is to keep reporting
separate from the business system. This is a good rule using DDD or not.</div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
If it is
more of "a small summary" or some accumulated totals that should be
shown inside the business application I'd try to keep it inside the domain
model. Most values, just like any attribute, would fit nicely inside an entity.
E.g. if we need to calculate an OrderSummaryByStock, it might be placed as
Stock.runningOrderSummary(). This makes it readable right of the entity it
belongs to. If running into performance problems I'd look into keeping those
numbers up-to-date as part of the command or update transaction, storing the
accumulated numbers in the database, as part of the entity.</div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
</div>
<div class="MsoNormal">
<span lang="EN-US">If you see
overall problems with read performance due to multiple reads per write I would
consider a <a href="http://martinfowler.com/bliki/CQRS.html" target="_blank">CQRS</a>-approach (with separate read-views kept up-to-date by events
exported from the domain model as it gets updated), at least for the views in
problem.<o:p></o:p></span></div>
<br />
Jörgen Anderssonhttp://www.blogger.com/profile/10229551989831053791noreply@blogger.com0tag:blogger.com,1999:blog-5418258901636981690.post-57970818400228490812012-01-28T20:38:00.000+01:002012-01-28T20:38:24.470+01:00Domain Driven Design and batch processing<div>In DDD the design of a system is very object centric and therefore focuses on individual objects (or aggregates of objects) that interacts through sending messages (commands, queries and events). This is very unlike traditional batch processing where one or several functions are applied iteratively to a batch of input data. Between the two there is a significant missmatch, but nevertheless, once in a while we need to offer a batch oriented interface to our domain logic or need to call a service that offers a batch interface.<br />
<br />
<b>Implementing a batch interface</b><br />
This is the easiest part. We just need to create a thin script that manages the iteration over the batch, for each entry makes a call to the proper application service (tasked with coordinating calls on the domain model) and collects any response data returned. All business logic needed to perform the batch operation is held inside the domain package, as usual. From a domain model point-of-view, the batch processing script is just another client calling the same application services as any online client would do to perform the same task. It is just that this one is making many requests over a short period of time.<br />
<br />
<b>Calling a batch interface</b><br />
In general, batch processing is just an asynchronous call. Yes, you combine several requests into one, but a call with just one request in the batch would still be a valid one. As long as there is no importance in which requests are made together and no relevance in the responses comming back together or aggregated in some way, that is. But then it isn't truly a batch, then it is just a single request with many input parameters. In the following I will discuss a possible solution for when the service is a true batch, i.e. serving many unrelated requests in one call, asynchronously.<br />
Let's consider a case where we have a type of domain object including a method that is to be implemented with a call to an external service. A service that happens to have a batch processing interface. From a domain perspective the nature of this method is asynchronous. We just don't know how long we will have to wait for the result. But the fact that the call is made in batches, and not one by one is an implementation detail to be handled by the infrastructure layer.<br />
The domain should be fully decoupled from the batch handling, which should be handled by infrastructure code. In the domain we define a service interface that takes the request for one domain object and returns nothing. This service could be called by an application service or by the domain object, or any other object or service in the domain. Then we define an event handler that gets notified when the result of the request arrive. The event handler would be responsible for taking appropriate action on the domain object depending on the result.<br />
In the infrastructure layer we will implement the service with a message queue and on the other side of that queue some code that combines individual requests into suitably sized batch calls. The frequency and size of those batches might be tuned for performance and response times. E.g. one batch for every X number of requests, but at least one batch every Y minutes provided at least one request has been made.<br />
Then we need a batch-driven routine that handles responses, splits them into individual messages and places them on a response queue for the event handler to process in the domain.</div>Jörgen Anderssonhttp://www.blogger.com/profile/10229551989831053791noreply@blogger.com0tag:blogger.com,1999:blog-5418258901636981690.post-76791912357770638742011-12-11T21:39:00.000+01:002011-12-11T21:39:18.085+01:00What about PMs in Scrum-based development?<div>In scrum terminology there is no such thing as a project manager (PM). However, in addition to the product owner (PO) and the scrum master (SM) <a href="http://blog.crisp.se/author/henrikkniberg">Henrik Kniberg</a> in his excelent <a href="http://www.infoq.com/minibooks/scrum-xp-from-the-trenches">"from the trenches"-book</a> talks about a "sponsor". Someone that provides for the teams long-term staffing and work environment as well as comitting to remove impediments outside the team. He describes himself, in his position as development line manager, as having this role.<br />
<br />
In my experience many large organisations are totally project organised bringing together people from different parts of the organisation as well as external consultants to form a team. In such organisations the traditional PMs, if accepting to have an empowered team, is often ideal for this sponsor role.<br />
That brings me to propose the following division of responsibility:<br />
<br />
<b>Product Owner:</b> Responsible for what the team is to build. Merges requirements from different stakeholders and manages priorities.<br />
<b><br />
Scrum Masters:</b> Guides the team and PO in following the agreed process. Makes sure the surroundings understand how the team works and do not interfere. The SM role is usually not full time and is therefore filled by a member of the team.<br />
<b><br />
Project Manager:</b> Responsible for long-term staffing and external contacts in order to provide the team with the best possible environment to meet the needs of the organisation.</div>Jörgen Anderssonhttp://www.blogger.com/profile/10229551989831053791noreply@blogger.com0tag:blogger.com,1999:blog-5418258901636981690.post-84187331505656885342011-10-21T22:29:00.000+02:002011-10-21T22:29:51.505+02:00Ideas and books forming me as a Software Engineer<div><br />
<span class="Apple-style-span" style="font-family: Arial, sans-serif;">This September it was 15 years since I joined university to study Software Engineering, the official name was “Informatics”, but I think software engineering (or authoring and gardening as argued in another post) is better describing what I was interested in and, luckily, what I'm currently doing. Since then I've got some 10+ years of experience from working in the industry and a whole 15 years of continuous learning. As an exercise for myself I've set out to go through the major ideas I've picked up and used along with the people and books that collaboratively have formed me to the software engineer I am today. Hopefully you find this post at least a bit interesting and inspiring as well.</span><br />
<span class="Apple-style-span" style="font-family: Arial, sans-serif;"><br />
</span><br />
<span class="Apple-style-span" style="font-family: Arial, sans-serif;"><b>Object Orientation (OO)</b></span><br />
<span lang="EN-US" style="color: black; font-family: "Arial","sans-serif"; mso-ansi-language: EN-US;"> In 1996, and the following years, <a href="http://en.wikipedia.org/wiki/Object-oriented_design">OO</a></span><span class="apple-converted-space"><span lang="EN-US" style="color: black; font-family: "Arial","sans-serif"; mso-ansi-language: EN-US;"> </span></span><span lang="EN-US" style="color: black; font-family: "Arial","sans-serif"; mso-ansi-language: EN-US;">was the new hot thing in software design, at least at my university. I learned to model domains (even though I didn't know that word by then) in objects using the OMT (<a href="http://en.wikipedia.org/wiki/Object-modeling_technique">Object Modeling Technique</a></span><span lang="EN-US" style="color: black; font-family: "Arial","sans-serif"; mso-ansi-language: EN-US;">) notation by <a href="http://en.wikipedia.org/wiki/James_Rumbaugh">James Rumbaugh</a><span class="apple-converted-space"> </span></span><span lang="EN-US" style="color: black; font-family: "Arial","sans-serif"; mso-ansi-language: EN-US;">(later on one of the main contributers to UML). This way of abstracting the real world into objects with attributes, methods and relations very much appealed to me as it offered the tools to unambiguously document the analysis result by drawing a diagram. I did also believe my teachers when they talked about OO as the enabler of universal libraries of small reusable objects. Now I, and I'm sure they too, know this is never going to happen, but the OO-paradigm still brings structure for reuse within a system and hence is a great tool for honoring the <a href="http://en.wikipedia.org/wiki/Don't_repeat_yourself">DRY-principle</a></span><span lang="EN-US" style="color: black; font-family: "Arial","sans-serif"; mso-ansi-language: EN-US;">. </span><br />
<span lang="EN-US" style="color: black; font-family: "Arial","sans-serif"; mso-ansi-language: EN-US;">Ever since those first courses I started to think of OO as <u>the</u> way to design and build software systems. Later on I've come to see OO as <u>the</u> way of designing and building transaction based information systems where manipulation of state is the primary focus. Other types of systems might be better suited to use some other paradigm, but this type of systems for administering information is what I've been working with almost the entire time since I graduated. In university I learned the theory of OO, but we didn't actually turn any of our OO-designs into working software. We sure learned programming, but then it was to solve more traditional algorithmic problems. When leaving university I was all eager to learn how to do OO in real world projects. And boy, was I disappointed!</span><br />
<span lang="EN-US" style="color: black; font-family: "Arial","sans-serif"; mso-ansi-language: EN-US;">For several years I was employed in projects where I learned lots of things about programming and projects in general but nowhere was OO used to anything but data diagrams (now drawn in UML notation which I learned through <a href="http://martinfowler.com/">Martin Fowler</a>'s “<a href="http://www.amazon.com/UML-Distilled-Standard-Modeling-Language/dp/0321193687">UML Distilled</a>”</span><span lang="EN-US" style="color: black; font-family: "Arial","sans-serif"; mso-ansi-language: EN-US;">), i.e. objects with attributes and relations but no behavior, that was later turned into database tables and data structures. All behavior was programmed into transaction scripts, either directly in the GUI components or as free standing functions or stored procedures. I was desperately looking for a real world example of true OO implementation because on my own I couldn't really figure out what it would look like in order to work properly. The first piece of the solution came when I got to read <a href="http://www.craiglarman.com/wiki/index.php?title=Main_Page">Craig Larman</a>'s “<a href="http://www.amazon.com/Applying-UML-Patterns-Introduction-Object-Oriented/dp/0131489062">Applying UML and Patterns</a>”</span><span class="apple-converted-space"><span lang="EN-US" style="color: black; font-family: "Arial","sans-serif"; mso-ansi-language: EN-US;"> </span></span><span lang="EN-US" style="color: black; font-family: "Arial","sans-serif"; mso-ansi-language: EN-US;">. In this book he describes all aspects of OOA/OOD, how the software can be structured with boundary classes, controllers and entities, and shows by example code how it all comes together. Still, this isn't the approach intuitively encouraged by the structure in popular frameworks such as <a href="http://www.oracle.com/technetwork/java/javaee/index.html">Java EE</a></span><span class="apple-converted-space"><span lang="EN-US" style="color: black; font-family: "Arial","sans-serif"; mso-ansi-language: EN-US;"> </span></span><span lang="EN-US" style="color: black; font-family: "Arial","sans-serif"; mso-ansi-language: EN-US;">and <a href="http://www.springsource.org/">Spring Framework</a></span><span lang="EN-US" style="color: black; font-family: "Arial","sans-serif"; mso-ansi-language: EN-US;">. Out-of-the-box they rather suggests a static service structure operating on <a href="http://en.wikipedia.org/wiki/Data_transfer_object">DTO</a>:s</span><span lang="EN-US" style="color: black; font-family: "Arial","sans-serif"; mso-ansi-language: EN-US;">, and that is, I think, the main reason why most systems are built with data and behavior separated and not using the true OO-paradigm.<o:p></o:p></span><br />
<span lang="EN-US" style="color: black; font-family: "Arial","sans-serif"; mso-ansi-language: EN-US;"><br />
</span><br />
<span lang="EN-US" style="color: black; font-family: "Arial","sans-serif"; mso-ansi-language: EN-US;"><b>Domain-Driven Design (DDD)</b><br />
In a <a href="http://www.jfokus.se/jfokus/index.jsp">Jfokus</a><span class="apple-converted-space"> </span></span><span lang="EN-US" style="color: black; font-family: "Arial","sans-serif"; mso-ansi-language: EN-US;">2008 tutorial I was first introduced to <a href="http://domaindrivendesign.org/resources/what_is_ddd">Domain-Driven Design</a></span><span lang="EN-US" style="color: black; font-family: "Arial","sans-serif"; mso-ansi-language: EN-US;">. The three hour tutorial only scratched the surface of DDD but it was enough for me to understand that this was the description of how the full OO-paradigm is used to build real world systems. Soon after, I got to read <a href="http://www.citerus.se/profile/211234-eric-evans">Eric Evans</a>' book “<a href="http://www.amazon.com/Domain-Driven-Design-Tackling-Complexity-Software/dp/0321125215">Domain-Driven Design - Tackling complexity at the heart of software</a>”</span><span lang="EN-US" style="color: black; font-family: "Arial","sans-serif"; mso-ansi-language: EN-US;">, which I have <a href="http://se-thinking.blogspot.com/2011/07/software-engineering-essential-reading.html">posted about previously</a></span><span lang="EN-US" style="color: black; font-family: "Arial","sans-serif"; mso-ansi-language: EN-US;">, and since then I firmly believe that DDD is the approach to use for designing and implementing complex software systems.</span><br />
<span class="Apple-style-span" style="font-family: Arial, sans-serif;">DDD is not a method, nor is it simply a technique or an architectural pattern. I would rather call it an approach to software development, including analysis, design and implementation.</span><br />
<span lang="EN-US" style="color: black; font-family: "Arial","sans-serif"; mso-ansi-language: EN-US;"> Perhaps what most of my colleagues first think of when DDD is mentioned is the design sketches I use to draw with the business domain at the center and technical integration packages all around. The domain being built on the conceptual building blocks, entities, value objects, repositories and services, as introduced by Eric Evans. But even though that part of the approach might be the easiest to pick up, and also important in building working software with DDD, I think what makes the biggest difference to me is the <a href="http://domaindrivendesign.org/node/132"><i>ubiquitous language</i></a></span><span lang="EN-US" style="color: black; font-family: "Arial","sans-serif"; mso-ansi-language: EN-US;">. This practice of building a model with a shared language based on, but with more precise definitions than, the language spoken by the domain experts is the basis of my philosophy to always build the software so that structure and logic follows the business domain. I think that is the only way to guarantee a flexible design such that a small change to the business is always a small change to the software, never a big one. My experience is that the second you deviate from that principle, most often due to time pressure or pure laziness, you are asking for trouble further down the road. It might take a year or so, but it will come back and bite you!<o:p></o:p></span><br />
<span lang="EN-US" style="color: black; font-family: "Arial","sans-serif"; mso-ansi-language: EN-US;"><br />
</span><br />
<span lang="EN-US" style="color: black; font-family: "Arial","sans-serif"; mso-ansi-language: EN-US;"><b><a href="http://en.wikipedia.org/wiki/Test-driven_development">Test Driven Development</a> (TDD)</b><br />
If DDD provides the approach on analysis, design and implementation, TDD is clearly what integrates that approach with quality awareness and assurance. TDD is my way to ensure a testable design and a correct implementation that is easy and secure to improve further. In a previous post, "<a href="http://se-thinking.blogspot.com/2011/01/are-you-test-infected.html">Are you testinfected?</a>"</span><span lang="EN-US" style="color: black; font-family: "Arial","sans-serif"; mso-ansi-language: EN-US;">, I told the story of how I got introduced to TDD, how I was skeptical at first and how I later on proved the usefulness to myself. Now I rarely, and definitely reluctantly, writes any code without first writing a failing test.<o:p></o:p></span><br />
<span lang="EN-US" style="color: black; font-family: "Arial","sans-serif"; mso-ansi-language: EN-US;"><b><br />
</b></span><br />
<span lang="EN-US" style="color: black; font-family: "Arial","sans-serif"; mso-ansi-language: EN-US;"><b>A Clean Coder producing Clean Code</b><br />
Ever since the start of my career I’ve wanted to produce code that I’m proud of. However, the benchmarks for that assessment have changed over the years. Now I consider well tested and easy to read code being the standard to achieve. Since any piece of code is read much more often than it is written I think readability is prime quality measurement for code. Lately I’ve been reading both <a href="http://www.amazon.com/Clean-Code-Handbook-Software-Craftsmanship/dp/0132350882">Clean Code</a> </span><span lang="EN-US" style="color: black; font-family: "Arial","sans-serif"; mso-ansi-language: EN-US;">and <a href="http://www.amazon.com/gp/product/0137081073/ref=pd_lpo_k2_dp_sr_1?pf_rd_p=1278548962&pf_rd_s=lpo-top-stripe-1&pf_rd_t=201&pf_rd_i=0132350882&pf_rd_m=ATVPDKIKX0DER&pf_rd_r=1SN94W5C5TW8E2ZT27N1">The Clean Coder</a></span><span lang="EN-US" style="color: black; font-family: "Arial","sans-serif"; mso-ansi-language: EN-US;"> by <a href="http://www.objectmentor.com/omTeam/martin_r.html">Robert C Martin</a></span><span lang="EN-US" style="color: black; font-family: "Arial","sans-serif"; mso-ansi-language: EN-US;">. Both are excellent books. The first one giving good advice on how to make the code readable, the second really demanding reflection on what it means to be a professional software developer.<o:p></o:p></span><br />
<b><br />
</b><br />
<b><span lang="EN-US" style="color: black; font-family: "Arial","sans-serif"; mso-ansi-language: EN-US;"><a href="http://en.wikipedia.org/wiki/Agile_software_development">Agile</a></span><span lang="EN-US" style="color: black; font-family: "Arial","sans-serif"; mso-ansi-language: EN-US;"> with <a href="http://en.wikipedia.org/wiki/Scrum_(development)">Scrum</a></span><span lang="EN-US" style="color: black; font-family: "Arial","sans-serif"; mso-ansi-language: EN-US;"> and <a href="http://en.wikipedia.org/wiki/Kanban_(development)">Kanban</a></span></b><span lang="EN-US" style="color: black; font-family: "Arial","sans-serif"; mso-ansi-language: EN-US;"><br />
Another interest of mine is software development processes and techniques. At first I learnt how to deliver in a world of waterfalls, later I got to experience <a href="http://en.wikipedia.org/wiki/IBM_Rational_Unified_Process">RUP</a></span><span lang="EN-US" style="color: black; font-family: "Arial","sans-serif"; mso-ansi-language: EN-US;">. Despite all bad things said about RUP, I’ve always found the core messages on iterative, risk driven development to be right and frankly quite agile. However, agile approaches such as Scrum, Kanban and <a href="http://en.wikipedia.org/wiki/Extreme_Programming#Practices">developmenttechniques gathered in XP</a></span><span lang="EN-US" style="color: black; font-family: "Arial","sans-serif"; mso-ansi-language: EN-US;"> take this a long way further. The major inspiration here has been <a href="http://blog.crisp.se/author/henrikkniberg">Henrik Kniberg</a></span><span lang="EN-US" style="color: black; font-family: "Arial","sans-serif"; mso-ansi-language: EN-US;">, first in his Jfocus tutorial in 2008 and through his two books on <a href="http://www.infoq.com/minibooks/scrum-xp-from-the-trenches">Scrum</a></span><span lang="EN-US" style="color: black; font-family: "Arial","sans-serif"; mso-ansi-language: EN-US;"> and <a href="http://blog.crisp.se/2009/04/03/henrikkniberg/1238795520000">Kanban</a></span><span lang="EN-US" style="color: black; font-family: "Arial","sans-serif"; mso-ansi-language: EN-US;">. <br />
Currently I’m working in a world of mixing and matching. I’m picking the best parts out of all those above to create a process and way of working that supports being as agile and lean as possible in an otherwise RUP-ified world. I think, at the current state it is somewhat of coming success.<o:p></o:p></span><br />
<span lang="EN-US" style="color: black; font-family: "Arial","sans-serif"; mso-ansi-language: EN-US;"><b><br />
</b></span><br />
<span lang="EN-US" style="color: black; font-family: "Arial","sans-serif"; mso-ansi-language: EN-US;"><b>“All problems are not worth solving”</b><br />
That was one of the advice I got from an experienced colleague of mine many years ago when I was aspiring to go from being a sheer hacker to become a <a href="http://se-thinking.blogspot.com/2011/03/only-certification-im-proud-of.html">CapgeminiCertified Software Engineer</a></span><span lang="EN-US" style="color: black; font-family: "Arial","sans-serif"; mso-ansi-language: EN-US;">. He probably gave me a bunch of other advice too, but this is the one I still remember. And I also think, for me personally, it might be the most valuable advice I’ve ever got. Since problem solving is my trade I happily go of designing solutions, then it is good to remember thinking for a minute or two about whether the problem really deserves being solved or if it is something we can as happily live with. At the bottom of this is that all resources are limited and needs to be applied where they make the best return on investment. On the other hand, we need to be careful not to spend too much time on deciding if a problem is worth solving, i.e. it is not always worth solving the prioritization problem if the solution offered is cheap enough.<o:p></o:p></span><br />
<span lang="EN-US" style="color: black; font-family: "Arial","sans-serif"; mso-ansi-language: EN-US;"><b><br />
</b></span><br />
<span lang="EN-US" style="color: black; font-family: "Arial","sans-serif"; mso-ansi-language: EN-US;"><b>The future</b><br />
This pretty much sums up where I am today, but of course I will continue to evolve and in the near future I see the following ideas as the most interesting.</span><br />
<span lang="EN-US" style="color: black; font-family: "Arial","sans-serif"; mso-ansi-language: EN-US;">- <a href="http://domaindrivendesign.org/strategicdesign">Strategic design</a></span><span lang="EN-US" style="color: black; font-family: "Arial","sans-serif"; mso-ansi-language: EN-US;"> – How DDD concepts such as <a href="http://domaindrivendesign.org/node/91">Bounded Context</a></span><span lang="EN-US" style="color: black; font-family: "Arial","sans-serif"; mso-ansi-language: EN-US;"> is applied at a larger scale to make strategic decisions on how to factor architectures and apply different design and development strategies. In most discussions “architecture” seems to mean frameworks, databases, application servers and technical layering. I think architectures would be much more interesting and usable if they concerned business domains.</span><br />
<span lang="EN-US" style="color: black; font-family: "Arial","sans-serif"; mso-ansi-language: EN-US;">- <a href="http://martinfowler.com/bliki/CQRS.html">CQRS</a></span><span lang="EN-US" style="color: black; font-family: "Arial","sans-serif"; mso-ansi-language: EN-US;"> – I have only touched the surface of this approach to structure systems with separated command and query sides and event stores and messaging instead of relational database schemas and object/relational mappers. However, I think is an interesting approach and I definitely plan to dig in deeper.</span><br />
<span lang="EN-US" style="color: black; font-family: "Arial","sans-serif"; mso-ansi-language: EN-US;">- <a href="http://en.wikipedia.org/wiki/Data,_Context_and_Interaction">DCI</a> </span><span lang="EN-US" style="color: black; font-family: "Arial","sans-serif"; mso-ansi-language: EN-US;"> – Even though DDD has come to be my preferred approach I must admit the structure of the code base with behavioral code localized in every domain object has its draw backs. With requirements based on use cases or user stories, it is sometimes hard to answer the not so infrequent question “how is this UC implemented?”. DCI (Data, Context, Interaction) solves this problem through keeping the behavioral code in role objects implemented within a context representing a single UC. The code in the role objects are then injected into the data objects representing the domain. To me, it sounds like an interesting add-on to the ideas of DDD and in the future I like to continue exploring the benefits of this approach and the demands it poses on the implementation language.</span><br />
<span lang="EN-US" style="color: black; font-family: "Arial","sans-serif"; mso-ansi-language: EN-US;">- <a href="http://en.wikipedia.org/wiki/Functional_programming">Functional programming</a></span><span lang="EN-US" style="color: black; font-family: "Arial","sans-serif"; mso-ansi-language: EN-US;"> – In order to build simpler programs that are safe to parallelize on multiple cores functional programming has gained new interest in the last years. Up until now I’ve lived only in the world of imperative programming and grasping the concepts and underlying mechanics of functional programming will pose a challenge. A challenge I’m happy to take on.</span><br />
<span lang="EN-US" style="color: black; font-family: "Arial","sans-serif"; mso-ansi-language: EN-US;">- <a href="http://www.scala-lang.org/">Scala</a></span><span lang="EN-US" style="color: black; font-family: "Arial","sans-serif"; mso-ansi-language: EN-US;"> – Of all the new languages having emerged on the JVM I think Scala is the most interesting. It combines functional programming with additional powers in OO-programming without losing the benefits of static typing. All of which I think are interesting qualities. I think Scala might be the enabling programming language for many of the ideas I listed here for future exploration.</span><br />
</div>Jörgen Anderssonhttp://www.blogger.com/profile/10229551989831053791noreply@blogger.com0tag:blogger.com,1999:blog-5418258901636981690.post-28866075967195993852011-10-18T21:37:00.001+02:002022-11-19T17:55:10.120+01:00The value of using Domain Driven Design<div><p>From time to time I come across the relevant question of the value of using Domain Driven Design. Are there any hard numbers to use in convincing management about the benefits of DDD?</p>
<p>Hard numbers are really hard to get in software development since you never ever develop the same system twice and even if you did, it wouldn't be the same since you would have learnt a lot from the first attempt. But I'll share some thoughts and observations on using DDD.</p>
<p>By emphasizing communication with domain experts in developing the ubiquitous language DDD helps get you started on the right foot. Any system is simple from the beginning. The first set of functionality is never complex to implement since you have no other code to conform to and you tend to start with something basic. It means it is easy to just hack something together mixing levels of abstraction and business logic with technical complexity. Since first requirements are simple, the first complexity tends to come from unfamiliar tools like ORMs or other technical frameworks. Since we all are technicians at heart these things often become most important and the architecture tends to be more about technical concerns than the domain. All of which, we know, lead to a mess when requirements get more complicated.</p>
<p>By using DDD we are turning the focus away from technical concerns and instead focusing our effort on the business domain. With a bit of domain modelling, including test driven implementations, we get to investigate a larger set of requirements before going into technical details of persistence and other layers. A lean principle is to delay hard to change decisions to the last responsible moment, e.g. not deciding on a hard to change persistence model until we know what the domain really looks like. All in all, this leads to better architecture because the architecture is centered around the domain, not around technical frameworks.</p>
<p>Another aspect is complexity. What you consider complex depends on what you already know. I once worked with a large system where some parts were implemented using DDD and other parts were, what I would call, "undisciplined" transaction scripts. I think TS have their proper place and use, but implementing them in a way mixing business logic with technical concerns is never right. Hence "undisciplined". In this project I heard people complain about different things they thought was complex. Those people being unfamiliar with DDD but having a long history of working on the system thought the TS code with mixed concerns and very few abstractions was fine since they saw exactly what was going on and they "knew" what the context was and how everything worked on a detailed level. And they thought the DDD approach of separating things into different layers and building abstractions for business concepts just added complexity.</p>
<p>Then we had the newcomers, that didn't have any prior DDD-experience either, that struggled with understanding the domain. Even though DDD was new to them they pretty quickly picked up on the concepts and thought it was a great help in understanding the domain. To them it was the TS code that was most complex since it didn't make any difference between lines of business logic and those of database interaction.</p>
<p>These are two aspects on "the value of DDD". In <a href="https://se-thinking.blogspot.com/2011/10/is-domain-driven-design-always.html" target="_blank">another post</a> I wrote about DDD being so much more than a set of technical building blocks and I think the most value from DDD is gained when wisely deciding on what parts to apply in each unique situation.</p>
</div>Jörgen Anderssonhttp://www.blogger.com/profile/10229551989831053791noreply@blogger.com0tag:blogger.com,1999:blog-5418258901636981690.post-45636662113334230372011-10-05T06:22:00.003+02:002022-11-19T17:53:08.848+01:00Is Domain Driven Design always applicable?<div>Disclamer: This blog post discusses use of DDD in information centric systems, which in my experience is most of the systems out there. It does probably not apply to technical software such as compilers and operating systems.<br />
<br />
In my opinion the answer to the question in the title depends on how you define DDD. Do you do DDD if you do not use all technical building blocks? I think so. Is DDD more than the tachnical building blocks? I definately think so.<br />
<br />
Personally, I would opt for using DDD in almost any situation but those that I know for sure will not evolve for more than a day or so and where I'm dead sure of the requirements. In any other situation I would start building the Ubiquitous language together with the domain experts to get a common vocabulary. This doesn't sound so much, but it is huge to avoiding expensive misunderstandings.<br />
<br />
For the systems architecture I would always deploy the strategy to keep business logic and infrastructure code separated. It makes wonders for readability and testability. My rule of thumb is that business logic should be easily testable using unit tests (and end-to-end tests independent of the run-time environment) and infrastructure code should be so simple you generally need no other testing but a few basic integration tests.<br />
<br />
For design my strategy is to keep the code close to the business in terminology, structure and logic. This is important to make sure that the impact of a change is about equal in size in both business and software. Here the ubiquitous language is an absolute need.<br />
<br />
Then we come to the technical building blocks and now it is time for some choices. In general I think you could talk about two types of systems, or parts of systems; those mostly concerned with changes in object state and those mostly concerned with processing data streaming through the system.<br />
<br />
In the first case entities, aggregates and repositories are a natural fit, in the second I think transaction scripts (in DDD context called domain services, since they do only concern domain logic, no infrastructure code, as per discussed above) are a nice fit. When the most important feature is to crunch some data, perhaps modify it and then route it further to some recepient (like another system or some persistent store) I think it is the "processing pipeline", i.e the stateless service code, that should be emphazised. So in those cases the internals of the data isn't very interesting and might be better left in some simple DTO format.<br />
<br />
Whether going with entities or servicies I think the practice of using typed value objects for input parameters give a lot in terms of clarity and IDE-support when writing and reading the code. From a technical perspective, this is the easiest DDD building block to use and I think it has great advantages even if it is the only part of DDD that you apply to the system.<br />
<br />
In conclusion, for the types of information systems I've been developing over the years DDD is the way to do it, however that doesn't necessarily mean everything should be entities or value objects. DDD is so much more.</div>Jörgen Anderssonhttp://www.blogger.com/profile/10229551989831053791noreply@blogger.com0tag:blogger.com,1999:blog-5418258901636981690.post-49723391112927553262011-08-28T20:51:00.001+02:002011-09-07T06:42:13.406+02:00Unit vs Integration tests<div>
Lately I've spent some time thinking and discussing pros and cons of unit and integration tests. Basically I think you need both but they have totally different characteristics and should therefore be combined carefully. Before we dive in, lets define the terms for the sake of this discussion since I know there are at least as many definitions of "integration test" as there are projects out there.<br />
<b>Unit test:</b> Test focused on one or a closely related set of classes. Defined and run <i>outside</i> any run-time environment (container). All dependencies are mocked/stubbed.<br />
<b>Integration test:</b> Test focused on one or several (sub)systems. Defined in code and run <i>in</i> the run-time environment. Some, but not all, dependencies might be stubbed.<br />
With definitions done, let's go for some characteristics I've found to be true.<br />
<b>Unit test - pros:</b><br />
1: Small units (single classes or a small set) are tested independently without interferences from the surrounding environment. <br />
2: Can be run instantly with a few keystrokes - no deployment necessary, the whole (sub-) system doesn't even have to compile. Makes it easy to test incrementally. <br />
3: Supports Test Driven Development. <br />
4: With small units it is easier to cover all interesting execution paths.<br />
<b>Unit test - cons:</b> <br />
1: Mocking frameworks could be a challenge to master, but they are oh-so useful when you get to know them. <br />
2: Different tests must be in sync to ensure that individual units work together. Can be tricky, especially when dealing with semantic changes in the interface of a class/unit. Requires disciplined top-down TDD.<br />
A colleague of mine pointed out that my con #1 actually could be a pro if using TDD (as in writing test and production code simultanouosly) since having a hard time mocking dependencies would be an incentive towards writing code with fewer dependencies, which in general is a good design practice. However, my experience is that a good mocking framework is necessary to effectively write unit tests, and as with every tool there is learning curve. Now for integration tests.<br />
<b>Integration test - pros:</b><br />
1: Tests the code in its true context with few or no stubbed dependencies. <br />
2: Supports testing of non-functional requirements, such as performance, security and transaction boundaries.<br />
<b>Integration test - cons:</b> <br />
1: Can only be run after the (sub-) system has been fully developed, built and deployed. Tests are written as an afterthought instead of driving the development. <br />
2: The code-build-deploy-test-cycle takes a long time which make debugging cumbersome. <br />
3: Might require other parts of the system than those under test to be fully functional in order to bring the system into the state for the test to start. This creates strong dependencies on parts of the system that are not targeted by the test.<br />
Most of my cons for integration tests only become severe when integration tests are used to test business logic, which I've seen from programmers claiming unit tests with mocked dependencies being too hard to write. When used to test real integration issues (such as inter-system communication or database connectivity), the cons sort of goes away since those issues cannot be detected anyway prior to having at least a "walking skeleton" implementation of the architecture.<br />
In conclusion, I think both unit and integration tests (as well as automated systems/acceptance tests) are really needed in a software project. However, each need to be used propely. For ensuring implementation of business logic I think unit tests are the proper choice. It might be a bit harder to write because you would have to mock dependencies instead of relying on the state currently set up in the database, but at the time of adding functionality or refactoring your domain code unit tests are so much easier and thereby cheaper to run since you do not need to bring your complete system into a known state, you do not even have to compile, let alone install, your complete system. Striving for a good coverage (some say a 100%, I don't) of your domain logic, preferably using a test-first approach, you also tend to achieve a better design since well designed objects are easier to test.<br />
Side note: Today we got lucky like a crazy when we found a bug in the business logic because we by chance happened to run the integration test designed to test that feature. It could be, and was, claimed to be an argument for the benefit of using integration tests for testing business logic. However, I think it is not. For starters, the bug had been in the code for about five months. Not even the guy writing the feature knew were in the system the code was located. We had to spend several hours to find the implementation. After that, the bug was easy to spot. It gives a perspective on cycle time and why quick feedback is important. Secondly, the bug could not be found by unit test since there was no coverage on this part of the system. I'm pretty sure the reason for that is that it is really hard to write unit tests for code directly manipulating a JPA entity manager. So, the problem was really that business logic was burried in the database access code and hence not unit tested. If a unit test had been attempted I'm pretty sure the logic had been moved into a more test friendly design. Which of course would have been better from other perspectives as well.</div>
Jörgen Anderssonhttp://www.blogger.com/profile/10229551989831053791noreply@blogger.com0tag:blogger.com,1999:blog-5418258901636981690.post-17727396411965378842011-07-11T06:51:00.001+02:002011-07-11T07:00:05.690+02:00Software Engineering Essential Reading 2 - Domain Driven DesignIn 2004 <a href="http://twitter.com/#!/ericevans0">Eric Evans</a> published his book “<a href="http://www.amazon.com/Domain-Driven-Design-Tackling-Complexity-Software/dp/0321125215">Domain Driven Design – Tackling complexity at the heart of software</a>”. It's quite a heavy bit of literature, but every page is filled with insightful discussions of the benefits and practices on designing object oriented software with a model true to the business domain, and thereby worth every minute spent reading and reflecting. Eric has solid experience from developing software in support of complex business domains and he openly shares and discusses both success and failures.<br />
The book discusses domain driven design (DDD) on several different levels:<br />
<br />
<ul><li>The importance to use the same domain model in all aspects of a project – in writing, in the spoken language and in the source code.</li>
<li>The stereotyped building blocks used in modeling, design and implementation of domain centric software systems. Entity, Value Object, Repository, Service, Aggregate and a few more.</li>
<li>How to use Strategic design and concepts such as Bounded Context to define current and future logical systems or enterprise wide architectures without having to unify the whole enterprise in a single domain model.</li>
</ul><br />
The book is the definitive starting point in learning and applying DDD, one of the really important design paradigms now and in the future. This book, together with the active community, both <a href="http://groups.google.com/group/dddsverige?pli=1">locally</a> and <a href="http://domaindrivendesign.org/">on the Internet</a>, has helped me in successfully designing and deliver complex software under complex technological and organizational circumstances. This is definitely one of the books I'm happy to come back to now and then and I think every professional software designer should have a copy. Before I first read it I was wondering why no one out in the “real” world used OO-design even remotely similar to what I learned and practiced in university. In his book Eric put this straight and he made me instantly feel “at home” with a true OO-design, carefully tuned to model the businesses, as the way to go about software design. Thanks Eric for sharing!Jörgen Anderssonhttp://www.blogger.com/profile/10229551989831053791noreply@blogger.com0tag:blogger.com,1999:blog-5418258901636981690.post-58299557071314573702011-07-07T07:27:00.000+02:002011-07-07T07:27:01.717+02:00Value Producers and SupportersIn the lean and agile world anything that produces value for the client is a good thing, everything else should be carefully scrutinized before carried out. This doesn’t mean you shouldn’t produce any documentation, it just means you should produce only the documentation that someone is likely to read. It doesn't mean you shouldn't produce any automated tests, it just means you should make sure you produce them in such a way you optimize value during the development period and for future maintenance. To do this right you need to understand which part of the project organization is producing direct value for the client and which part is supporting the Value Producers.<br />
<br />
In many projects you could find three groups of Value Producers: programmers who produce code that builds the product, testers who make sure the product does what it is supposed to do and technical writers who produce the user documentation. Everyone else, requirements analysts, architects, designers, etc, are usually in a supportive role. The Supporters does not themselves produce anything of direct value to the client, however that doesn’t mean that their work is not important. It is. As long as they make it easier for the Value Producers to do their work, now and in the future.<br />
<br />
That said one of the challenges in running an efficient agile project is to understand who is supporting who and how to make it as easy as possible to produce maximized value for the client. In order to do that you first need to understand what deliverables (both artifacts and activities) that produce value to the client. In some projects the client might get great value from the structuring and questioning on business processes done by the requirements analysts in order to define the system requirements. In other projects the client is perfectly clear on their business and the requirements specs are only needed to formally document the systems functionality and drive testing. Next you remove any activity or artifact that does not add value to the client or is invaluable in order to support efficient value production and make sure everyone inside and outside of the project understands their role and for whom they are supposed to produce value, in other words: if they are a Value Producer or a Supporter.Jörgen Anderssonhttp://www.blogger.com/profile/10229551989831053791noreply@blogger.com0tag:blogger.com,1999:blog-5418258901636981690.post-61479125186996198132011-06-29T06:09:00.000+02:002011-06-29T06:09:16.153+02:00Define and apply “The boy scout rule”<div style="margin-bottom: 0in; margin-left: 0px; margin-right: 0px; margin-top: 0px;">“<a href="http://programmer.97things.oreilly.com/wiki/index.php/The_Boy_Scout_Rule">The boy scout rule</a>”, as defined by Robert C. Martin, tells us to make small improvements to existing code as we pass by implementing new features. It doesn't mean we need to make the whole class/file perfect, it just means we should make some small improvement in addition to the new feature, which of course should have a clean implementation. This is a good principle to include in the culture of any project. But how to do make it happen in a team with many developers, each bringing their own experiences and standards? In my project we are currently making a serious effort. Here are the actions we are taking. Please leave a comment with your own experiences/ideas.</div><div style="margin-bottom: 0in; margin-left: 0px; margin-right: 0px; margin-top: 0px;"><br />
</div><div style="margin-bottom: 0in; margin-left: 0px; margin-right: 0px; margin-top: 0px;">First, talk about it. Introduce the name “The boy scout rule” - it is a quite funny name, and easy to remember. Load it further with the reference to real world scouts not caring who left the garbage found at the camping site, but always making sure the site is cleaned and left a bit better than found. Transfer the values into software development as a way to continuously refactor an existing code base while at the same time implementing new features.</div><div style="margin-bottom: 0in; margin-left: 0px; margin-right: 0px; margin-top: 0px;"><br />
</div><div style="margin-bottom: 0in; margin-left: 0px; margin-right: 0px; margin-top: 0px;">Second, define, in concrete terms, what “The boy scout rule” means in the project. In our project we have made the following definition, with reference to <a href="http://se-thinking.blogspot.com/2011/05/proposed-design-principles-for-internal.html">our design principles</a>:</div><ul><li><div style="margin-bottom: 0in; margin-left: 0px; margin-right: 0px; margin-top: 0px;">Increase readability</div><ul><li><div style="margin-bottom: 0in; margin-left: 0px; margin-right: 0px; margin-top: 0px;">Choose intention revealing names from the business domain for classes/methods/attributes/parameters/variables.</div></li>
<li><div style="margin-bottom: 0in; margin-left: 0px; margin-right: 0px; margin-top: 0px;">Extract methods to enforce the principle of Command Query Separation (CQS) and intention revealing method names.</div></li>
</ul></li>
<li><div style="margin-bottom: 0in; margin-left: 0px; margin-right: 0px; margin-top: 0px;">Enhance the structure</div><ul><li><div style="margin-bottom: 0in; margin-left: 0px; margin-right: 0px; margin-top: 0px;">Move methods to the“right” location to enforce the Single Responsibility Principle (SRP) for classes.</div></li>
</ul></li>
<li><div style="margin-bottom: 0in; margin-left: 0px; margin-right: 0px; margin-top: 0px;">Make it easier to do changes without introducing bugs:</div><ul><li><div style="margin-bottom: 0in; margin-left: 0px; margin-right: 0px; margin-top: 0px;">Increase readability (as above).</div></li>
<li><div style="margin-bottom: 0in; margin-left: 0px; margin-right: 0px; margin-top: 0px;">Enhance the structure (as above).</div></li>
<li><div style="margin-bottom: 0in; margin-left: 0px; margin-right: 0px; margin-top: 0px;">Use domain objects for method parameters (e.g. use a ZipCode object instead of a simple String).</div></li>
</ul></li>
</ul><div style="margin-bottom: 0in; margin-left: 0px; margin-right: 0px; margin-top: 0px;">Third, lead by example. Start with the the informal leaders (often the most experienced developers) and have them spread the practice through interaction and pairing in daily work. Personally, when pairing with colleagues I try to explicitly point out what part of the work we are doing is implementing the new feature and what part is applying “The boy scout rule”. By doing so I hope to spread both the knowledge and practice to apply the rule in daily work.</div>Jörgen Anderssonhttp://www.blogger.com/profile/10229551989831053791noreply@blogger.com0tag:blogger.com,1999:blog-5418258901636981690.post-37408995530532896942011-05-19T07:44:00.002+02:002011-05-19T07:54:23.490+02:00Proposed design principles for internal designIn my current project we are to implement additional functionality in an existing code base. The system is SOA-based, composed of many sub systems, each implementing a set of services, as detailed by the architectural guidelines given to the project. In practice this means that each sub system hosts a set of domain logic behind a façade of Java EE Stateless Session Beans or Message Driven Beans. As the service façade approach is described in detail in the guidelines, any guidelines of the internal design is equally absent. Therefore I’ve, with great inspiration from the <a href="http://domaindrivendesign.org/">DDD community</a>, put together this list of design principles for proposed use within the project, but I think they could be equally applied to internal design of most administrative IT-systems. What do you think? Should something be added or subtracted?<br />
<br />
<b>General design principles</b><br />
<br />
<ul><li>Adhere to the Single Responsibility Principle (<a href="http://en.wikipedia.org/wiki/Single_responsibility_principle">SRP</a>), i.e. every module (class/method) has one responsibility, not several.</li>
<li>Choose names from the business domain for classes, methods, attributes, parameters and variables.</li>
<li>Use domain objects as parameters instead of primitives like string or int.</li>
<li>Design the system such that domain classes can be unit tested individually.</li>
<li>Each (part of a) sub system should use one design metaphor, either </li>
<ul><li>domain centered - business logic placed in domain objects, preferable when changes to state is most important - or</li>
<li>service centered – business logic placed in stateless services that operate on data in <a href="http://en.wikipedia.org/wiki/Data_transfer_object">DTO</a>:s, preferable when the data flow is most important. </li>
</ul></ul><br />
<b>From this follows</b><br />
<br />
<ul><li>Keep business logic and technical implementation details separate, i.e. do not combine these two types of complexity.</li>
<li>Adhere to Command Query Separation (<a href="http://en.wikipedia.org/wiki/Command-query_separation">CQS</a>), i.e. each method should be either a command (having a side-effect and possibly returning a result) or a query (returning a result without any side-effect).</li>
</ul><br />
<b>On a more detailed level</b><br />
<br />
<ul><li>As a tool for communication use the following stereotypes for objects implementing the business domain logic:</li>
<ul><li>Value Objects – immutable, created when needed, not persisted.</li>
<li>Entities – mutable (as per allowed by business rules), loaded/persisted through Repositories.</li>
<li>Repository – interface describing load/persist of Entities.</li>
<li>Factory – interface describing creation of Entities and Value Objects when external dependencies (to Repositories, Factories or Services) must be fulfilled.</li>
<li>Service – stateless class housing business logic that operate on data in DTO:s</li>
</ul><li>Dependencies are preferably handled through dependency injection to facilitate loose coupling between objects.</li>
</ul><br />
<b>Application of principles</b><br />
These principles define a solid approach to design of new (sub) systems. When used in work with an existing code base it can be used for continuously applying small improvements in accordance with “<a href="http://programmer.97things.oreilly.com/wiki/index.php/The_Boy_Scout_Rule">the boy scout rule</a>” and as vision to compare with current state in order to form a road map for larger refactoring efforts.Jörgen Anderssonhttp://www.blogger.com/profile/10229551989831053791noreply@blogger.com0tag:blogger.com,1999:blog-5418258901636981690.post-6003793151692563382011-05-06T06:59:00.000+02:002011-05-06T06:59:52.717+02:00No man is an islandWhat’s the most common reason for failure in a software project, or perhaps in any project? You will get many different answers depending on who you ask. The designers might say “The requirements were not clear enough and the programmers didn't follow the architecture”. Testers on the other hand might say “The programmers never delivered what we were supposed to get when we were supposed to get it” and programmers tend to say “We got lots of ‘bug-reports’ that where really outside the scope”. The list can be long, but at the end of the day it all comes down to lack of communication.<br />
<br />
Let’s face it, developing software is a complex and creative handiwork. That’s why we try to stay away from the over-planned, defined, way of waterfall and turn to iteration based approaches like RUP or Scrum. <a href="http://se-thinking.blogspot.com/2011/01/true-iterations-deliver-value.html">True iterations</a> give us great feedback to base the next, (hopefully) better, version on. However, working with iterations is not enough. The saying “No man is an island” origins back to about year 1600 and is as true as ever when it comes to software projects. For many years we have set up the projects so that specialist testers form a test team, specialist designers form a design team, specialist programmers form implementation teams, and so on. This for sure brings advantages in having specialists learning from each other becoming even more specialised. But it also introduces a great deal of hand-overs between the teams, each missing some vital information. I believe this way of organising the development project is one of the key reasons why projects fail.<br />
<br />
In the agile community we talk about “<a href="http://en.wikipedia.org/wiki/Cross-functional_team">Cross-functional teams</a>“, meaning that people with different specializations work together to achieve a common goal. In software development that means that a programmer sits next to a designer and a tester, working together on transforming a requirement into a piece of well designed, well tested code ready for production. The agile practices are all about maximizing communication, e.g. by locating people in the same room with access to broadband communication aids such as whiteboards.<br />
<br />
If you are currently in a project where the test team and development team sit at different floors and the primary tool for communication is release notes, then you have a superb opportunity to increase the likeliness of project success just by re-locating so that developers and tester form collaborative teams working together on both design, test design, implementation and test execution.<br />
<div><br />
</div>Jörgen Anderssonhttp://www.blogger.com/profile/10229551989831053791noreply@blogger.com0tag:blogger.com,1999:blog-5418258901636981690.post-23598583806576877922011-04-16T21:26:00.001+02:002020-08-19T21:45:05.605+02:00Automated functional tests - an organisational change<!--[if gte mso 9]><xml> <w:WordDocument> <w:View>Normal</w:View> <w:Zoom>0</w:Zoom> <w:TrackMoves/> <w:TrackFormatting/> <w:HyphenationZone>21</w:HyphenationZone> <w:PunctuationKerning/> <w:ValidateAgainstSchemas/> <w:SaveIfXMLInvalid>false</w:SaveIfXMLInvalid> <w:IgnoreMixedContent>false</w:IgnoreMixedContent> <w:AlwaysShowPlaceholderText>false</w:AlwaysShowPlaceholderText> <w:DoNotPromoteQF/> <w:LidThemeOther>SV</w:LidThemeOther> <w:LidThemeAsian>X-NONE</w:LidThemeAsian> <w:LidThemeComplexScript>X-NONE</w:LidThemeComplexScript> <w:Compatibility> <w:BreakWrappedTables/> <w:SnapToGridInCell/> <w:WrapTextWithPunct/> <w:UseAsianBreakRules/> <w:DontGrowAutofit/> <w:SplitPgBreakAndParaMark/> <w:DontVertAlignCellWithSp/> <w:DontBreakConstrainedForcedTables/> <w:DontVertAlignInTxbx/> <w:Word11KerningPairs/> <w:CachedColBalance/> </w:Compatibility> <w:BrowserLevel>MicrosoftInternetExplorer4</w:BrowserLevel> <m:mathPr> <m:mathFont m:val="Cambria Math"/> <m:brkBin m:val="before"/> <m:brkBinSub m:val="--"/> <m:smallFrac m:val="off"/> <m:dispDef/> <m:lMargin m:val="0"/> <m:rMargin m:val="0"/> <m:defJc m:val="centerGroup"/> <m:wrapIndent m:val="1440"/> <m:intLim m:val="subSup"/> <m:naryLim m:val="undOvr"/> </m:mathPr></w:WordDocument> </xml><![endif]--><!--[if gte mso 9]><xml> <w:LatentStyles DefLockedState="false" DefUnhideWhenUsed="true"
DefSemiHidden="true" DefQFormat="false" DefPriority="99"
LatentStyleCount="267"> <w:LsdException Locked="false" Priority="0" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Normal"/> <w:LsdException Locked="false" Priority="9" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="heading 1"/> <w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 2"/> <w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 3"/> <w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 4"/> <w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 5"/> <w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 6"/> <w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 7"/> <w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 8"/> <w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 9"/> <w:LsdException Locked="false" Priority="39" Name="toc 1"/> <w:LsdException Locked="false" Priority="39" Name="toc 2"/> <w:LsdException Locked="false" Priority="39" Name="toc 3"/> <w:LsdException Locked="false" Priority="39" Name="toc 4"/> <w:LsdException Locked="false" Priority="39" Name="toc 5"/> <w:LsdException Locked="false" Priority="39" Name="toc 6"/> <w:LsdException Locked="false" Priority="39" Name="toc 7"/> <w:LsdException Locked="false" Priority="39" Name="toc 8"/> <w:LsdException Locked="false" Priority="39" Name="toc 9"/> <w:LsdException Locked="false" Priority="35" QFormat="true" Name="caption"/> <w:LsdException Locked="false" Priority="10" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Title"/> <w:LsdException Locked="false" Priority="1" Name="Default Paragraph Font"/> <w:LsdException Locked="false" Priority="11" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Subtitle"/> <w:LsdException Locked="false" Priority="22" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Strong"/> <w:LsdException Locked="false" Priority="20" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Emphasis"/> <w:LsdException Locked="false" Priority="59" SemiHidden="false"
UnhideWhenUsed="false" Name="Table Grid"/> <w:LsdException Locked="false" UnhideWhenUsed="false" Name="Placeholder Text"/> <w:LsdException Locked="false" Priority="1" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="No Spacing"/> <w:LsdException Locked="false" Priority="60" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Shading"/> <w:LsdException Locked="false" Priority="61" SemiHidden="false"
UnhideWhenUsed="false" Name="Light List"/> <w:LsdException Locked="false" Priority="62" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Grid"/> <w:LsdException Locked="false" Priority="63" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 1"/> <w:LsdException Locked="false" Priority="64" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 2"/> <w:LsdException Locked="false" Priority="65" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 1"/> <w:LsdException Locked="false" Priority="66" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 2"/> <w:LsdException Locked="false" Priority="67" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 1"/> <w:LsdException Locked="false" Priority="68" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 2"/> <w:LsdException Locked="false" Priority="69" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 3"/> <w:LsdException Locked="false" Priority="70" SemiHidden="false"
UnhideWhenUsed="false" Name="Dark List"/> <w:LsdException Locked="false" Priority="71" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Shading"/> <w:LsdException Locked="false" Priority="72" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful List"/> <w:LsdException Locked="false" Priority="73" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Grid"/> <w:LsdException Locked="false" Priority="60" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Shading Accent 1"/> <w:LsdException Locked="false" Priority="61" SemiHidden="false"
UnhideWhenUsed="false" Name="Light List Accent 1"/> <w:LsdException Locked="false" Priority="62" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Grid Accent 1"/> <w:LsdException Locked="false" Priority="63" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 1 Accent 1"/> <w:LsdException Locked="false" Priority="64" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 2 Accent 1"/> <w:LsdException Locked="false" Priority="65" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 1 Accent 1"/> <w:LsdException Locked="false" UnhideWhenUsed="false" Name="Revision"/> <w:LsdException Locked="false" Priority="34" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="List Paragraph"/> <w:LsdException Locked="false" Priority="29" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Quote"/> <w:LsdException Locked="false" Priority="30" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Intense Quote"/> <w:LsdException Locked="false" Priority="66" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 2 Accent 1"/> <w:LsdException Locked="false" Priority="67" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 1 Accent 1"/> <w:LsdException Locked="false" Priority="68" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 2 Accent 1"/> <w:LsdException Locked="false" Priority="69" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 3 Accent 1"/> <w:LsdException Locked="false" Priority="70" SemiHidden="false"
UnhideWhenUsed="false" Name="Dark List Accent 1"/> <w:LsdException Locked="false" Priority="71" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Shading Accent 1"/> <w:LsdException Locked="false" Priority="72" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful List Accent 1"/> <w:LsdException Locked="false" Priority="73" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Grid Accent 1"/> <w:LsdException Locked="false" Priority="60" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Shading Accent 2"/> <w:LsdException Locked="false" Priority="61" SemiHidden="false"
UnhideWhenUsed="false" Name="Light List Accent 2"/> <w:LsdException Locked="false" Priority="62" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Grid Accent 2"/> <w:LsdException Locked="false" Priority="63" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 1 Accent 2"/> <w:LsdException Locked="false" Priority="64" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 2 Accent 2"/> <w:LsdException Locked="false" Priority="65" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 1 Accent 2"/> <w:LsdException Locked="false" Priority="66" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 2 Accent 2"/> <w:LsdException Locked="false" Priority="67" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 1 Accent 2"/> <w:LsdException Locked="false" Priority="68" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 2 Accent 2"/> <w:LsdException Locked="false" Priority="69" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 3 Accent 2"/> <w:LsdException Locked="false" Priority="70" SemiHidden="false"
UnhideWhenUsed="false" Name="Dark List Accent 2"/> <w:LsdException Locked="false" Priority="71" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Shading Accent 2"/> <w:LsdException Locked="false" Priority="72" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful List Accent 2"/> <w:LsdException Locked="false" Priority="73" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Grid Accent 2"/> <w:LsdException Locked="false" Priority="60" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Shading Accent 3"/> <w:LsdException Locked="false" Priority="61" SemiHidden="false"
UnhideWhenUsed="false" Name="Light List Accent 3"/> <w:LsdException Locked="false" Priority="62" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Grid Accent 3"/> <w:LsdException Locked="false" Priority="63" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 1 Accent 3"/> <w:LsdException Locked="false" Priority="64" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 2 Accent 3"/> <w:LsdException Locked="false" Priority="65" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 1 Accent 3"/> <w:LsdException Locked="false" Priority="66" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 2 Accent 3"/> <w:LsdException Locked="false" Priority="67" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 1 Accent 3"/> <w:LsdException Locked="false" Priority="68" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 2 Accent 3"/> <w:LsdException Locked="false" Priority="69" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 3 Accent 3"/> <w:LsdException Locked="false" Priority="70" SemiHidden="false"
UnhideWhenUsed="false" Name="Dark List Accent 3"/> <w:LsdException Locked="false" Priority="71" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Shading Accent 3"/> <w:LsdException Locked="false" Priority="72" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful List Accent 3"/> <w:LsdException Locked="false" Priority="73" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Grid Accent 3"/> <w:LsdException Locked="false" Priority="60" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Shading Accent 4"/> <w:LsdException Locked="false" Priority="61" SemiHidden="false"
UnhideWhenUsed="false" Name="Light List Accent 4"/> <w:LsdException Locked="false" Priority="62" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Grid Accent 4"/> <w:LsdException Locked="false" Priority="63" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 1 Accent 4"/> <w:LsdException Locked="false" Priority="64" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 2 Accent 4"/> <w:LsdException Locked="false" Priority="65" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 1 Accent 4"/> <w:LsdException Locked="false" Priority="66" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 2 Accent 4"/> <w:LsdException Locked="false" Priority="67" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 1 Accent 4"/> <w:LsdException Locked="false" Priority="68" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 2 Accent 4"/> <w:LsdException Locked="false" Priority="69" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 3 Accent 4"/> <w:LsdException Locked="false" Priority="70" SemiHidden="false"
UnhideWhenUsed="false" Name="Dark List Accent 4"/> <w:LsdException Locked="false" Priority="71" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Shading Accent 4"/> <w:LsdException Locked="false" Priority="72" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful List Accent 4"/> <w:LsdException Locked="false" Priority="73" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Grid Accent 4"/> <w:LsdException Locked="false" Priority="60" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Shading Accent 5"/> <w:LsdException Locked="false" Priority="61" SemiHidden="false"
UnhideWhenUsed="false" Name="Light List Accent 5"/> <w:LsdException Locked="false" Priority="62" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Grid Accent 5"/> <w:LsdException Locked="false" Priority="63" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 1 Accent 5"/> <w:LsdException Locked="false" Priority="64" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 2 Accent 5"/> <w:LsdException Locked="false" Priority="65" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 1 Accent 5"/> <w:LsdException Locked="false" Priority="66" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 2 Accent 5"/> <w:LsdException Locked="false" Priority="67" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 1 Accent 5"/> <w:LsdException Locked="false" Priority="68" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 2 Accent 5"/> <w:LsdException Locked="false" Priority="69" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 3 Accent 5"/> <w:LsdException Locked="false" Priority="70" SemiHidden="false"
UnhideWhenUsed="false" Name="Dark List Accent 5"/> <w:LsdException Locked="false" Priority="71" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Shading Accent 5"/> <w:LsdException Locked="false" Priority="72" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful List Accent 5"/> <w:LsdException Locked="false" Priority="73" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Grid Accent 5"/> <w:LsdException Locked="false" Priority="60" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Shading Accent 6"/> <w:LsdException Locked="false" Priority="61" SemiHidden="false"
UnhideWhenUsed="false" Name="Light List Accent 6"/> <w:LsdException Locked="false" Priority="62" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Grid Accent 6"/> <w:LsdException Locked="false" Priority="63" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 1 Accent 6"/> <w:LsdException Locked="false" Priority="64" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 2 Accent 6"/> <w:LsdException Locked="false" Priority="65" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 1 Accent 6"/> <w:LsdException Locked="false" Priority="66" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 2 Accent 6"/> <w:LsdException Locked="false" Priority="67" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 1 Accent 6"/> <w:LsdException Locked="false" Priority="68" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 2 Accent 6"/> <w:LsdException Locked="false" Priority="69" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 3 Accent 6"/> <w:LsdException Locked="false" Priority="70" SemiHidden="false"
UnhideWhenUsed="false" Name="Dark List Accent 6"/> <w:LsdException Locked="false" Priority="71" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Shading Accent 6"/> <w:LsdException Locked="false" Priority="72" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful List Accent 6"/> <w:LsdException Locked="false" Priority="73" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Grid Accent 6"/> <w:LsdException Locked="false" Priority="19" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Subtle Emphasis"/> <w:LsdException Locked="false" Priority="21" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Intense Emphasis"/> <w:LsdException Locked="false" Priority="31" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Subtle Reference"/> <w:LsdException Locked="false" Priority="32" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Intense Reference"/> <w:LsdException Locked="false" Priority="33" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Book Title"/> <w:LsdException Locked="false" Priority="37" Name="Bibliography"/> <w:LsdException Locked="false" Priority="39" QFormat="true" Name="TOC Heading"/> </w:LatentStyles> </xml><![endif]--><!--[if gte mso 10]> <style>
/* Style Definitions */
table.MsoNormalTable
{mso-style-name:"Table Normal";
mso-tstyle-rowband-size:0;
mso-tstyle-colband-size:0;
mso-style-noshow:yes;
mso-style-priority:99;
mso-style-qformat:yes;
mso-style-parent:"";
mso-padding-alt:0cm 5.4pt 0cm 5.4pt;
mso-para-margin-top:0cm;
mso-para-margin-right:0cm;
mso-para-margin-bottom:10.0pt;
mso-para-margin-left:0cm;
line-height:115%;
mso-pagination:widow-orphan;
font-size:11.0pt;
font-family:"Calibri","sans-serif";
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:"Times New Roman";
mso-fareast-theme-font:minor-fareast;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;}
</style> <![endif]--> <br />
<div class="MsoNormal"><span lang="">Introducing automated systems (functional) testing to a project used to run manual functional and regression tests at the end of every development cycle is not a technical issue – well, it is to some extent, but there are some good solutions out there</span> – it is a fundamental change to the work your testers perform.</div><div class="MsoNormal"><br />
</div><div class="MsoNormal"><span lang="">In the manual approach a typical set up of roles dictate that test designers create written test specifications in some tool or in ordinary documents. The specifications contain everything the tester needs to execute the test, including detailed step-by-step instructions. <span> </span>The tests are then manually executed over and over again making sure to catch any regression in the system functionality.</span></div><div class="MsoNormal"><br />
</div><div class="MsoNormal"><span lang="">When moving to automated testing the previously text based test specifications are transformed into an executable format. Perhaps the <a href="http://behaviour-driven.org/">behaviour driven</a> approach is used with some short textual description that underneath is connected to code exercising the system, preferably through the GUI if such exists. Since execution of the tests is automated the testers previously doing the manual test execution are no longer needed. Instead test implementers, with the ability to build the code that executes the tests, are working alongside the test designers. Tests are executed by the build system, perhaps nightly or even more frequent making the feedback loop faster.</span></div><div class="MsoNormal"><span lang=""><br />
</span></div><div class="MsoNormal"><span lang=""></span></div><div class="MsoNormal"><span lang=""></span></div><div class="MsoNormal"><span lang=""><span></span>This is a big change - and yes, it is a good one - but it is important to understand the impact to the organization of your test/project team and the new skills required. It is also important to understand that the automated tests are now a fundamental artifact for your system, just like the written test specifications used to be, and it is to be owned by the test lead. Just because it is automated and uses code to run, it is not to be handed over to the developers.</span></div>Jörgen Anderssonhttp://www.blogger.com/profile/10229551989831053791noreply@blogger.com0tag:blogger.com,1999:blog-5418258901636981690.post-10155348984846285592011-04-01T07:08:00.000+02:002011-04-01T07:08:00.002+02:00The main pillars of Scrum according to me<div class="MsoNormal"><span>In this post I'll share with you what I think are the most valuable pieces of Scrum, pieces I'm never willing to compromise over.</span></div><div class="MsoNormal"><span><br />
</span></div><div class="MsoNormal"><span><strong>Iterative/incremental</strong></span></div><div class="MsoNormal"><span>Scrum is truly iterative mandating to produce a fully functional increment of the final product, including at least some business functionality, in each sprint. In addition each sprint ends with a sprint review and retrospective making the feedback loop explicit.</span></div><div class="MsoNormal"><br />
</div><div class="MsoNormal"><span><strong>Transparency</strong> </span></div><div class="MsoNormal"><span>In Scrum everything is visible. Any misconception in the organisation, design or infrastructure is pulled into the light. Honesty is just the first name!</span></div><div class="MsoNormal"><br />
</div><div class="MsoNormal"><b><span>Continuous improvement</span></b></div><div class="MsoNormal"><span> Fast feedback loops, in larger and larger circles, and transparency are a solid ground for continuous improvement both to the product under development and to the process itself. Always strive to do things faster and with higher quality than the last time around.</span></div><div class="MsoNormal"><br />
</div><div class="MsoNormal"><span><strong>Self-managing teams</strong> </span></div><div class="MsoNormal"><span>Scrum is based on the belief that a team of professionals is most effective if left alone to organise, solve its problems and work towards the sprint goal instead of having management telling them what to do and how to do it. Managers are to take the role of facilitators/sponsors making sure the team has everything it needs to produce as much value as possible.</span></div><div class="MsoNormal"><br />
</div><span><strong>Definition of done</strong> </span><br />
<span>When is a task done? In order to truly complete a piece of work and deliver a high quality increment it is absolutely vital to have a solid definition of done. Any work that should have been done to get the increment ready for production will not go away if left undone. Instead it will accumulate over the following iterations and add up exponentially to a large, undefined, piece of work requiring a stabilisation phase of unknown length to meet quality criteria.</span><br />
<br />
<span>These are my fundamental values which I try to stick to. What do you think? Do you have other/additional most pressures parts? </span>Jörgen Anderssonhttp://www.blogger.com/profile/10229551989831053791noreply@blogger.com0tag:blogger.com,1999:blog-5418258901636981690.post-3608980178551760302011-03-15T13:51:00.000+01:002011-03-15T13:51:42.082+01:00The only certification I'm proud ofIt seams the popularity and importance of certifications goes up and down in the IT-industry. The only thing constant is the ongoing debate on whether certifications are valuable as a "receipt" on competence or a total scam. I think there is a value but the value vary greatly depending on the type of certification.<br />
<br />
After a bit over ten years in the industry I've gathered a set of certifications. However, the value of them are in most cases very little over following a coarse or reading a book. I'm certified in both OOAD by IBM/Rational and in Java programming by Sun/Oracle. I have also ample experience in both fields but I got the certifications prior to that, by just reading the books and completing a test. For my Scrum Master certification I didn't even have to complete a test, following a two day course was enough. This is the reason for this particular certification being so heavily criticised, and I agree, the "certification" does not bring any additional value. But the two day course was great for gaining fundamental knowledge on values and core practices and I have since practised implementing Scrum and other agile techniques and value systems successfully.<br />
<br />
These types of certifications are, despite that it might involve hard work to complete them since they tend to test on so specific things, entry level in most cases and as such adding no more value than successfully completing an exam at a small course at university. But there are other types of certifications that are based on real world experience and proven capabilities evaluated by experienced software engineers. Within the <a href="http://www.capgemini.com/">Capgemini</a> group we've had such certifications for different professions for a long time and more recently it has been aligned with the publicly known <a href="http://www.opengroup.org/itsc/">Open Group ITSC</a>. This is a certification based on what I've done in real world assignments and I think it is also a valid quality guarantee for the knowledge and experience I can bring into a new project. This is the only certification I'm proud of!Jörgen Anderssonhttp://www.blogger.com/profile/10229551989831053791noreply@blogger.com0tag:blogger.com,1999:blog-5418258901636981690.post-85950184121207859592011-03-09T21:02:00.001+01:002011-03-10T06:10:42.022+01:00Borders make communication break down. Don’t let them pile up!Communication involves transferring information from the sender to the receiver. In human communication the sender and receiver are each in separate personal contexts based on their personal history, interests and the reason for which they engage in the act of communication. The data being transferred (i.e. the words spoken or written, pictures drawn, the body language being used, etc) are formed based on the information being processed and filtered through the sender's context and then re-interpreted to information through the context of the receiver. Successful communication relies on the information not being to much obscured through this process.<br />
<br />
This said, all human communication occurs between contexts, the recipe for successful communication is that the differences between the contexts are as small as possible. So what is it that makes these context differ? What is it that makes communication hard? I like to use the term "borders". In professional communication, e.g within a project or within a department or a company, many different types of borders can be identified. If you were to communicate without crossing any of those boarders the communication wouldn't be very interesting or valuable, so that is not a success factor. But when laying out your organisation you should take care not to place different borders at the same place since borders tend to reinforce each other and pile up to something higher than the sum of its parts, i.e. making important communication unnecessary hard.<br />
<br />
Let's have a look at some commonly found borders in professional life to make the idea clear.<br />
<br />
<b>Borders between professions</b><br />
It is easy for a developer to talk to a fellow developer and for a tester to talk to another tester since they share the same professional context, same professional interests, similar experiences, etc. It is much harder for a developer to talk to a tester or a requirements analyst since that communication is crossing the border of professions.<br />
<br />
<b>Borders between area of responsibility</b><br />
Area of responsibility could be the project you are currently working in or the sub-system (within a larger project) that you are assigned to, but it could also be the requirements for a project as opposed to the code base of the same project. People with different areas of responsibilities tends to have different focuses and priorities and hence communicates out of different contexts.<br />
<br />
<b>Borders between geographic locations</b><br />
Many software projects of today are distributed over several different locations. It might be different countries, different cities, different buildings or as little as different floors or even different rooms. Never the less we, being humans, quickly develops a common context with those people sitting in the same room as opposed to those at another location, hence communication over geographical boarders is harder. Having to consider not being able to speak face to face with people at other locations but being bound to telephones, e-mail, IM, etc, doesn't get communication any better<br />
.<br />
<b>Borders between chains of command</b><br />
People separated into teams with different managers also tend to have a harder time communicating than those being assigned to the same team. In some cases management policies might not allow any communication except through the chain of command and that is obviously a problem. But even when direct communication is allowed it suffices from the different context resulting from different priorities in the two teams.<br />
<br />
<b>Borders between cultures</b><br />
The clash of cultures is pretty common in more or less any software project these days. It might be due to organisational mergers, consultants being brought in to work alongside employees or part of the project being outsourced to another company or another country. The more cultural differences the higher the communication border.<br />
<br />
So, what to do with all this? It is something to think carefully about when setting up your project organisation. E.g. consider a pretty common organisational pattern: You start with placing, analysts, architects, developers, testers and CM:s each in their separate team with a separate manager only reporting to you, being the top line project manager. Then you decide that development should be outsourced to another company placed in another country (typically with lower cost per developer) and you provide them with no other means of communication than e-mail, IM and phones. Said and done, after some time you start to wonder why there is so much frustration within the other teams, everyone saying it is impossible to work with the development team. Yeah, not so hard to understand, right? You have managed to put the development team in a context where all borders, profession, area of responsibility, location, chain of command and culture, are placed at the exact some place, i.e. right between the development team and the rest of the project. Given this you should be happy if any communication at all is taking place.<br />
<br />
Of course, it isn't possible to have the whole project in the exact same context, since it wouldn't be very practical to have only developers or only testers making up the whole project. So, some cross-boarder communication is necessary and that is just find. Usually it works well if you take care to set up your organisation so that borders do not coincide. E.g., this is one of the reasons why cross-functional teams are good. The team members have to cross the professional boarders and perhaps also cultural once to communicate but they are all within the same area of responsibility and (hopefully) in the same location. If you need to have an off-site development team (perhaps with another cultural background) it is a very good idea to ease the communication by having at least one developer on-site handling the communication with the development team over the geographical boarder.<br />
<br />
The bottom line advice: Watch out for context borders and don't let them pile up to destroy communication within your project!Jörgen Anderssonhttp://www.blogger.com/profile/10229551989831053791noreply@blogger.com0tag:blogger.com,1999:blog-5418258901636981690.post-12923189446958550292011-02-25T13:03:00.002+01:002020-08-19T21:42:22.548+02:00Iterations based on feedback deepens your understandingA few weeks ago I came across a challenge in the form of a competition from one of the JFokus exhibitors. It seemed fun and small enough for a spare-time project so I set out to solve it.<br />
<br />
For those of you not reading Swedish I'll give you a translation of the problem statement:<br />
<div style="font-family: inherit;">"What should be done is to reduce a string as much as possible. You do this by repeatedly removing sub-strings. Not any sub-string, but only from a given set.</div><div style="font-family: inherit;"></div><div style="font-family: inherit;">The string to reduce is:</div><pre style="font-family: inherit;"><span style="font-size: small;">VOCDIITEIOCRUDOIANTOCSLOIOCVESTAIOCVOLIOCENTSU</span></pre><pre style="font-family: inherit;"></pre><pre style="font-family: inherit;">To reduce it you are allowed to remove occurrences of the following words:</pre><pre style="font-family: inherit;">TDD, DDD, DI, DO, OO, UI, ANT, CV, IOC, LOC, SU, VO</pre><div style="font-family: inherit;"><br />
<span style="font-size: small;">E.g. some repeated removals could be:</span></div><pre style="font-family: inherit;"><span style="font-size: small;">1) ...BIDDDOCDDD... (remove DDD)
2) ...BIDDDOC... (remove DDD again)
3) ...BIOC... (remove IOC)
4) ...B... and so on... </span></pre><div style="font-family: inherit;"><br />
</div><div style="font-family: inherit;">What does the shortest string you can produce in this way look like?<br />
<br />
To be a correct solution the program should be able to produce the shortest string given these in data and also for other start and reduction strings. In other words, it should be a general solution for this type of problem."</div><br />
<br />
In an initial analysis session I managed to solve the given string by hand and thereby deduced an algorithm in three steps:<br />
<ul><li>First split the start string based on "unreducible" chars, i.e. chars in the start string not part of any of the reduction strings.</li>
<li>Reduce each of the reducible sub-strings by recursive removal of reduction strings.</li>
<li>Concatenate the results into the shortest possible string.</li>
</ul>Due to a limited (and unknown) amount of time to spend on the problem I decided to split the work into two iterations each resulting in a delivered solution to maximize my chances of being able to post at least some solution to the competition.<br />
<br />
<b>Iteration One</b><br />
In the first iteration I decided to work on the second step of my deduced algorithm, i.e reduce the string by recursive removal of reduction strings, since it could stand as a solution on its own. Yes, this is definitely a brute-force, not so elegant, solution but never the less it solves the initially stated problem. I made the recursion happen inside a loop over all reduction strings to explore the shortest rest of applying the reduction strings in different order. From a performance point-of-view this pretty soon get out of control when the length of the start string and number of reduction strings grow, but for a iteration-one-solution I think it was OK.<br />
Before packaging up my application I worked it over to ensure easily readable and neatly factored code guided by unit tests.<br />
<br />
<b> Iteration Two</b><br />
Soon I got feedback that my solution did not solve all invariants of in-data used to evaluate submissions, i.e. it wasn't a good enough general solver. I supposed this was due to the severe performance problems kicking in for a bit longer start strings and higher number of reduction strings.<br />
I quickly wrote some additional end-to-end unit tests with tougher in-data and got to work on adding implementation for the first and third steps of my initial algorithm. The well tested and factored code made it easy to add the new functionality. In addition I added caching to ensure that no sub-string is ever evaluated more than once. Running my end-to-end tests through the profiler tool provided evidence for a strong performance boost; the brute-force solution of iteration one did almost 140 000 recursions over the initially stated in-data to find the shortest string, now I was down to 92 recursions!<br />
Architecturally-wise my solution wasn't fancy or innovative, I rather stayed with the OO-best-practices I normally use. The solution was implemented in Java with a simple interface describing the API for the logic.<br />
<br />
<div style="font-family: "courier new", courier, monospace;"><span style="font-size: x-small;"><span style="font-size: small;"><span style="background-color: white;">public interface StringReducer {<br />
String reduce(String startString);<br />
}</span></span></span></div><br />
I now had three implementations of this interface (<span style="font-size: small;"><span face="" style="font-family: "courier new", courier, monospace;">UnReducibleCharBasedStringReducer</span></span>, <span style="font-size: small;"><span face="" style="font-family: "courier new", courier, monospace;">RecursionBasedStringReducer</span></span> and <span style="font-size: small;"><span face="" style="font-family: "courier new", courier, monospace;">CachingStringReducer</span><span face="" style="background-color: white; font-family: "courier new", courier, monospace;"></span></span>) that could be linked together to implement the complete algorithm.<br />
<br />
<b>Iteration Three</b><br />
Again I got feedback that my solution did not solve all invariants of in-data used to evaluate submissions. At first I was a bit surprised since I really thought the solution was complete, but the feedback made me think harder (and open up another iteration) and soon I realized that my algorithm was lacking an important piece: For the general case, there is no guarantee that there will be a single string that is the shortest. It might very well be that the shortest result includes two or more strings with equal length. Consider for example the start string "ABC" and reduction strings "AB" and "BC". Reduction using "AB" gives the rest "C" while reduction using "BC" gives the rest "A", i.e. two different strings with equal length depending on the order in which the reduction strings are applied. So far my solution had randomly returned the latter of these results, omitting the first. Now I changed the <span style="font-size: small;"><span face="" style="font-family: "courier new", courier, monospace;">StringReducer</span> </span>interface to include the notion of a <span style="font-size: x-small;"><span face="" style="font-family: "courier new", courier, monospace;">Rest</span></span> wrapping one or more strings with equal length. This required a pretty significant re-write of the implementations but the tests already in place guided the work and secured a high quality result.<br />
<br />
In the process of re-write I also found another situation that could yield multiple results. Consider the start string "IOIOI" and the reduction string "IOI". This set-up could return two alternative results, "IO" and "OI", depending on which of the two occurrences of the reduction string that is removed first. Adding this to the <span style="font-size: small;"><span face="" style="font-family: "courier new", courier, monospace;">RecursionBasedStringReducer</span> </span>was as easy as adding a new test and adding another loop in the recursive function.<br />
With these modifications I submitted my solution for the third time, a couple of days before the dead-line of the competition. <br />
<br />
<b>Iteration Four</b><br />
This time the feedback said that my submission was accepted and I waited for the jury to make their decision on the perceived best solution. I didn't win - but I followed the heated discussions taking off right after the jury had announced their decision. In this discussions several suggestions for hard-to-handle in-data were put forward. For each I added new tests to my solution and found that it could handle most of them. The exception was a rather long start string (not possible to split) with associated rather many reduction strings that could be used to reduce the entire start string. I'm pretty confident my solution eventually would have come to the right conclusion but the large number of reduction strings made the number of needed recursions huge. But of course the solution is to pay attention to the first reduction path returning an empty string. After that there is no need to continue searching since no string could possibly be shorter. Again adding a test and injecting a simple condition in the right place<span style="font-size: small;"><span face="" style="font-family: "courier new", courier, monospace;">RecursionBasedStringReducer</span> </span> was enough to solve the problem.<br />
In doing so I also found a case of "false reduction strings", i.e. reduction strings containing characters not found in the start string. These strings will never be able to reduce the start string and hence can be removed right away instead of causing extra reduction paths to be evaluated. This finding caused another small update to the implementation and that's the current state of my StringReducer.<br />
<br />
<b>Conclusion</b><br />
My take-away from solving this challenge is this real life example showing that working iteratively based on feedback helps you to arrive at a deeper understanding of the problem domain and a step-wise better solution, something we should always strive for in any project regardless of scope and domain.<br />
In addition it was great fun!Jörgen Anderssonhttp://www.blogger.com/profile/10229551989831053791noreply@blogger.com0tag:blogger.com,1999:blog-5418258901636981690.post-31399131638653205182011-02-15T07:08:00.000+01:002011-02-15T07:08:10.557+01:00We are all Gardeners and AuthorsIn the <a href="http://se-thinking.blogspot.com/2011/01/software-engineering-essential-reading.html">previous book</a> on Software Engineering that I read the work of designing and developing software systems was characterised as gardening. The authors talked about the ever ongoing effort of sustaining and improving the internal quality of the code in terms of nurturing, guiding and pruning. I kind of liked that picture of the system under development as something fragile and tender that we should lovingly care about to make sure it will grow into something really valuable for the business we set out to serve.<br />
<br />
In terms of software development this work of a gardener's can be translated into finding the proper input of both domain knowledge and reusable pieces of existing code (nurturing), applying a sound architecture to shape the system form (guiding) and constantly refactor to improve existing code and get rid of what is not serving its purpose any more (pruning). At our disposal we have tools like Domain Driven Design (DDD), Test Driven Development (TDD), different levels of software patterns and a powerful IDE. Growing a system is also a great metaphor in these times of agility. Doing the agile development right we will continuously add small pieces of new functionality to a working system always being careful not to wreck the tender "plant" we are growing. In other words - we should all be Gardeners.<br />
<br />
We should also be Authors. In the current book I'm reading, <a href="http://www.amazon.com/Clean-Code-Handbook-Software-Craftsmanship/dp/0132350882">"Clean Code" by Robert C Martin</a>, "Uncle Bob" argues that good source code should be just like prose. It should be easy and enjoyable to read as well as informative and clear about its intentions. After all the code we write will be read by ourselves and others many, many, many times in the future. Writing code that can be correctly and efficiently executed by a computer is not that hard. In most cases the compiler and run-time optimizer will take care and by adding some tests to the mix we should not get that many surprises. The really hard part is to write code that can correctly and efficiently be read, understood and changed by humans. So, next time you sit down to do some programming, try to think of yourself not as someone talking to the computer but as an Author talking to your fellow and future developers with the source code as your means of communication.Jörgen Anderssonhttp://www.blogger.com/profile/10229551989831053791noreply@blogger.com0tag:blogger.com,1999:blog-5418258901636981690.post-52421577293784269592011-01-25T10:34:00.001+01:002011-01-25T10:40:19.056+01:00True iterations deliver valueEveryone wants to be iterative and incremental these days. Iterations are indeed nothing new; they have been around inside and outside of RUP and other methodologies for a long time and they show up again in SCRUM (called Sprints) and XP. Many projects claims to be iterative, but are they really?<br />
<br />
In my view you are truly iterative only if you for each iteration go through the whole cycle of analysis, design, implementation, integration, test and deploy to really produce a high quality increment of the final system. Many projects I’ve seen claims to be iterative but tends to focus only on analysis and design in the first iteration, hence implicitly converting the iterations to phases in a good old fashioned waterfall.<br />
<br />
The only way you get to benefit the true value of iterative development is to run through the whole cycle and collect feedback from everyone involved. Then you know how to do even better in your next iteration. As a Software Engineer we should really push for true iterations in our projects since that is the only way we are sure to be involved and able to give and receive early feedback.Jörgen Anderssonhttp://www.blogger.com/profile/10229551989831053791noreply@blogger.com0