As the Data Guy on the team, much less of my historical experience has had formal Test Driven Design (TDD) principals involved. I have a clear understanding of the concept; however putting in practice has always yielded inconsistent results. I have come to realize that inconsistency can be directly driven back to the amount of effort given to developing the test cases (duh!). It’s not that I don’t consider what needs to be tested, but that I have historically considered it from my perspective rather than the point of view of the person who’ll be using the output of my efforts. In iteration 4 I have had 3 situations where I identified a “gap” in my logic before releasing the code to another developer. This meant one less iteration (not Iteration :) ) of “I am not getting the data I want”, or read differently – three less defects. Certainly I still have room to improve here and I also have the benefit of having my “users” get back to me in pretty quick fashion if there’s a problem. However the lesson is important and relevant to all of us – make sure your “head” is in the right place when you’re working through test cases and they’ll be worth their weight in gold to you.
Posted by MadMan | Posted in Coding Agile, Tools and Techniques | Posted on 12-12-2010
One of the greatest benefits of being agile is not being chained to an office chair in a cubicle for 60 hours a week. These days one can be productive anywhere. One example of this is that most of the best work I do is away from the computer. Many times I have been mountain biking or riding my motorcycle and an elegant solution to a problem I had been stumped on to the point of realizing I needed to get away from it will bubble up to my consciousness. I then return to the keyboard to type in the solution. In my process of getting from an idea to a loose design to more detailed user stories and tasks that evolve into the concrete reality of a release, the hardest part for me is the act of actually typing the code into the IDE. For me it is a massive bottleneck, and will probably continue to be that until we develop “Matrix” style interfaces where one can code at the speed of thought. To that end I make it a point to distract myself from the bottleneck by each week finding a new, interesting place to code. My top 10 places to enter code so far (in no particular order) are:
1. The Laundromat – kill two birds with one stone. In one 4 hour coding session I was able to get 6 loads of laundry done!
2. A bar – one of the best places I have found to code is sitting at a bar. With an unlimited supply of carbohydrates to keep the blood sugar up you can’t go wrong. Also it helps that I am rarely distracted by the constant woos of the fairer sex anymore.
3. The airplane – obvious. What else are you going to do.
4. Sitting by a creek – depending on battery life this could be good or a waste of time.
5. Visiting the in laws – turn what would be a dreary time into a productive time.
6. Driving – this takes a lot of practise, but once you get it down it is a real productivity booster. Turn 90 minutes of bumper to bumper commuting time into a productivity feast.
7. Playing golf – have someone else drive the cart. There is no better feeling than only getting a double bogey and then writing a constructor.
8. A hot air balloon – witnessing Archimedes’ principle in action is an excellent reminder to keep it simple stupid. If you are overthinking things, it’s probably more complicated than it needs to be. Also, if the balloon is sinking, heat the air up some more.
9. At a baseball game – with 15 minutes of actual playing time in a 3+ hour game, the ballgame is a hidden goldmine of productivity.
10. At the beach – enter code, surf, surf, tan, surf, surf, go home.
Next week I will discuss the 10 places you never ever ever want to enter code. Unless you’re a madman. Until then I would like to hear of some of the best places you have found!
Posted by Roscoe | Posted in Agile Thoughts, Coding Agile, SDLC Approaches | Posted on 05-12-2010
Continuous Quality and Testing
You’ve heard of continuous integration, continuous delivery and maybe even continuous spectrum. But I believe Agile puts a big emphasis on continuous quality and testing. The problem is, it’s one of the easiest items to forget, throw out, or justify why not to, when you are stuffed in a time box.
What makes up continuous quality and testing?
· Understanding the importance of up front design, and doing everything in your power to remove the “illusion of agreement” that always seems to occur when the requirements are passed down from customer, to product managers and finally ending at the developer. The more Sketchflow the better.
· Try to think in terms of solving problems. We are not just adding features, writing stories to tasks, and satisfying them. We are solving some problem for a customer, or user. What problem are you solving? If we continuously verify that we are in fact solving the problem(s), we are creating quality and value.
· Test each step of the way. Unit testing, integration testing, and manual verification are very important. Every story and task written should have associated time set aside for testing. Add this in to your time estimates.
· Final acceptance testing by the product managers, QA and scrum masta. It’s crucial to verify that all problems we set out to solve have been solved properly, and when we put our entire suite of software together, everything plays well together.
I remember reading this somewhere, and it’s been embedded in my head ever since.
· Defects/Bugs cost the least when caught in development
· Defects/Bugs cost more when caught in quality testing and acceptance testing
· Defects/Bugs cost the most when in production
The last item in the list really got me thinking. In my career, I can look back on hundreds of scenarios where defects have ended up costing a company so much more while in production than anywhere else. This is one reason TDD/BDD approaches are considered so important in the SDLC.
I shared this with our team already, but I’ll share the article again here. It’s a great case study that shows the importance of test driven development and the direct relationship it has to releasing software with a lot less defects from the start. And thinking about the statements above, it should be every employees desire to find defects as early as possible.
“Case studies were conducted with three development teams at Microsoft and one at IBM that have adopted TDD. The results of the case studies indicate that the pre-release defect density of the four products decreased between 40% and 90% relative to similar projects that did not use the TDD practice. Subjectively, the teams experienced a 15–35% increase in initial development time after adopting TDD.”
I was blown away reading this…the pre-release defect density decreased between 40% and 90% relative to teams that did not use TDD practice. I can only imagine, as their products grow the velocity of the TDD team eventually surpassed the non TDD team.
As Murdock stated in another post here:
“Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.”
I believe the only way to pull this off is with continuous quality testing.
Posted by LoonDog | Posted in Agile Thoughts, Agile Values, Being Agile, Coding Agile, Collaboration, Planning, Execution | Posted on 24-11-2010
I have a confession. I’m a code junkie.
I love coding the way some people love drugs. I think about coding when I’m away from it and I get cranky when things keep me away unnecessarily. I get a buzz every time I see the “New Project” screen in Visual Studio. I love the challenges and the creativity that coding requires, and I love the rush of seeing all the pieces come together in the end.
For code junkies, the “waterfall” era of software development were dark times. Months could go by where we wouldn’t write any code at all. We slogged through seemingly perpetual cycles of meetings and documentation trying, in vain, to fully understand every aspect of large systems before we did any coding.
Agile, of course, changed all that, and we code junkies couldn’t have been happier. By putting the focus on software over documentation and adaptability over planning, Agile enabled us to spend a lot more time coding, fixing, refactoring… all the things we love.
But there’s a downside to the Agile Manifesto for the code junkie. You see, to be Agile, one must not only focus on software and adaptability, but also customer needs and team collaboration. These are things that require social interaction, and (maddeningly, for us logical types) social interaction is anything but efficient. To truly and deeply communicate with another person — and even moreso with a group of people — requires a lot of back and forth, false starts, restating of ideas, deep listening, time for reflection, etc.
As a code junkie I find this half of Agile challenging. Talking to a perfectly logical compiler seems to me to be a very different thing from talking to an infinitely-complex and constantly-changing human. In fact, it seems like the two skills require completely separate and mutually exclusive portions of my mind. When I’m having a really good day with the compiler, for example, I’ll have trouble forming even simple sentences in English. And on the days where I spend all day in meetings or writing communications, I come home mentally exhausted, unable to wrap my head around even the simplest chunks of code.
One of the things our team does to help balance these two halves of Agile is segregating “in office” days from “work from home” days. When we’re planning, designing, integrating, or deploying our software we do so in the office. But when it’s time to sit down with the compiler for a while we work from home. Of course, being Agile and responsive to change, we still end up having to flip our internal switches a few times each day. But for the most part we are able to give each other the long stretches of uninterrupted time required by coding.*
Sadly, even this separation isn’t always enough for us junkies. We love coding so much that we can lose sight of the importance of collaboration in software development and start treating “in office” days as evil, time-wasting, buzz-killing enterprises. A good way to fight this to remember: It’s not what we build that’s important, it’s who we build it for.
Put another way, if we truly want to make the world a better place, we have to move from being “code junkies” to “casual users of code who occasionally overdo it when they get together with their friends.” I think that’s the Agile way.
* Some XP folks say that coding doesn’t require deep concentration or isolation, but I haven’t found this to be true, personally.
Posted by Howser | Posted in Coding Agile, Tools and Techniques | Posted on 16-11-2010
I was surprised and happy to see that jEdit was SourceForge’s Project of the Month for October. jEdit is a Java-based text editor that has been around for about 10 years. I’ve been using it for about 1/2 of that time.
jEdit is an important tool in my toolbox. To quote from The Pragmatic Programmer, “Tools amplify your talent. The better your tools, and the better you know how to use them, the more productive you can be.” TPP also has a tip dedicated to text editing:
Pragmatic Programmer Tip 22: Use a Single Editor Well
Why Use a Single Editor Well?
Again, I can’t outdo TPP’s answer to this question:
If you use a single editor (or set of keybindings) across all text editing activities, you don’t have to stop and think to accomplish text manipulation; the necessary keystrokes will be a reflex. The editor will be an extension of your hand; the keys will sing as they slice their way through text and thought.
Now I can’t go so far as to say that I do all my text editing in jEdit. There are too many features in Visual Studio that just don’t translate well to jEdit (not the mention the blasphemy of developing in .NET with a Java editor).
I am closet ‘vi’ user. I did all of my college programming and the first 8-10 years of professional experience in ‘vi’. I then decided to try the other side of the vi vs. emacs debate. My biggest issue with vi and emacs is that, if you forget a keystroke editing sequence, you have to go hunting for a reference guide to remember it. The user interface isn’t all that intuitive because, well, there isn’t one! I wanted a text editor that had robust keystroke support, but also had menu options and a good UI for doing the less common tasks that your fingers don’t remember.
In addition to being a very flexible text editor, with a bevy of supported syntax highlighting, it has a plug-in model. There are a number of great plug-ins that expand jEdit’s functionality. It’s easier to use jedit as my Single Editor because if there’s something that it doesn’t do or do well, there’s probably a plug-in that improves on the core feature set. Below are a few examples:
- CtagsSidekick: Uses Exuberant Ctags to parse a provide outlines for a huge number of file types. I use to navigate C#, Java, Ruby, an XML files.
- RubyPlugin: Features to make jEdit an intelligent Ruby editor, including Ruby API Documentation browser and auto-completion.
- XML: XML formatting, indenting, parsing, DTD generation, etc.
- Console: a command-line shell; makes it easy to write and run programs in scripting languages
Here’s my jEdit, with the colors customized, showing Ruby syntax highlighting, the CtagsSidekick parse on the left, and the Whitespace plug-in highlighting tabs.
jEdit also supports macro recording, saving, and playback. Macros are saved in BeanShell, a script-based Java derivative. If there’s a text transformation I need to do over and over, I just record it, edit it to tweak as needed, and then I have it for future use.
Another piece of advice from TPP is:
Always be on the lookout for better ways of doing things. If you come across a situation where you feel your current tools can’t cut it, make a note to look for something different or more powerful that would have helped. Let need drive your acquisitions.
Although I’ve used jEdit for years, there are things it doesn’t do very well. For example, it does not handle large files. For this, I’ve started using Notepad++, which I’ve noticed has a very similar feature set. As part of being on the lookout for better tools, I may eventually migrate to Notepad++ if it can prove itself as more powerful.
Posted by Murdock | Posted in Agile Thoughts, Being Agile, Coding Agile, Collaboration, Planning, Execution | Posted on 28-10-2010
Agile development is not just about iterative development and daily stand-ups, another important piece includes agile design. From my perspective, agile design is really our current set of industry best practices for designing and structuring our software. In most cases, these best practices should be followed. When they are not, the software starts to smell. The following is a list of software design smells from book Agile Principals, Patterns, and Practices in C#:
- Needless complexity
- Needless repetition
Design smells are attributes of rotting software. When your software rots, it will become more and more difficult to maintain. This is because of a number of reasons. With each new change, pieces of the original design philosophy may get lost. Or maybe a developer did not fully understand the original design and made changes not aligning to the intent. Regardless, as software evolves (and if it is not properly being re-factored), it becomes more and more difficult to maintain. It takes more time and expense to make each incremental change, and the changes are done less reliably, affecting stability. To ensure this doesn’t happen to your software, you need to be acutely aware of each these smells; and one of the most important of these is ‘Needless Complexity’.
Needless complexity is a trap that is easy to fall into. Everybody wants to build a robust and flexible piece of software that can not only satisfy the original requirements, but can easily accommodate any future change that may get thrown at it. Unfortunately, instead of improving the design of the software, this clutters the software with unused design artifacts that will likely never be used, making the code needlessly complex. And when new requirements are implemented, most times those changes don’t align with what the original developer anticipated, making the change that much more difficult and error prone to make. In most cases, it is better to implement a solid design using only patterns needed to satisfy the known requirements. When changes are needed down the road, you can start from that clean design and re-factor accordingly. Now that the new requirements are known, you are now in the best position to implement the best overall design. This makes adding new features more fun for the developer, but also improves the speed in which changes can happen, and improves the overall agility of the software (thus “Agile Design”).
I consider this an incredibly important smell because this is one that doesn’t necessarily have to be the result of numerous changes over the course of time. Software can smell overly complex right out of the gate! I’ve seen newly developed, large scale software systems that come out with serious issues, and took twice as long to build as they should have. Performance issues, resource issues, stability issues, expensive to change and maintain…. And a key reason behind the issues were because of design philosophies that led to over-engineered code. Too much anticipation for ‘potential’ future requirements reeks of needless complexity, and bad software.
Posted by LoonDog | Posted in Coding Agile, SDLC Approaches | Posted on 24-10-2010
I’d like to spend the next few posts talking about Quality. In particular, the quality of software and the practices Agile development teams can follow to ensure the consistent delivery of quality software.
We need working definitions of “Agile” and “Quality” before we start. Both terms are hopelessly overloaded in our industry, so, for our purposes here, let’s define “Agile” thusly:
An Agile software development team is a small, self-organizing, cross-functional group of people focused on delivering chunks of working software at regular intervals. Agile teams maintain constant communication with their customers and team members, enabling the team to quickly react to changing requirements and new ideas.
And “Quality” like so:
Quality software is working software that is easy-to-use, valuable to both user and creator, and easily adapted to fit other types of users, workflows, or industries. In other words, quality software is simple, powerful, and flexible.
With these definitions in place, I’d like to start the discussion by asking: Does Agile guarantee Quality? In other words, if you do everything described in the definition of “Agile” above, will you end up with everything described in the definition of “Quality”?
If you’ve had the opportunity to work in an Agile shop for a while you already know the answer: No. Agile does not, in practice, guarantee quality. You can, quite easily, follow every Agile practice in the book and still end up with software that is clunky, useless, and/or brittle.
That said, in my experience, Agile teams tend to produce much better software than their traditional, waterfall-and-Gantt-chart-based counterparts. There are lots of ways software projects can fail, but, if you live by the core tenets of Agile, you eliminate a lot of these possibilities.
By keeping teams small, empowered, and fluid, for instance, you avoid many of the headaches of silo-ing and baton passing that have plagued project management professionals for years. By maintaining a focus on working software you avoid accumulating “technical debt”, a cancer that has killed countless software projects. And by engaging in constant communication you avoid wasting resources on unnecessary features or complications (aka “gold plating”).
In short, Agile helps you avoid many of the anti-patterns endemic to requirements gathering and project planning. What Agile fails to address, however, are numerous other anti-patterns that can cause quality to decrease or projects to fail.
For example, any software developer worth his salt will tell you that you should avoid jumping into code too quickly. Often, five minutes with pencil and paper can save you hours of pounding away inside the IDE. Unfortunately, because of time boxing and the intense focus on customer needs inherent in Agile, it’s all to easy to fall into this trap. If you only have two weeks to get a new feature working, and you know can always re-do things in a later iteration (AND you love coding more than anything else), the temptation is almost too much to bear.
My point here is that there exists a gulf between “working software” and “quality software”. If you follow Agile principles you’ll end up with working software — which, it must be said, is infinitely better than no software at all — but you might not end up with quality software.
And that’s what I want to talk about in this series.
Up next: What is Quality?
Posted by Howser | Posted in Agile Thoughts, Coding Agile | Posted on 15-10-2010
This week was the second part of an iteration late in our release cycle. The area of the product I’ve been working in has undergone substantial re-design in the past 2-3 iterations, causing some instability. Unfortunately, this led me in to a brief period of bad programming practice — the “duct tape and bailing wire” approach. Perhaps you’ve been there:
You’ve spent a reasonable amount of time on design. You’ve coded it, the main use cases work well, stability seems to be good. Then your QA gets hold of it, and they find a few things, nothing really big, and you fix a few issues, but then you uncover a few more issues of your own along the way, and pretty soon, you have a steady stream of defects that either you or QA is finding. You address them as fast as you can, but things keep popping up. You’re running out of time because there’s a deadline looming, and you’ll be done when you just fix this one more bug and it feels productive because you’re fixing and fixing and checking in and fixing and it sorta feels like things are getting better but then another defect comes in and oh no it’s gonna take hours of rewriting to fix!
Duct tape and bailing wire. This is the phase I got into for a day this week. What’s wrong with it?
- As defects come in, you address them as fast as you can. Up-front design? Don’t have time. Unit testing? Nah. So you code from the hip, and maybe it’ll work, but chances are it’ll come back and bite you. You need have your eyes open to how your changes will affect the bigger picture and the other use cases or your changes will generate new defects.
- Your code smells. If you’re going into your code to fix a bug and find one in the process, you likely have one or more code smells. The Pragmatic Programmer calls these broken windows. If you fix a bug and think you’ve fixed it good but something else breaks, you have broken windows that you’re not paying attention to. If there’s a simple problem that takes modifying two classes, changing a method signature, and creating a new method overload to fix, you probably have a code smell.
The realities of some company situations (say, for example, a SaaS-model startup software company working to sell potential customers on our awesome product) can allow for postponing code smell fixes. Sometimes you just have to get the code working the way it is and refactor in future iterations. But you should always take all use cases into account and have the larger code architecture in mind when addressing defects to avoid fixing one defect and creating others.