ACCU Conference 2010

The Roots of Scrum: How the Japanese Lean Experience Changed Global Software Development

Jeff Sutherland

Dr. Jeff Sutherland covers the history of Scrum from its inception thru his participation with Ken Schwaber in rolling out Scrum to industry, to its impact on Google, Microsoft, Yahoo, Oracle, Siemans, Philips, GE, and thousands of other companies. He describes the relationship of Scrum to experience at Bell Labs, MIT, iRobot, and the Grameen Bank, his communications with Kent Beck who used Scrum experience to help create XP, and how the Agile Manifesto accelerated Scrum adoption. Most important, he concludes by describing how team spirit is at the root of product innovation and hyperproductive teams and how that spirit can transform organizations.

Just as last year with Robert Martin's key note, the 2010 ACCU conference got off to a storming start with this key note from Jeff Sutherland, that I wish I could have got my entire team, the managers and to some extent the board to see. It contained good explanations about so many things about Scrum that I have been trying to explain for some weeks now.

Jeff Sutherland, one of the fathers of Scrum, described how Scrum is based on leadership, honesty and transparency and how, in comparison, fraud, greed and corruption fuelled the current global financial crisis. He went on to explain how Toyota's lean Scrum methodologies helped it to put its main competition, General Motors out of business and how, when it let leadership, honesty and transparency slip, it got into trouble that resulted in car recalls. He also described how lean allowed Toyota to develop the Prius in half the normal time by removing waste and impediments form the project. This case study also helped to demonstrate how lean agile methodologies helped define Scrum.

In talking about Scrum itself, Dr Sutherland went on to describe how Scrum:

• Removes management practices in favour of leadership
  
• Empowers employees
  
• Uses rules and procedures as enabling tools
  
• Uses hierarchy to support organisational learning
  

With the use of a diagram, Jeff Sutherland, showed how simple a framework Scrum really is. He went on to explain how Gantt charts are always 100% wrong, as things change quickly and they become out of date and how teams must be able to self-organise in order to be truly agile.

Introduction to Scrum: Shock Therapy -- How new teams in California and Sweden systematically achieve hyperproductivity in a few sprints

Jeff Sutherland

New teams need to learn how to do Scrum well starting the first day. This talk will describe how expert coaches at MySpace in California and Jayway in Sweden bootstrap new teams in a few short sprints into a hyperproductive state. This requires new teams to do eight things well in a systematic way. Good ScrumMasters with make sure their teams understand these basics for high performance and great ScrumMasters will make sure the teams execute all of them well. This session will review the critical success factors for new Scrum team formation.

Hoping to learn some more about Scrum I want into Dr Sutherland's follow-up presentation and I wasn't disappointed as there was more of the same! It was good to have the idea that Sprints could be one, two, three or four weeks as I had previously believed that Sprints had to be 30 days to be Scrum.

Jeff Sutherland described how he encourages Scrum to be implemented and understood from board level down and two of the main things that get enforced, by the board, are Test Driven Development (TDD) and Continuous Integration (CI). I see these as two of the bare minimum requirements for software development.

I especially like the definition of “done” given:

• Feature complete
  
• Code complete
  
• No known defects
  
• Approved by product owner
  
• Production ready
  

Dr Sutherland reiterated his previous point that Scrum should be lean and reminded me that although the right way to estimate stories is with relative story points, in a Sprint planning meeting those stories should be broken into tasks and real time estimates given to each one.

Real QA - the argument for smarter quality assurance than mere testing

Tom Gilb

Traditional testing, as a means towards quality assurance, is far too costly and far too ineffective. There are much smarter ways to approach software quality. This session will argue with facts for a half dozen more-cost-effective ways to get reliability and all other qualities, like usability, security, adaptability, into your systems, than conventional testing.

It seems that Scrum themed sessions are the order of the day for me as Tom Glib works closely with Jeff Sutherland and both attend each others presentations. I went into this presentation unsure what to expect and hoping to learn something. Although I enjoyed it and may just possibly have missed something subtle, it appeared the whole point was that quality should be specified quantitatively in requirements, and those requirements should also be high quality and bug free, rather than trying to ensure it with only testing at the end.

Jon Jagger's DOJO

I was invited, well it was more like press ganged, to a late collaborative programming session. I was a member of one of five pairs who were developing a simple unsplice function and accompanying tests. We ran several iterations and were informed we should be working as group, rather than compete. At the end of each iteration the people in each pair were changed, and each pair would work on another pairs code. It was an interesting look at the way teams collaborate together in an agile environment. In retrospect I wish we'd planned and shared a bit better, but with the information we were given I think we did quite well.

Hello, I'll Be Your Tester Today

James Bach

This talk is how I introduce myself to programmers. It is especially aimed at programmers who may wonder why any intelligent person would willingly be a tester, and why projects needs testers, and how to work with testers. I will talk about what makes skilled testers different and special and about the commitments I make to the programmers I work with. I will help you set a high, but reasonable standard for the testers you work with.

If I am successful, then by the end of my talk, at least a few programmers in the audience will have become testers.

The ACCU conference has always been about quality, but this year is very special, especially with the keynote speakers. James back is a highly charged and very amusing individual. In fact he;s the sort of tester I'd like to be working with.

He explained that he started his testing career as a test manager for Apple about 20 years ago. Throughout his key note, James highlighted the many differences between testers and developers. Including the fact that, apparently, testers have more social interaction. Another subtlety he pointed out was that testers don't break code, developers do and testers do not steer projects, they make suggestions and hold the light so developers can create great code.

One of the best parts of the keynote were the testing related subtitles James added to a clip from The Towering Inferno where Steve McQueen is gathering the requirements he needs to fight the fire. I can't describe its brilliance in a way that will do it justice here.

The point that resonated the most for me is that testers need to be able to think their way around a system with or without a specification or requirements. They need to think outside the box as, after all, requirements are often rumours, right? This is the oppostie end of the scale from what Tom Gilb was saying yesterday. From an understanding of what the system should do and bags full of common sense testers should be able start testing and test effectively.

Genemodulabstraxibilicity

Steve Love

The title says it all! If it's hard to write unit tests, or they take too long to run, if your plugins need the whole application for their distribution, if you can't (re)use a bit of your colleague's code without importing the entire team's work, if your multi-threaded code performs better when you run it sequentially, or you've got 5 versions of the same 3rd party library littering your source repository, then you're suffering from this. Or perhaps just part of it...

Modern languages provide us with many tools for creating beautiful, modular, general, flexible and simple abstractions, yet it seems they give us even /more/ tools for writing ugly, monolithic, specific, rigid and complicated, er, /concretions/.

With examples of both kinds in C#, Python, C++ and maybe even C++0x (if it stays still long enough to get the syntax right :-)), this is a talk about our (code's) propensity for wearing too many hats.

Steve is one of my oldest friends from the ACCU, going back to the first conference I attended in the early naughties. The lessons and observations he discussed in the session are what the ACCU is all about for me. He described lots of the common pitfalls of the code that a lot of people write and how to avoid them sensibly.

There was of course just the right amount of singleton bashing and plenty of humour, not to mention of course, the subliminal elephant that no one was meant to be thinking about. Other important points made were that too many comments is a sign of poorly written code and public inheritance is the strongest form of coupling.

What surprised me the most was Steve's description of a painful code review where he was pulled up for using single entry, single exit. The company Steve was working for actually explicitly stated that they tried to enforce multiple exit points. Just when I thought I'd heard it all....

Data warehouse, OLAP, Data Cubes, Business Intelligence - Buzzwords explained

Sven Rosvall

In the database world there are many buzzwords that most software engineers only hear about but never get experience with. This talk aims to clarify what lies behind some of these buzzwords and describe key differences from the more common transactional database. The talk will also provide enough insight for engineers to decide if any of these technologies are useful in their current or future projects.

The presenter has work for many years on the periphery of the databases, wondering about these buzzwords. He eventually got involved with data warehouses and now wants to share his experiences with fellow engineers.

Sven is another ACCU member I've known for many years. His presentation did exactly what it says on the tin. I came away understanding the difference OLTP and OLAP and that we really need to try and break our own databases up onto separate servers to improve performance.

Although I'm still a little hazy on data cubes, I now understand dimensions and how data is reorganised when it is transferred from OLTP to OLAP to make it easier query. This was certainly one of the most useful sessions of the conference so far.

Database Testing Demystified

Sven Rosvall

Testing databases is not as easy as unit testing of classes and functions. Databases are full of state and internal logics which must be set up before testing can start. There are also lots of dependencies that are difficult to isolate or stub out.

This presentation will look at some techniques to create automated tests for databases and how to debug SQL code including single stepping stored procedures. We will automate database testing with the use of popular unit testing frameworks. We will test simple CRUD statements, calls to stored procedures and verify triggers. Test suites in Java and .Net will be demonstrated.

We will also see how a database can be developed with agile methods. Databases are traditionally developed up front as the database schema is difficult to change later.

Another great session from Sven (immediately following the previous one) even if the material from the second half does overlap what I'm speaking about a fair bit. The highlight here for me was seeing the SQL Unit Testing Framework SPUnit in action and the ability to step through stored procedures in SQL Server Management Console.

TDD at the System Scale

Steve Freeman, Nat Pryce

We present our experience applying "system-test first" test-driven development (TDD) in the development of large systems and systems-of-systems. We try to address integration and system testing as early as possible. The sooner the system is in a deployable state, the easier it is to react to changing business needs because we can deliver new features to users as soon as is deemed necessary. We therefore start by writing tests that build and deploy the system and interact with it from the outside and, to make those tests pass, add code to the system in the classic "unit-test first" TDD style.

Many teams applying TDD start writing unit-tests and leave integration and system testing until late in the project because they find it difficult to write tests that cope with the distributed architecture and concurrent behaviour of their system. We will describe how we address common pitfalls, including unreliable tests, tests that give false positives, slow-running tests and test code that becomes difficult to maintain as the system grows.

We will also describe how writing system tests guides our architectural decisions. Writing unit tests first guides design of the code to be flexibile and maintainable. In a similar way, we have found that writing system tests first guides the architecture to be easier to monitor, manage and support.

A couple of colleagues and myself saw Steve and Nat speak at JP Morgan in February and it was a little bit like disorganised chaos with Steve doing most of the talking and Nat chipping in. There was a marked improvement this time, with Nat doing the majority of the speaking.

I was a little bit concerned to see that they were advocating starting with system level tests before unit tests, but I can see how it might work in some circumstances. The same old system level problems, such as driving the GUI, faking external systems, etc, are still there, but Nat and Steve came up with some interesting ideas to overcome theses.

The main points that stood out for me where:

• Making the system scheduler external so that it can be faked during test.
  
• Use an audit interface instead of direct logging so the system can decide what it wants to report as a whole.
  
• Add hooks so that you can easily ask the system if it is happy.
  

Another point I realised is that I really need to read their book!

The Multicore Revolution: We've Only Just Begun

Russel Winder

I will and have travelled a long way to hear Russel speak, even for a short period of time, on many occasions. He is very entertaining and today was no different. Before talking about the architecture of multi-core processors, Russel spoke about the different forces occurring in atoms. Unfortunately it's so long for me since A-level physics, I was rather lost!

The two important points that Russel made were that we need to stop thinking in a single processor, single memory model and consider a mulit-core, multiple memory model and that although the hardware is mostly there, there is a lot of catching up to do on the software side.

Fortunately no kittens were harmed during the production of Russel's slides as he is using open office instead of power point.

Understanding User Stories

Rachel Davies

On traditional projects, software developers are often presented with volumes of requirements that seem to be definitive and yet are full of holes. On agile projects, the whole team gets involved in establishing user stories on index cards. Come along to this session to understand the life-cycle of a user story from inception to implementation. You'll get a chance to practice working through some real user stories. We'll also investigate ways that agile teams handle requirements that don't seem to be user stories.

I didn't read the description of the session sufficiently! So I was a little shocked when the layout of the room indicated it would clearly be interactive. I don't really enjoy that sort of thing, but this was superb and the 90mins just flew by!

This session was, for me, really about confirming what I have read recently about writing user stories. It was nice to know I was generally right and to find ways to smooth some of the rough edges.

We played a game where one person wrote down a story, e.g. “A man opens a door.”. Then the next person draws it and the following person then writes down what they think has been drawn and this goes right around the group. Each person only sees what the previous person has written or drawn. The idea of the exercise is to show how if one person describes something and another person documents it, things get lost and other things get added. The final description was “A man with a hat, closes a door.”

We also did a brief exercise where one member of a three or four person team is the product owner and the other members of the team ask them questions to determine what is needed for a system and write stories. This also worked very well.

Based on what I saw I'd be very happy to have Rachel come and coach my team.

FindBugs: Testing (Java-) Code Against Bugs And Defects

Bernhard Merkle

This session should be really fun. We will make a deep dive into findbugs, a very useful tool which every programmer should use to test Java code against potential bugs, defects and antipatterns.

After a first overview of the problem domain (static analysis), we will take some of Joshua Bloch's "Effective Java" Rules from his book (a ACCU mentored developer mailing lists currently runs...) and see how / if they can be implemented in a tool like findbugs. I will show how findbugs works internally and we will implement a new bug detector (findbugs rule) to find specific defects we are interested in.

Additionally we cover different levels of static analysis like code-, design- and architecture-level. Interestingly findbugs checks code and design level, but fails itself (!) on the architecture level; which means the tool itself suffered major architecture erosion from version 0.7 to the current 1.3.8.

I am participating now in the findbugs project and contributed several architectural refactorings on the findbugs 2.0 branch (see the sourceforge/googlecode SVN repository) which is currently under development and should hopefully be finished during the time of the ACCU 2010 conference. We will see, how far an automated and tool supported approach can lead to better results and how tools like findbugs can discover areas of code- and design-erosion and suggest improvements. Additionally we will look at the next generation of analysis which is architecture analysis.

Bernhard took us through a very interesting look at what findbugs provides and showed us many examples of bugs in established code bases, such as Eclispse and the Java SDK itself, that could have been found simply by running findbugs. He included some great photos of non-programming bugs and lots of excellent humour.

Findbugs can also highlight architectural problems and Bernhard showed us how the architecute of findbugs itself has got steadily worse over the years.

Bernhard convinced me that we should add findbugs to or static analysis arsenal, which current consists of just PMD.

Saturday Keynote

Due to the volcano eruption in Iceland, Dan North didn't make it to the conference for the Friday keynote, so Russel Winder (as described above) moved his keynote forward. So the Saturday keynote was replaced with four fifteen minute “Lightning talks”. Lightning talks are just very brief talks, usually only five minutes and usually for people who aren't already speakers to give them a chance to try it out. On this occasion the talks were given by four conference speakers:

• Bob "Unclude Bob" Martin – The birthday problem.
  
• Walter Bright – The biggest problem with C.
  
• James Bach – Testing models.
  
• Jim Hague – Are you getting enough?
  

All were humorous in one way or another and all very good.

Coupling as resistance to change, understanding and fixing it

Tony Barrett-Powell

Coupling, the number of things a unit of code depends on, is an important consideration when designing and maintaining software. Without due care a slow increase in coupling between units over a few releases of a software product, or even over a number of iterations within a release, can lead to software that is difficult to change, or worse results in a ripple effect throughout seemingly unrelated units. We cannot write software without some coupling, how else would the code achieve anything? Thus, coupling is not a bad thing in itself, it is the degree to which a unit is coupled to other units that can be undesirable, yet is not an absolute measure. Understanding of what coupling is, its many forms and how to recognise them is useful if we want to avoid unnecessary coupling in the software we create. It is important to control coupling in software if it is to be improved over a number of releases, as high coupling slows development, increases time to market and inevitably lost revenue.

This session attempts to explore coupling in all its manifestations, examine the difference between good and bad coupling and to consider its bedfellow cohesion. The session will also explore the techniques used before, during and after the act of design which can be used to reduce unnecessary coupling and as an aside looks at those which can lead to increased coupling. The intent of the session is to arm those designing and writing software with the understanding and techniques to create loosely coupled and maintainable software.

I have known Tony a good few years, but this is the first time I have seen him speak. He is very good and this subject is close to my heart.

Tony explained how lots of coupled code is very difficult to change and even harder to understand. He took us through inheritance coupling and temporal coupling and explained cohesion as a measure of the strength of functional relatedness of elements within a module. If it's all together it's cohesive, if it's split up it's coupled.

Tony continued by explaining Afferent and Efferent coupling. Afferent coupling occurs at the package level and efferent coupling is the number of classes inside a package that reference classes outside of a package.

A Simple Matter of Configuration - how can we tame the complex world of configuration?

Roger Orr

Configuration is a vital element of many programs. However it is often hard to get configuration right, leading to people wasting time and programs that do not work correctly.

In my experience explicit discussion of design options for the configuration of a program is rare and, all too often, the choice is made arbitrarily. I believe that looking at the usage patterns of the program early on helps to pick the best method(s) for configuring it and hence reduce the cost of problems caused by configuration issues.

Configuration is a complex subject and there doesn't seem to be an obvious single solution that works in every case, but we can try to fight against the common 'anti-pattern' of using multiple, unrelated, configuration techniques at the same time.

I will look at some of the issues surrounding configuration; firstly by trying to answer the "six key questions" (who, why, what, where, when and how) for this subject to help understand the size of the problem and the forces at work.

I'll then sketch out some possible design patterns and look at their trade-offs and interaction with the intention that I (and you) can reduce the pain of getting programs working in different environments by making more informed and deliberate decisions.

I am looking at the problem in the general case, using examples from various problem domains, and expert knowledge of a particular language or API is not assumed.

This was the last session I attended as I was speaking in one of the final slots of the day. Again, I have known Roger, through the ACCU a long time and shared many a drink and lunch with him. He took us through a humorous look at how to configure applications. This wasn't a look at build configuration, but the way we actually pass settings to a program.

I have come across many of the same problems in my career, and although there was no specific solution to the problem, one of the proposed solutions, storing settings in a database is one I use regularly. Roger showed me lots of methods and their pitfalls, that I had never even considered!

Enterprise Web Application Development in Java with AJAX and ORMs

Paul Grenyer

Recently Java enterprise web application programming has been leaning towards a more classical J2EE approach. Traditional Java Server Page (JSP) programming, and even libraries such as Struts, are being replaced by new AJAX libraries that make GUI programming more straight forward, robust and easier to unit test.

In this session I will look at what an enterprise web application is.

I will demonstrate how to develop a more robust GUI with an AJAX library and how to create a more object orientated Data Access Layer (DAL) with an Object Relational Mapping (ORM) library.

After defining what an enterprise web application is I will move on to demonstrate how to create a DAL, with real code examples, and explain how to use a registry to abstract away Data Access Objects (DAOs) so that the real DAOs can be used in production and integration testing while seamlessly substituting mock objects for unit testing.

Then I will look at an AJAX library and demonstrate how to create the presentation layer in Java, again with real code examples, and make Remote Procedure Calls to access the DAL. I will then look at how to integrate the AJAX library into traditional Spring MVC in order to tap into the vast library of functionality that the Spring Framework can provide for web based enterprise application. I will explain how the tools provided by Spring make integration testing of DAO objects very simple.

Finally I will look at how to use Spring Security to authenticate users of the application and secure individual Remote Procedure (RPC) calls made from the client application, running in a browser, to the server.

I don't know how good I was, but this presentation flowed for me easier than any other presentation I have done. I have never managed to speak for the full ninety minutes before and I came in on time.

It was one of the final sessions of the conference, I was up against some stiff competition and most people had actually already gone home. Even so I had an audience of about half a dozen.

My slides are available here: http://paulgrenyer.net/dnld/Enterprise_Web_Application_Development_in_Java_with_AJAX_and_ORMs.pdf.

Finally

This was the best ACCU conference I have ever been too, simply because of the amount directly useful information I was able absorb. I'm sure next year will be even better.