← Implementation phase Integration →

10. Agile development and integration

Diagram showing who should read step 10

Read this if you are building software in-house, hired a vendor to build custom software or hired a vendor to customize existing open source software. Skip ahead to step 11. Integration if you bought off the shelf software.

Goals

To plan and execute an agile development process.
To incrementally deploy working software modules (use the “Development and Operations” or “DevOps” model).
To produce thorough technical and user documentation.
To create clear plans for staff training and software maintenance.

Actions

Creating a full agile work plan

An agile development project is only planned in broad strokes at the outset – the work plan shouldn’t be overly prescriptive about the timing or focus of future sprints. A key aspect of agile development is that it is iterative and incremental – it can accommodate changes based on what you learn as you go. This reduces overall risk of failure, and minimizes expensive and time-consuming revisions in the future. At this point, the plan should describe the overall technical architecture of the project, showing how a number of independent and interoperable modules will fit together to address the problem statement.

List criteria of success for the larger initiative (the problem to be solved and KPIs).
Have a general sense of what features will be included in the full software.
Decide on an overall technical architecture for the project (if you are working with a vendor, this should be largely their decision). The development team should consider how each individual piece is interoperable and fits coherently into the larger whole.

Sprint planning

In this phase you will begin developing software modules quickly, in 2-4 week sprints, integrating, testing, and operationalizing them as you go. Each module is:

Based on insights from the discovery research;
Written out as a clear user journey;
Planned and resourced according to complexity and necessary staff effort;
Developed in a short (2-4 week) sprint;
Fully operational at the end of the sprint;
Integrated, tested and operationalized at the end of the sprint.

Plan your first sprint (or work with the vendor team to do so.

Write the first user journey. This addresses a very specific problem that a future user will face. For example, a web application for trash removal may include:
- “I am a resident. I need to know which day trash will be collected this week, and whether it will be trash + recycling or recycling-only.”
- “I am a trash truck operator. I need to have a route for pickups at the beginning of every day.”
- “I am a waste management services operator. I need to know if hazardous conditions are expected, and notify residents of service delays.”
Sprint planning and scope should be determined by developer teams themselves. Some teams use a simple point-scoring system to assign an amount of work involved to complete each sprint (e.g. “1” for a simple module and “4” for a complex module). This helps with staffing and planning. If you are working with a vendor, revise the focus, scope, and timeline of each sprint together with actual software developers. If the software will be developed in-house, work with the technical team at every step.
Agile development depends on regular meetings. The development team will meet frequently (usually every day). If an external vendor is doing the development, there should be a meeting with the client at the end of every sprint (every 2-4 weeks).

Using a DevOps approach

“Development and operations” involves continuously integrating and deploying software modules in the actual working environment (technical and operational) as they are built. That means launching a piece of software on the city’s servers and/or having users start using it in their daily workflow. This reduces the risk of incompatibility at the end of a long process and the burdensome troubleshooting that comes with it. It also means that key associated staff become familiar with the product along the way, and can even provide feedback that helps guide iterations or future modules.

At the end of every sprint milestone, ensure that the module is deployed and put into use.
Coordinate with end-users to test and integrate the module in their daily workflow. They are the best resources for evaluation. If it isn’t functional from a technical or user standpoint, then it isn’t good enough – you can do another sprint to refine it.

Creating a documentation, training and maintenance plan

Using a DevOps approach means that modules will be continuously deployed into the city’s working environment. End users will have first-hand experience with the new software, and can give feedback as the product is being built. However, it is still important to document the software thoroughly. This includes technical documentation (overall architecture, modular integration, specific module function, ongoing maintenance considerations) as well as user documentation (what end users should be aware of, how to train new users, and best practices for getting the most out of the software). Hand in hand with the documentation, it is important to create a simple plan for ongoing staff training and software maintenance. These can be time- and budget-intensive, so be sure to allocate resources appropriately. Both should be continuously updated as working drafts throughout the software development process, and completed when all features are built and perform at a satisfactory level.

Ensure that the technical team and the product owner collaborate on writing thorough documentation (technical and user documentation).
Create a maintenance plan that includes:
- Regular (annual) performance audits.
- Staff training plan to onboard new users in the future.
- Staff time for software maintenance and updating (particularly if the city’s digital infrastructure changes in ways that affect the software performance).
- Strategies for sharing with other jurisdictions (if it is custom software or OSS).

Common challenges

Staff turnover

Retention and continuity can be a significant challenge, particularly for technical expertise – where city governments have to compete with private sector software companies for talent. Staff turnover can confuse a process and cause delays.

Lack of familiarity with and/or lack of involvement in the agile development process

City staff assume that they cannot or should not be involved with the developer team. Coders should have decision-making power when it comes to technical architecture, timelines, and required effort, but they do not know what end users need.

Lack of communication with end users

Frontline staff members will be affected by continuous integration. It is important to notify them of changes to their daily workflow, and to seek their feedback throughout the development process.

Lack of decision-making power granted to technical experts

This can result in over-specifying or under-specifying feature sets. It can also cause misalignment on sprint plans, timelines, effort required, and budget.

Lack of attention and resources for lifetime ownership

A contract is not the end of a procurement process, nor is final product delivery. It is important to have a detailed plan for ongoing use and ownership.