Suppose you have a software or system development organization of 400 people, working in 4 development programs with each 5 projects and 20 people per project, on average. You are using a heterogeneous tooling landscape of different tools from different vendors to support the projects, such as a requirements management tool (e.g. Doors), an architecture modeling & analysis tool (e.g. Enterprise Architect), integrated IDE editing-build-test tools (e.g. Eclipse, Visual Studio), a test management tool (e.g. HP ALM), source control tool (e.g. Git, SVN, RTC), defect control tool (e.g. Jira, Bugzilla, Trac), Scrum support tool (e.g. Jira Agile, Rally, RTC, TFS).
Of course, the tooling infrastructure did not drop out of thin air. It is the result of an evolutionary growth, where also practices evolved over time. Some tools are used in isolation but at a certain moment the need for integration of tools emerges. Let’s go through some of those integrations:
- Defect control – Source control
We need this integration to understand which source code was changed to solve a particular defect. Oppositely, it helps understand why a piece of code was modified.
- Test management – Defect Control
We need this integration to understand which defect was submitted because a test run failed, or more accurately because a particular step of a test case in a test run fails. Oppositely, we need to understand which test cases to re-execute when the defect is solved.
- Requirement management – Test management
When a test case passed, we need to know which requirements are OK. And oppositely, if a test case fails, we need to know which requirements are not OK. In addition, if we want to release our product, we need to understand how much of the requirements are OK – the essential requirements must be OK 100% and less essential requirements may be less than 100% OK. So we need to integrate requirements with tests
- Agile Planning – Requirement management
Most projects have limited capacity to develop the product or systems. We need to plan which requirements will be delivered in the next releases. That requires integration between requirements and plans.
- Agile Monitoring – Test management
A burn-down chart visualizes the amount of stories that are ready over the course of a sprint. A story is ready when it passes the acceptance criteria which typically involves testing. We need the integration between the stories counted as ready in the burn-down chart and the status of the test. Typically, this is a 3-way integration from the agile planning/tracking to requirements to tests.
- Agile Planning – Defect Control
To understand that you can finish the delivery of a release, you need to understand the amount of open work (e.g. stories) and the amount of open rework (defects).
- Source control – Requirement management & Architecture modeling
To implement a piece of code, you need to understand what it is expected to do. In traditional organizations you would have a specification document; in modern organizations the specifications are stored as requirements and architecture models in the corresponding tools. Oppositely, if you want to change the requirements or the architecture, you need to understand which source code is affected.
- Reporting – any tool
Many tools have limited reporting capabilities and when there are reporting capabilities it reports are often only accessible within the tool itself or as a PDF or Excel file. Integration with a reporting or data warehouse tool is often a neglected integration.
Now suppose you want to implement those tool integrations. Some integrations are supported by a plugin, adapter, integrator or synchronizer of the vendor of the tools. By the way, if you have tool A from vendor AA and tool B from vendor BB, it is not always obvious which vendor provides the integrator and how far that vendor is willing to support integration issues with the other tool (free of charge as part of the license agreement). So it takes effort to find the integrator, to install it, to configure it, to test it, to troubleshoot it and to solve problems of the integration. If there is no vendor with an out-of-the-box integrator, you might make your own scripts or application to integrate the tools. That takes effort to learn the APIs of the tools, to discuss and define the integration needs, to implement it, to test it, to troubleshoot it and to solve problems.
And then the vendors come with a newer version of the tools, that you need because bugs are solved and extra functionality is needed in the projects. When upgrading a tool, you may need a new version of the integrators, you need to reinstall it, to reconfigure it, to retest it, to troubleshoot it and to solve problems. And if the integrator belongs to vendor BB while you are upgrading tool A, you might run into issues with vendor BB to support it.
And in case of your own integrations, you need to learn the updated APIs for the new tool version (hopefully, no functions are deprecated that you use), to redefine the integration needs (to take advantage of the new capabilities of the new tool version), to adjust the implementation, to retest it, to troubleshoot it and to solve problems.
And then you want to add a new tool and/or replace an existing tool. You need to reconsider all your integrations with those tools, (re)install or (re)define and (re)implement them, (re)configure them, (re)test them, troubleshoot issues and solve problems.
To make things worse, you need to maintain resources with technical knowledge and expertise with the integrations and keep that knowledge up-to-date.If someone leave, he/she must be replaced. For integrations that you made in-house, you need to maintain that knowledge in-house; for vendor-based integrations you may depend on the vendor or external service provider partners, but still the organization specific knowledge needs to be maintained in-house. For open-source tools it may even be more difficult.
And to make things even more worse, software engineers have a tendency to invent their own tools and integrations in the margins of their work. What starts by retyping information from one tool into another tool, often ends in a “handy” little script that extracts information and import that in another tool. Run it in a crontab on a local computer and your done, until another projects wants the same thing. Before you know, your machine is overloaded with processes that are not yours, but that you need to support anyway in case there are issues.
“How much resources does maintenance and development of tool integrations cost?”
As I started, we assumed an organization of 400 people. What do you think, would you be able to support, develop and maintain all tool integrations (including maintaining technical knowledge of the APIs and the integrations) for about 10 tools for 400 people with 1 person or 1 FTE? And how much time do projects spend in the margins to work on integrations? Would 0.1% be realistic, which is 2 hours/year per person on average? And how much overhead would the projects lose because an integration does not work properly, is missing or down, and synchronizations need to be recovered? Would 0.2% be realistic, which is only 4 hours per year per person? Would that be a realistic representation of reality?
Anyway, in that case the total costs would be 1 + (0.001 * 400) + (0.002 * 400) = 2.2 FTE which is about $250k per year. My estimate would be that this is highly underestimated!
Now if I can do the same amount of tool integrations and more, keep it up-to-date with the latest version of those tools, better tested and of higher quality, for the cost of 1 FTE (about $120k per year), which solution would you prefer? If a tool integration is missing, I could add it for another $50k with no yearly cost increase, it would still be cheaper than the underestimated current reality.
“Impossible? Unrealistic? Are you mad?”
This is a absolutely realistic and possible. The only problem is that I have never met a development manager who is willing to increase the operational overhead costs by $120,000 per year on tool integrations. Why not? Because less effectiveness in projects are hidden costs counted as “normal” development costs and because a tools support team of – say – 4 people is already considered “overhead” costs.
“We ‘d rather hire an extra engineer than spend an extra $120k per year on tool integrations that we already have or don’t need!”