Maintainability

Summary

The majority of software costs are not tied to initial development but to ongoing maintenance, which includes fixing bugs, keeping systems operational, adapting to new platforms, and managing technical debt. To minimize maintenance challenges, software should be designed with three key principles: operability, simplicity, and evolvability. Operability focuses on making it easy for operations teams to manage and maintain the system efficiently. Simplicity aims to reduce complexity, making the system easier to understand and manage, which helps avoid maintenance difficulties and bugs. Evolvability ensures the system can adapt to new requirements and changes, supporting future modifications with minimal effort. By adhering to these principles, software can be more maintainable and adaptable, reducing long-term costs and improving overall effectiveness.

Details

The majority of the cost of software is not in its initial development, but in its ongoing maintenance:

Fixing bugs
Keepings its system operational
Investigating failures
Adapting it to new platforms
Modifying it for new use cases
Repaying technical debt
Adding new features

Software should be designed in such a way it will minimize pain during maintenance, by following three design principles.

Design principles

Operability Make it easy for operations team to keep the system running smoothly.

Simplicity Make it easy for new engineers to understand the system, by removing as much complexity as possible from the system.

Evolvability (definition) Make it easy for engineers to make changes to the system in the future, adapting it for unanticipated use cases as requirements change. Also known as extensibility, modifiability or plasticity.

Operability: Making life easy for operations

Operations teams are vital to keeping a software system running smoothly. A good operations team typically is responsible for the following, and more:

Monitoring the health of the system and quickly restoring service if it goes into a bad state
Tracking down the cause of problems, such as system failures or degraded performance
Keeping software and platforms up to date, including security patches
Keeping tabs on how different systems affect each other, so that a problematic change can be avoided before it causes damage
Anticipating future problems and solving them before they occur (e.g., capacity planning)
Establishing good practices and tools for deployment, configuration management, and more
Performing complex maintenance tasks, such as moving an application from one platform to another
Maintaining the security of the system as configuration changes are made
Defining processes that make operations predictable and help keep the production environment stable
Preserving the organization’s knowledge about the system, even as individual people come and go

Good operability means making routine tasks easy, allowing the operations team to focus their efforts on high-value activities. Data systems can do various things to make routine tasks easy, including:

Provide visibility into runtime behavior and system internals with effective monitoring.
Support automation and integration with standard tools.
Avoid dependency on individual machines, allowing maintenance without system interruption.
Offer good documentation and a clear operational model (“If I do X, Y will happen”).
Provide good default behavior, while allowing administrators to override defaults when necessary.
Implement self-healing features, but allow manual control over the system state when required.
Ensure predictable behaviour, minimizing surprises.

Simplicity: Managing Complexity

Complexity in Software Projects

Small Projects: Typically simple and expressive. Large Projects: Often become complex and difficult to manage, described as a “big ball of mud.” Symptoms of Complexity:

Explosion of the state space
Tight coupling of modules
Tangled dependencies
Inconsistent naming and terminology
Hacks for performance
Special-casing to address issues

Challenges of Complexity

Maintenance Difficulties: Increased complexity can lead to budget overruns and delays. Bug Risks: Harder to understand systems increase the risk of hidden assumptions, unintended consequences, and unexpected interactions.

Reducing Complexity

Objective: Simplify systems without necessarily reducing functionality. Accidental Complexity: Complexity arising from implementation rather than the problem itself. Goal: Remove non-essential complexities to improve maintainability.

Role of Abstraction

Definition: Abstraction hides implementation details behind a simple interface. Benefits:

Simplifies understanding and use of the system
Promotes reuse, leading to efficiency and higher-quality software Examples:
High-Level Programming Languages: Hide machine code, CPU registers, and syscalls.
SQL: Abstracts complex data structures, concurrency, and crash recovery.

Challenges in Finding Good Abstractions

Difficulty: Designing effective abstractions is challenging, especially in distributed systems. Distributed Systems: Good algorithms exist, but creating useful abstractions to manage complexity is less clear.

Evolvability: Making Change Easy

Understanding Evolvability

Definition: Evolvability refers to the ease with which a system can be modified and adapted to changing requirements. Importance: Systems are unlikely to have static requirements; changes in facts, use cases, business priorities, user requests, platforms, regulations, and system growth necessitate adaptation.

Agile Framework

Agile Practices: Agile methodologies help manage change at an organizational level.

Technical Tools:

Test-Driven Development (TDD): Focuses on writing tests before code to ensure system functionality.
Refactoring: The process of restructuring existing code without changing its external behavior.
Scope of Agile Techniques: Often applied to small-scale changes (e.g., individual source code files).

Scaling Agile to Data Systems

Challenges: Applying Agile principles to larger data systems or multi-application environments. Example: Refactoring Twitter’s home timeline assembly from Approach 1 to Approach 2.

Factors Influencing Evolvability

System Simplicity: Simple and understandable systems are easier to modify. Abstractions: Effective abstractions can enhance a system’s adaptability to changes.

Terminology

Evolvability: Preferred term for agility at the data system level, emphasizing the ability to evolve and adapt.

INDBS Noter

Explorer