Skip to content

Introduction

In this unit we'll recapitulate the historic approach to software development, provide a formal definition of software development, retrace common issues, and present the elements pertaining to nowadays most widespread agile software development.

Lecture upshot

Software development is complex, because requirements are often unknown or tend to evolve. A development approach that attempts to plan all eventualities does not promote success. Nowadays we rather set on factors that foster fast adoption and flexibility.

Software processes and models

Two definitions:

  • Software (development) process: A set of related activities that lead to a software product.
  • Software (development) process model: Simplified representation of a software process.

Key elements, part of any software process:

  1. Specification: Defining expectations and limitations of the ideal outcome.
  2. Design and implementation: Producing the software.
  3. Validation: Ensuring the software actually does what the client wants.
  4. Evolution: Adapting the software to changed requirements.

Multiple models have been proposed and followed in practice. In the following, we will take a closer look at three representative examples.

Waterfall

  • The first software development process model.
    • Derived from existing, general system engineering models, 1970.
  • Document oriented and plan-driven. Ideally:
    • Each activity concludes in a document. Must be "signed off", before proceeding to next activity.
    • All process activities must be planned before realization.
  • Philosophy: If every phase has been perfectly realized, the outcome is a perfect software.
    • A single linear run could be enough.

Activities

Note: The waterfall model appears strictly linear, but occasional revisions to previous activities are considered.

Waterfall activities, according to Ian Sommerville:

  1. Requirements analysis & definition: "The system’s services, constraints, and goals are established by consultation with system users. They are then defined in detail and serve as a system specification."
  2. System & software design: "The systems design process allocates the requirements to either hardware or software systems by establishing an overall system architecture. Software design involves identifying and describing the fundamental software system abstractions and their relationships."
  3. Implementation & unit testing: "During this stage, the software design is realized as a set of programs or program units. Unit testing involves verifying that each unit meets its specification."
  4. Integration & system testing: "The individual program units or programs are integrated and tested as a complete system to ensure that the software requirements have been met. After testing, the software system is delivered to the customer."
  5. Operation & maintenance: "Normally (although not necessarily), this is the longest life cycle phase. The system is installed and put into practical use. Maintenance involves correcting errors which were not discovered in earlier stages of the life cycle, improving the implementation of system units and enhancing the system’s services as new requirements are discovered."

Reality

  • It is impossible to anticipate all requirements at any given stage, communication over documents may distort client needs.
    • Activities and their documents will be signed off, although they are incomplete.
    • A single run is never enough.
    • Revisions will be the norm, not the exception
  • The waterfall model is not how humans solve problems:
    • We do not block on one issue until it is solved perfectly, and only proceed if everything is sorted out.
    • We often do not know all requirements beforehand, we often change our minds.

Warning

The waterfall model is still popular among managers, as it provides an illusion of planability.

When to use

The waterfall model should only be used, when...

  • Little to no requirement evolution to expect.
  • Context can be well formalized, e.g. a very math oriented context.

Info

The most extreme form of the waterfall model is a generative, formal approach: Documents and code are not written by hand, but generated or inferred from previous stages. Ideally, when the formal transitions are provably correct, everything generated must be likewise correct.

Incremental development

  • The most common software development model
  • Philosophy:
    • Interleave specification and development
    • Expose the product to the client as early as possible
    • Evolve with every iteration

Activities

  • Start: Begin with rough description of required functionality. Focus on core scenarios.
  • Iterations:
    • Specification: Revise software specification
    • Development: Revise software implementation
    • Validation: Revise software validation
  • End: Client accepts final version

Pros and cons

Advantages over waterfall model:

  • Cost of changes is reduced, fewer documents need revision on change
  • Customer feedback already throughout development, reduced risk of missing the requirements
  • Through demos, progress is more visible to the client
  • Rapid delivery and deployment allows earlier benefit of core functionality

Inconveniences:

  • Progress is less visible to the management
    • Creating frequent document to track progress may even constitute overhead and slow things down
  • Changes tend to corrupt system, as it grows
    • Additional time needed to refactor / rearrange (clean up) system
  • Evolving core structures may be problematic for large-scale projects with many team
    • Teams need clear structures and responsibilities to collaborate

When to use

The incremental (agile) model should be used, when...

  • Changing user requirements are expected (business, e-commerce, personal applications)
  • Customer requirements, or customer wishes are unclear
  • Client needs fast access to operational core functionality

Warning

Agile should not be considered as an excuse for lowering engineering standards, especially with respect to code quality and testing. Agile is about frequent iterations to reach fast client feedback. It is not about jeopardizing the project for temporary speedups.

Reuse-Oriented

  • A popular software development model
  • Philosophy:
    • Do not reinvent the wheel. Make use of existing software components to speed up development.
    • Existing components can be:
      • Similar systems, to find inspiration from
      • Existing code of legacy or related systems
      • Dedicated frameworks and libraries

Activities

  1. Requirements specification: (Same as other software processes)
  2. Component Analysis: Search for existing components to implement that specification. Usually, there is no exact match. Components can only provide partial functional coverage.
  3. Requirements Modification: Analyze requirements regarding discovered components. Adapt requirements to components, where possible. If needed, extend search for components.
  4. System Design with Reuse: Design a framework for the system, or reuse existing framework, based on selected components. Design tailored solutions where no off-the-shelf components are available.
  5. Development and Integration: Fill, the gaps, i.e. develop the missing components and integrate them to a system.
  6. System Validation: (Same as other software processes)

Pros and cons

Advantages over waterfall model and incremental model:

  • Development speedup for standard components
  • Security gains for standard components

Inconveniences:

  • Invitation to dependency hell
  • Unpredictable outcome, when components are used in foreign context

Recommendation

Be selective with project dependencies. Every component is also a dependency and potential risk. Make a clear gains-to-costs assessment before integrating whatever technology or component.

A prominent reuse catastrophe

Simplified explanation of the Arianne V88 disaster:

  1. Engineers reuse software of the Arianne 4 rocket, to create Arianne 5
    • Arianne 5 is required to transport more satellites per launch, to make launches cheaper
  2. Arianne 5 is designed larger, and with a more powerful engine
    • Thrust Arianne 4 + boosters: ~ 3780 kN
    • Thrust Arianne 5 + boosters: ~ 8090 kN
  3. Arianne 5 lifts of, 4 June 1996
    • Engines work as intended, thrust is significantly higher than for Arianne 4
    • Arianne 5 goes up "faster" than Arianne 4
    • Numeric values of resulting flight path are in higher ranges
  4. Rocket computer tracks flight path with reused Arianne 4 software
    • Arianne 5 computer has a 64-bit processor, uses unsigned floats
    • Arianne 4 software is written for 16-bit processors
      • Arianne 5 converts values down to unsigned 16-bit int
      • Maximum value is a 16-bit unsigned integer: 65535
      • Converting values above 65535 leads to overflow
    • 40 seconds after lift-off, a numeric flightpath value surpasses 65535
      • Arianne 5 converts value without overflow check
      • Arianne 5 feeds false data to reused Arianne 4 software
  5. Reused software module for flight path computation crashes
    • Flight path tracking software cannot overflow value
    • Module crashes, now returns crash reports instead of numeric results to main computer
    • Main computer misinterprets crash report as flight data
      • Misinterpretation suggests rocket is off-course
      • A rocket off-course is potentially live threatening
    • Main computer triggers self destruction
  6. Rocket and all satelites on board destroyed
Who's to blame?

Three engineering flaws: Software reuse in foreign context, type conversion without overflow check, lack of type safety for result interpretation.

Agile software development

  • Agile is an approach based on four manifesto pillars:
    1. Individuals and interactions over processes and tools
    2. Working software over comprehensive documentation
    3. Customer collaboration over contract negotiation
    4. Responding to change over following a plan
  • Agile development maximizes the iterative model
    • Adapting to uncertain or changing requirements is considered more important than to-the-minute planning
    • The user not knowing what they want before seeing a demo is considered the rule, not the exception

Note: Agility is not a criteria when dealing with safety critical systems (aviation, medical equipment, nuclear reactor...)

Agile is not per se good

Recently the agile approach (especially management oriented frameworks) have become more and more unpopular, linking the work style to increased project failure rates. However, the critique mostly aims at agile management methods, rather than the openness to iterative development iteself.

Scrum

Scrum is one way to materialize the agile approach as a project management philosophy.

Key factors:

  • Organizing progress into Sprints of manageable progress, typically 2-4 weeks.
    • Each sprint defines a goal / target state of desirable functionality.
  • Delegation of sprint responsibility to a scrum master.
    • Scrum master role may take turns among developers.
    • Scrum master is closer to developers than a "Product owner"

Sprints

Central element of sprints is a precise tracking of achieved and missed functionality.

  • "Sprint planning": Scrum master and product owner elicit a reasonable sprint duration and goal
  • "Daily stand-up": The scrum master begins each day with a 15-20 minute meeting
    • Per attendant, answer:
      1. What did I complete yesterday?
      2. What will I work on today?
      3. Am I blocked by anything?
  • "Sprint review": Team gathers to assess which sprint goals were met. Can be at intermediate sprint moments.
  • "Spring recipes": Team gather for discussion group dynamics / tools. Can be at intermediate sprint moments.

Image credits: Atlassian

Kanban

Kanban is an alternative agile management methodology.

  • Kanban sets on a strong visualization of ongoing tasks
  • Kanban: Japanese for "Cardboard"
  • Tasks are represented as cards
  • Each developer has a column in a grid (board)
  • Each task moves through stages:
  • To do
  • In progress
  • Code review
  • Done

Image credits: Atlassian

Most collaborative coding solutions (GitLab, GitHub) natively support Kanban, to coordinate on tasks:

Source: GitLab documentation

Kanban philosophy

  • Continuous flow of task assignment (in contrast to Scrum sprints)
  • A manager can easily redistribute, or prioritize tasks, because they "see" who is available, or in the middle of something.
  • Limiting the "Work in progress" is a visual activity, developers do not suffer from constant context switching.

Engineering practices

  • Agile is a philosophy around dynamically adapting to changed requirements.
  • The ability to quickly for adapting to changed requirements rises and falls with the ability for fast iterations.
    • Faster iterations cannot be imposed, but are the product of good engineering habits
    • Agile preconditions are almost exclusively around developer habits and code quality.

Code quality

  • Ultimately, it all comes down to engineering practices, notably code quality
  • Consider the following code example:
/**
 * Vaj ghob Hutlhbogh vay''e'
 */
public static boolean test(int input) {
  int x = -1;
  boolean[] yyy = new boolean[input];
  while (!(x > input - 2)) {
    if (input % (x + 2) == 0) {
      yyy[x + 1] = true;
    } else {
      yyy[x + 1] = !true;
    }
    x++;
  }
  for (int k = input - 2; k >= 1; k--) {
    if (yyy[k] == true) {
      return false;
    }
  }
  return true;
}
  • Three questions:
    1. What on earth is this code doing?
    2. Is the implementation correct?
    3. How easy will it be to be "agile" and adapt this code to a changed requirement?

Guidelines for good code:

  • Code should be must be as easy to read as possible (style)
    • Comment your code
    • Document your code
    • Use good variable and function names
    • Use consistent indentation
    • Favour clarity over brevity
    • Avoid performance tweaks unless necessary
    • Use polymorphism instead of type checks
  • Code should be must be correct and secure (semantics)
    • Test all of your code
    • Test all conditions
    • Test as often as possible
    • Limit variable access
    • Code against interfaces
    • Code defensively
  • Code should be must be easy to change

The hardest problems

There are only two hard things in Computer Science: cache invalidation and naming things.
--Phil Karlton

CI/CD

Even the best coder makes mistakes or has a bad day. What can be done to ensure code quality?

Use automated quality checks. Rigorously reject bad code.

  • "Continous integration / Continous delivery" pipelines make sure, that:
    • Every software change respects code quality
      • Code compiles
      • All tests pass
      • Code is fully documented
      • Code respects formatting rules
      • ... and much more
    • Every accepted change automatically triggers a build
      • Executables are continuously generated and delivered to the client
  • Main interest: Whatever happens, code can only get better, not worse
    • There is always a stable version of the software
    • Everything that already worked will also work in the future

Good practice

A good CI/CD pipeline fosters good engineering practices and ensures you actually can adapt to changing requirements.

Homework

For next week, please contribute the following homework:

Worst practices

Write the most convoluted and hard-to maintain version of the below function that still works. Create your list of bad-code characteristics.

import java.util.LinkedList;

/**
 * Removes all elements from a provided lists of strings that contain an "M".
 */
public List<String> removeAllElementsWithM(List<String> unfilteredList) {
  List<String> filteredList = new LinkedList<>();
  for (String element : unfilteredList) {
    if (!element.contains("M")) {
      filteredList.add(element);
    }
  }
  return filteredList;
}

Share the worst working version on mattermost.

Literature

Inspiration and further reads for the curious minds: