Abstraction, Encapsulation, Information Hiding and Behavioural Compatibility
Abstraction, encapsulation, information hiding and Behavioural compatibility are four fundamental principles of software engineering. They're closely related and frequently confused, but they describe different things. None is specific to a particular programming language or paradigm, and all three apply to software design at every level, from individual functions to distributed systems.
I've been writing software professionally for more than twenty-five years, and I've certainly used these terms without always considering the differences between them. Recently, I wanted to understand their origins and, in particular, the design principles that allow an implementation to change without affecting its public interface or externally observable behaviour.
The aim is to design software so that its implementation can be changed in the future without affecting its clients. Refactoring immediately comes to mind, but that describes the process of restructuring existing code without changing its externally observable behaviour. I was interested in the principles that make such changes possible in the first place.
In other words, how do we change the way something works without breaking everything that depends on it?
I suspected the answer was information hiding, as described by David Parnas. It turns out I was right, although there are some important distinctions worth understanding. As with so many principles in software engineering, we've known about this for more than fifty years.
Four Different Concepts
Before looking at their origins, it's worth establishing what each term means.
These concepts complement one another, but they're not interchangeable.
Abstraction provides a way of representing something in terms of its essential characteristics and capabilities, without requiring clients to understand the underlying details. In Multi-Paradigm Design for C++ (1999 - ISBN-13: 978-0201824674), James O. Coplien describes abstraction as follows:
Abstraction deals with the general case without seeing the details of any particular instance.
Coplien's definition emphasises the distinction between the general characteristics of something and the details of a particular implementation. In his 1972 Turing Award lecture, The Humble Programmer, Edsger W. Dijkstra explained:
The purpose of abstracting is not to be vague, but to create a new semantic level in which one can be absolutely precise.
This is an important distinction. Abstraction isn't simply about removing details. It's about establishing a level at which we can reason precisely about something without needing to understand everything beneath it.
Encapsulation groups related data and behaviour together within a defined boundary, often controlling how they can be accessed.
Information hiding conceals design decisions, particularly those which may change, so that other parts of the system don't become dependent on them.
Although abstraction, encapsulation and information hiding are often used together, they serve different purposes.
Behavioural compatibility ensures that a replacement implementation continues to meet the expectations of its clients.
These distinctions matter because using one doesn't necessarily mean you're using the others. A class can encapsulate its state and behaviour while exposing that state through public properties, allowing clients to modify it directly. Similarly, a repository can abstract database access while exposing methods that require clients to understand SQL and the underlying database schema. In both cases, implementation details are exposed that clients shouldn't need to understand.
In his 2002 article, The Law of Leaky Abstractions, Joel Spolsky argues that all non-trivial abstractions leak to some degree. In other words, details of the underlying implementation can become visible to clients, despite the abstraction intended to conceal them.
This reinforces the distinction between abstraction and information hiding. An abstraction can simplify how something is used without completely hiding how it works. Information hiding, on the other hand, is concerned with preventing clients from depending on design decisions that are likely to change.
Behavioural compatibility is slightly different. It's not another mechanism for structuring software, but a property of a change or replacement. It ensures that the expectations established by a component's public contract continue to be met.
Where Did These Ideas Come From?
Abstraction has a much longer history than software engineering. It's a fundamental idea in mathematics, philosophy and other disciplines, where unnecessary details are removed to concentrate on the essential characteristics of something.
In programming, procedural abstraction allowed programmers to define operations without requiring their callers to understand the individual steps involved. A caller could use a procedure based on what it promised to do rather than how it was implemented.
Data abstraction extended this idea to data types, allowing them to be defined by their operations rather than their internal representation. A related distinction appears in the Gang of Four's Design Patterns: Elements of Reusable Object-Oriented Software (1995 - ISBN: 978-0201633610). In section 1.6, Specifying Object Interfaces, the authors distinguish between an object's type and its class. The type describes the interface an object provides, while the class defines how that interface is implemented. This reinforces the distinction between what clients can depend on and how something works internally.
Ole-Johan Dahl and Kristen Nygaard's work on Simula during the 1960s introduced classes which brought data and procedures together. Simula 67 helped establish many of the ideas which would later become central to object-oriented programming.
Encapsulation developed alongside modular programming and became particularly associated with object-oriented programming.
However, encapsulation isn't exclusive to object-oriented programming. Modular programming languages established boundaries between interfaces and implementations independently of class-based object-oriented programming. Modules, packages and services can all establish boundaries around related state and behaviour.
Information hiding has a more specific origin.
In 1972, David L. Parnas published On the Criteria to Be Used in Decomposing Systems into Modules in Communications of the ACM, Volume 15, Issue 12, pages 1053–1058.
Parnas challenged the conventional approach to dividing software systems into modules. At the time, systems were often decomposed according to the steps involved in processing data. A program might have separate modules for reading input, processing it, sorting the results and producing output.
Parnas argued that this wasn't necessarily the best approach, particularly when considering how software changes over time. Instead, he proposed that modules should be organised around design decisions which were difficult or likely to change, with each module hiding those decisions from the others.
This became known as information hiding, although Parnas had discussed the idea in earlier work, including his 1971 technical report Information Distribution Aspects of Design Methodology. The important point is that Parnas wasn't simply suggesting that implementation details should be private or inaccessible. He was arguing that other modules shouldn't need to know about decisions which might change.
If one module knows how another module works internally, it can become dependent on those details. When the implementation changes, the dependent module may need to change as well. By hiding those decisions behind a stable interface, a module can change independently of its clients.
Around the same period, Barbara Liskov was developing ideas around data abstraction. Her work on the CLU programming language during the 1970s helped establish abstract data types as a practical programming technique. Liskov later described this work in A History of CLU, explaining how data abstraction developed and how CLU provided direct language support for it.
An abstract data type is characterised by the operations it provides, rather than the representation used to implement those operations. A set, for example, might be represented internally using an array, a linked list or another data structure. Clients should only depend on operations such as adding, removing and checking for values rather than the representation itself.
This is closely related to information hiding, but the emphasis is different.
Data abstraction establishes a conceptual model and its operations. Information hiding prevents clients from becoming dependent on the implementation decisions behind that model.
The distinction is also explained in MIT's Reading 12: Abstract Data Types, which discusses abstraction, encapsulation, information hiding and representation independence.
Representation independence is particularly relevant here. It describes the property that allows the representation of an abstract data type to change without requiring changes to its clients.
Consider a component which represents a position using two numbers. Internally, those numbers might represent Cartesian coordinates (x and y) or polar coordinates (distance and angle). Although both representations use two numbers, they have different meanings. By exposing operations for working with positions rather than the underlying coordinates, the component can change its internal representation without requiring its clients to change.
Let's take a look at how these ideas work in practice. I use TypeScript for the examples, but the principles apply regardless of programming language.
Changing an Implementation
Consider a function which sorts an array of numbers into ascending order.
Our first implementation uses bubble sort:
export function sort(numbers: number[]): number[] {
const result = [...numbers];
for (let i = 0; i < result.length - 1; i++) {
for (let j = 0; j < result.length - i - 1; j++) {
if (result[j] > result[j + 1]) {
[result[j], result[j + 1]] =
[result[j + 1], result[j]];
}
}
}
return result;
}
Bubble sort is relatively straightforward. It repeatedly compares adjacent values and swaps them when they're in the wrong order.
Unfortunately, it isn't particularly efficient for large arrays. Its average and worst-case time complexity is O(n²).
Tests can verify the externally observable behaviour of the function.
import { sort } from './sort';
describe('sort', () => {
it('sorts numbers in ascending order', () => {
const numbers = [5, 2, 8, 1, 9];
expect(sort(numbers)).toEqual([1, 2, 5, 8, 9]);
});
it('does not modify the original array', () => {
const numbers = [5, 2, 8, 1, 9];
sort(numbers);
expect(numbers).toEqual([5, 2, 8, 1, 9]);
});
it('preserves duplicate values', () => {
expect(sort([3, 1, 3, 2])).toEqual([1, 2, 3, 3]);
});
});
They don't know or care which sorting algorithm is being used.
For this example, we're assuming ordinary finite numbers. Special numeric values, such as NaN, would require additional consideration when defining the function's behaviour.
Now imagine we've measured the performance of the application and established that sorting large arrays is taking too long.
We decide to replace bubble sort with quicksort:
export function sort(numbers: number[]): number[] {
if (numbers.length <= 1) {
return [...numbers];
}
const pivot = numbers[Math.floor(numbers.length / 2)];
const left = numbers.filter(n => n < pivot);
const middle = numbers.filter(n => n === pivot);
const right = numbers.filter(n => n > pivot);
return [...sort(left), ...middle, ...sort(right)];
}
Quicksort has an average time complexity of O(n log n), although its worst-case complexity remains O(n²). In principle, this makes it a better choice than bubble sort for large arrays. However, this particular implementation creates several intermediate arrays and performs repeated filtering, so we'd still need to measure its performance to establish whether it provides a meaningful improvement.
What's important here is that the public interface hasn't changed.
The function still accepts an array of numbers and returns a new array containing those numbers in ascending order. It doesn't modify the original array, and it preserves duplicate values.
The unit tests remain unchanged and continue to pass.
This demonstrates all three concepts.
The sort() function provides an abstraction for sorting numbers. The function boundary encapsulates the implementation. Information hiding means clients don't depend on the choice of sorting algorithm, allowing it to change without affecting them.
The tests also help us establish behavioural compatibility. They verify that the replacement continues to meet the expectations we've specified, although they don't prove that every possible behaviour is identical. As Dijkstra also said:
Tests only prove the presence of bugs, not their absence
The strength of the behavioural contract matters too. For example, a stable sorting algorithm preserves the relative order of elements which compare as equal. An unstable sorting algorithm might produce a correctly sorted result while changing that order. If clients depend on stability, replacing one algorithm with another could break their expectations even though the function's signature remains unchanged.
There's also a connection to my recent article, Premature Optimisation: We Knew Better 50 Years Ago, where I explored the importance of establishing correctness before improving performance.
Here, we've established the expected behaviour through tests before changing the implementation. We can then improve performance while retaining confidence that the behaviour hasn't changed.
Information Hiding Between Components
The same principles apply at the component level.
Consider a Node.js package called number-sorter, which exposes our sorting function.
Initially, its structure might look like this:
number-sorter/
├── package.json
├── src/
│ ├── index.ts
│ └── bubble-sort.ts
The package's public interface is defined in index.ts:
export { sort } from './bubble-sort';
A client imports the sorting function from the package:
import { sort } from 'number-sorter';
Later, we replace bubble sort with quicksort, and the internal structure changes:
number-sorter/
├── package.json
├── src/
│ ├── index.ts
│ └── quick-sort.ts
And we update index.ts:
export { sort } from './quick-sort';
Nothing changes for the client. It continues to import sort from number-sorter, and the function continues to behave as expected.
The package provides an abstraction through its public interface. It encapsulates the sorting implementation within the package, and information hiding allows the internal implementation to change independently of its clients.
Unfortunately, Node.js packages can also allow clients to bypass their intended public interfaces.
Rather than importing from the package itself, a client might import directly from an internal module:
import { sort } from 'number-sorter/src/bubble-sort';
This is known as a deep import.
The client is now dependent on the package's internal file structure and the choice of sorting algorithm. When bubble-sort.ts is replaced with quick-sort.ts, the import breaks, even though the package still provides exactly the same functionality.
The client has become dependent on a design decision which should have remained hidden.
Earlier this year, I wrote about this problem in Deep Imports Considered Harmful, where I explored how deep imports increase coupling and undermine the ability of components to evolve independently.
Node.js provides a mechanism for explicitly defining a package's public interface through the exports field in package.json:
{
"name": "number-sorter",
"exports": {
".": "./dist/index.js"
}
}
Assuming the TypeScript has been compiled to JavaScript in the dist directory, this allows clients to import from the package root while preventing them from accessing internal paths which haven't been explicitly exported through normal Node.js package resolution.
Packages can deliberately expose additional subpaths, of course. The problem isn't using a subpath import, but depending on internal modules which aren't part of the intended public interface.
This is a good example of the difference between encapsulation and information hiding. Putting code into a package establishes a boundary, but that boundary is of limited value if clients can freely depend on the implementation behind it.
This is exactly the kind of problem Parnas was describing in 1972.
Changing a Database Without Changing Its Clients
The same principles apply at the service level.
Consider a customer service which is consumed by a web application, a mobile application and one or more other services.
The customer service initially stores customer information in PostgreSQL. It exposes an HTTP API which allows clients to retrieve customer information:
GET /customers/123
The response is:
{
"id": "123",
"name": "Jane Smith",
"email": "jane@example.com"
}
The service provides an abstraction of customer information. Clients know how to retrieve a customer, but they don't need to understand how or where that information is stored.
The service encapsulates the data and behaviour associated with managing customers. Database access, validation and business logic are grouped within the service boundary, and clients interact with them through the HTTP API.
The choice of PostgreSQL is an implementation decision which is concealed from the clients through information hiding.
Now imagine that the team responsible for the customer service decides to replace PostgreSQL with DynamoDB.
The service's internal implementation changes, but the public API remains the same:
GET /customers/123
And the response is still:
{
"id": "123",
"name": "Jane Smith",
"email": "jane@example.com"
}
The web application, mobile application and other services don't need to change. They don't even need to know that the database has been replaced.
This is information hiding at the service level. The choice of database is a design decision hidden behind a stable public interface, allowing it to change independently of the applications which depend on it.
However, imagine the original service had exposed database-specific details through its API:
GET /customers?table=customers&column=id&value=123
Now the clients need to understand something about the underlying database schema. If the service moves from a relational database to a different storage model, those assumptions may no longer be valid.
The service still provides an abstraction and encapsulates its database access, but it hasn't successfully hidden the database design decision.
This is particularly important with independently deployed clients. A web application can usually be updated centrally, but a mobile application might remain installed on a user's device for months or years without being updated.
Changing an implementation detail which has leaked through a public API can therefore require changes across multiple applications, teams and release schedules.
OData (Open Data Protocol) provides an interesting example of how information hiding can be compromised at the service level. It allows clients to query and navigate data exposed by a service, including filtering, selecting properties and following relationships between entities. While this can be useful, implementations that expose the underlying data model directly risk coupling clients to internal structures.
Changes to those structures can then require changes to consuming applications. OData doesn't inherently violate information hiding, but it makes it particularly important to distinguish between the public model exposed by a service and its internal implementation.
Of course, preserving the endpoint and JSON response isn't necessarily enough. The new implementation must also honour the service's behavioural contract, including any guarantees about consistency, error handling and other observable behaviour which clients rely on.
The important point is that clients should only depend on what the service promises to provide, not on how it happens to provide it.
This is the same principle demonstrated by replacing bubble sort with quicksort and by changing the internal structure of a Node.js package. The scale is different, but the underlying problem is exactly the same.
The Interface Isn't the Whole Contract
There's one more distinction worth making.
Throughout these examples, I've described changing an implementation without changing its public interface or externally observable behaviour.
These are two separate requirements.
A function can retain exactly the same signature while changing its behaviour. A service can continue to expose the same HTTP endpoint and JSON structure while returning different results or handling errors differently.
For example, imagine our customer service initially returns HTTP 404 when a customer doesn't exist. After replacing its database, it starts returning HTTP 500 for the same situation.The endpoint and response format for successful requests might be unchanged, but the behaviour has changed.
Similarly, our sorting function might continue to accept and return arrays of numbers while unexpectedly modifying the original array. The public interface hasn't changed, but the behavioural contract has been broken.
However, there's another complication. In Hyrum's Law, Hyrum Wright observes that with enough consumers of an API, somebody will eventually depend on every observable behaviour, whether it's part of the documented contract or not.
For example, our customer service might return customers in a particular order, even though that ordering isn't guaranteed by its API. Clients may nevertheless come to depend on it. Replacing PostgreSQL with DynamoDB could change that ordering, breaking those clients without violating the documented contract.
This doesn't invalidate information hiding, but it does demonstrate its limitations. We can conceal implementation decisions, but we cannot always prevent clients from becoming dependent on their observable consequences.
This is why automated testing is so important.
Tests can verify that an implementation continues to meet its expected behaviour when internal changes are made. They don't guarantee complete correctness, particularly when the tests are incomplete, but they provide confidence that changes haven't introduced unexpected behaviour.
Information hiding allows us to change implementation decisions independently. Preserving the behavioural contract ensures those changes don't break clients.
It's also important to recognise that information hiding doesn't mean public interfaces must never change. Sometimes requirements change, and the capabilities a component provides need to change with them. That's a separate concern involving API evolution, backward compatibility, and versioning. The objective is to avoid forcing clients to change simply because an internal implementation decision has changed.
Design Patterns and Information Hiding
These principles also appear in several of the design patterns described by the Gang of Four in Design Patterns: Elements of Reusable Object-Oriented Software.
The Bridge pattern separates an abstraction from its implementation so that both can vary independently. This is closely related to information hiding, particularly when an implementation needs to change without affecting its clients.
The Strategy pattern defines a family of algorithms, encapsulates each one and makes them interchangeable. Our sorting example demonstrates the underlying idea, although it doesn't implement the Strategy pattern itself. Bubble sort can be replaced with quicksort without requiring clients to change.
Adapter and Facade are also relevant, although their emphasis is slightly different. Adapter allows an existing implementation to be used through an interface which its clients expect, while Facade provides a simplified interface to a more complicated subsystem. Both can help prevent clients from becoming dependent on implementation details.
However, none of these patterns guarantees information hiding. An abstraction can still expose implementation details, and an interface can still create dependencies on decisions which might change.
This brings us back to Parnas's original argument. The important consideration isn't simply which patterns or interfaces we use, but which design decisions we allow other parts of the system to depend on.
Finally
What I find particularly interesting is that we've understood these principles for more than fifty years, yet we still build software which makes changing an implementation unnecessarily difficult.
The Gang of Four's advice to:
program to an interface, not an implementation
is closely related to Parnas's work on information hiding. Both encourage us to think carefully about the dependencies we create. However, simply introducing an interface doesn't guarantee that we've hidden the decisions behind it.
Separation of concerns is part of this. A caller should be concerned with what a component does, rather than how it does it. But separating those concerns isn't enough if the interface still exposes details of the implementation. We need to consider not only where we draw the boundaries, but what we allow to cross them.
Of course, we can't anticipate every change or prevent every dependency. Nor should we introduce abstractions simply because something might change one day. That would risk adding complexity without providing any real benefit.
What we can do is recognise the decisions we're making, consider which are likely to change, and avoid unnecessarily exposing them to the rest of the system.
For me, that's the most useful lesson from Parnas's work. Good software design isn't about hiding as much as possible. It's about being deliberate about what we expose and, just as importantly, what we don't.
The question I'll be asking myself more often is: What decisions am I making today that I don't want other parts of the system to depend on tomorrow?
Acknowledgments
Thank you to Chris Oldwood and Alan Stokes for the detailed reviews and many, many comments.
Comments
Post a Comment