This is the second post in the Architecture Snippets series. These posts aim to get developers up-to-speed quickly with software architecture patterns and principles.
It’s time to talk about Event Sourcing.
What is Event Sourcing?
Event Sourcing is simple to understand once you start thinking about application state in terms of events. Application state is usually stored in a persistent data store, such as a database.
We’re used to thinking about databases as static entities. When we want to create, update or delete data, we think about mutating state.
In relational database management systems (RDMS), such as SQL Server, an update query like the one below replaces one data value with another.
UPDATE Inventory SET Quantity = Quantity - 1 WHERE ItemID = 12345;
If more than one user (or application) were to run this query at the same time, the RDBMS needs to handle the concurrency conflict that would arise. It does this using transactions and locks. But the process of managing conflicts slows down performance and makes scaling difficult.
Another problem with mutating data is that information is lost during a transaction. We don’t know why, how or when data was changed because the history of changes are not recorded anywhere.
We can, of course, write changes to separate log tables. But this means writing extra code to handle logging and may have a performance impact because we’re increasing the number of database writes.
Say no to mutation
What if, instead of mutating state, we recorded a sequence of events as the authoritative source of truth?
An event is a notification that something just happened. We can record each event once without worrying about data mutations or managing log tables.
When we want to update application state, we simply record the change as an event and add it to the list. The ‘database’ then becomes a record of all the data mutations that have ever happened!
Persisting this sequence of mutations as events is called Event Sourcing. The persisted collection of events is called the Event Store.
The Event Store usually records the details of the mutation as well as the name of the event and a time stamp. Here’s a simplified example.
If you want to read the current state of an entity, just select all relevant events and replay them back. You might think reading the data in this manner would be slow but it can actually be very fast.
Why might you use Event Sourcing?
Event Sourcing is often used with the CQRS pattern. Recall that CQRS means logically separating database reads (queries) and writes (commands). This results in two pipelines, one that can be optimised for querying data and one optimised for writing data.
The Event Store acts like a queue within the command pipeline. Commands have side effects (they mutate state) which are represented as events and are stored in the event store. This decouples mutations of state from the processors of those mutations.
2. Audit trails
The Event Store provides a complete audit trail of all changes. The events in the store contain a comprehensive view of what data was changed and by whom.
This auditing ‘out of the box’ can be extremely valuable in highly regulated industries.
3. Enhanced debugging
With Event Sourcing, you can reconstruct the state of entity to any point in time. This greatly enhances debugging as you can ‘time-travel’ between different application states! Imagine trying to do that with a traditional relational database.
If you’ve used the Redux devtools browser extension, you’ll know just how powerful (and fun!) this is.
4. Fuelling multiple read stores
The Event Store contains a complete record of all mutations. One of the advantages of this is that you can project the current state into multiple read optimised data stores.
For example, you could replay events and transform them into graph or search optimised databases like Lucene or Azure Search. Read-optimised databases become consumers of the events and could even listen for specific event types.
Event Sourcing solves some of the performance issues of traditional databases. This is because it decouples the producers of events from the processors. Since the Event Store is append only it avoids data conflicts too.
Event streams can also be easily replicated since they’re immutable. This makes scaling a breeze!
Why might you not use Event Sourcing?
While Event Sourcing isn’t difficult to understand, implementing it can be. So you should only use it when really need to.
If any of the following apply then you probably don’t need to use Event Sourcing:
- Your domain or business logic is simple.
- Your application has a very low occurrence of conflicting updates to the underlying data. For example, systems that predominantly add data rather than updating it.
- Your system doesn’t require comprehensive auditing.
- Your system doesn’t require easy roll backs.
- You don’t need to optimise read and write databases (e.g. CQRS).
- You need strong transactional consistency.
- Your application requires real-time updates to the views of the data.
You don’t have to use Event Sourcing for all of your application state. For example, it might make sense to store some state in an Event Store but other state in a relational database. This especially applies if you’re using a microservices architecture.
So that concludes this quick summary of Event Sourcing. Be sure to keep an eye out for future posts where I’ll cover other architecture recipes that every developer should know.