--- title: Event-Carried State Transfer Pattern url: https://www.grahambrooks.com/event-driven-architecture/patterns/stateful-event-pattern/ date: 2017-05-18T19:11:14+01:00 author: Graham Brooks description: In contrast to the notification pattern the Event-carried state transfer discourages calls to the publishing service by containing all the required data in the event payload. This decoupling leads to a very clean architecture but not without significant costs. tags: - architecture series: - event-driven-architecture --- # Event-Carried State Transfer Pattern ```asciidoc = Event-Carried State Transfer Pattern :permalink: "/event-driven-architecture/patterns/stateful-event-pattern/" :sample: _code-samples/event-driven-architecture/samples.json One thing about developing software - it is never dull. I have been lucky to be on projects that are on the leading edge with technologies and patterns that become mainstream. Event driven architectures and particularly distributed event driven architectures using state transfer feels like one of those trends. Driving applications through events is not new. What makes it interesting is scale and distribution that we are now developing. 100s or 1000s of services redundantly deployed into clusters all interacting with each other using events. This architecture gets interesting ... fast. For the last 3 years I have been developing event driven systems based on event-carried state and event sourcing. The link:/event-driven-architecture/patterns/notification-event-pattern/[notification event pattern ] decouples the publisher and subscriber in design (the publisher does not know who is listening). The subscriber receiving the events still needs to know about the publisher and be given an API it can use to read the current publisher state. Publisher and Subscriber are still runtime coupled because the subscriber needs to call the publisher for details of the change. This callback reduces availability and increases load and traffic. By adding state to the published events the Event-Carried State Transfer pattern reduces or removes this run-time dependency. A publisher could send an event and immediately quit. Because the event contains state the subscriber only needs the event to do its work .Direct Subscription - Shipping subscribes to orders from Sales [plantuml, stateful-sequence,svg, align="center"] ---- include::_pages/event-driven-architecture/patterns/plantuml.skin[] participant Sales participant Shipping Shipping -> Sales: Subscribe (Orders) Sales -> Shipping : New Order\n { order-id: 1234 items: []} ---- ''' .Broker based publish subscribe - Shipping subscribes to orders, Sales publishes orders [plantuml, stateful-broker-sequence, svg, align="center"] ---- include::_pages/event-driven-architecture/patterns/plantuml.skin[] participant Sales participant Broker participant Shipping Shipping -> Broker: Subscribe (Orders) Sales -> Broker: New Order { order-id: 1234 items: []} Broker -> Shipping: New Order { order-id: 1234 items: []} ---- Decoupling the publisher and subscriber is a really nice characteristic of event driven patterns. *But* what we gain from this decoupling also introduces significant complexity in the form of consistency or more accurately *inconsistency* in the overall state of the system. The link:/event-driven-architecture/patterns/notification-event-pattern/[notification event pattern ] introduces inconsistency that is resolved by rapid processing of the notification event. The system of components is still inconsistent for a period until the event has been processed but because the subscriber calls back to the publisher it sees the current state. The system does not need to be concerned with event race conditions and event ordering. Like any API the publisher design needs careful consideration to provide enough information to be useful to known and unknown subscribers. == Historical reliance Systems that use events carrying state have to track multiple event streams. To understand orders we need to know about the customers they are for. .JSON example event structure [source,json] ---- include::{sample}[tag="stateful-example-event-structure"] ---- In this simplified model the order has an `id` and reference to the customer with the `customer-id` on its own this event is of little use to the subscriber. .A more complete event might look something like: [source, json] ---- include::{sample}[tag="stateful-new-order-event"] ---- When subscribers have received and process the *new-order* event. Lets say that the customer updates the order adding more pencils. .Basic order amendment [source, json] ---- include::{sample}[tag="stateful-basic-order-amendment"] ---- So now we have two events about the same order. In a distributed multi-node environment processing these events in sequence becomes significant because `new-order` and `order-amended` could be received in either order. Subscribers could be made tolerant perhaps by using an event store themselves to store events until they need to be used in sequence but this puts a lot of burden on the subscriber. Delivering events in the right order is one of the things we can ask a message broker to do. Generating a sequence number in the publisher gives us a ordinal for sequencing. While this helps solve problems when data is generated from a single publisher but lets assume that we have a system managing customers and another orders. customer events are not going to share sequence numbers with the order system. Any system that listens to both the customer and orders systems will need to handle events arriving in a non-deterministic sequence. Back to the real world where we have to assume that orders can have many (unbounded?) amendments before it is takes action and goods shipped. .Multiple amendments [source, json] ---- include::{sample}[tag="stateful-multiple-amendments"] ---- The event now contains a unique id and an ordinal to indicate publication/processing order. If the events arrive at two subscriber instances in a cluster at the same time then the subscriber can resolve ordering by making sure that the events are processed in ordinal sequence. Most message brokers can be configured to supply events in order which can really help simplify subscriber processing. *But* the order references a customer by an opaque id and contains no other details. The shipping system does not know how to address the customer or any other details essential to good customer experiences. The sales system could provide customer details as part of the order, e.g. the customers name and preferred shipping address. By considering these details to be part of the order the shipping system only needs to worry about events about orders. This approach adds some coupling. The sales system is second guessing the subscribers need for customer information. The sales system could provide a reference to the customer `http://customer/{id}` as part of the event. The subscriber would then be able to call the customer system to get the required details. This simple addition makes it easy for the subscriber to read the customer details it needs. By calling back to the customer management system for the current state of the customer adds some runtime coupling, reducing availability and service independence. To maintain low coupling and higher availability the shipping system can subscribe to customer events, using the customer id to build its own data model specific to shipping orders. This is much more complex, events are now sourced from multiple publishers (customer and orders). While events from a single component can maintain order we now have race conditions in update from multiple sources. Often these conditions don't cause a problem because event processing is much faster than event production. Relying on slow event production is risky. link:https://martinfowler.com/eaaDev/EventSourcing.html[Event Sourcing] and particularly when replaying events breaks the slow producer assumption. ''' [IMPORTANT] ==== Events are a statement of fact (point in time) and not a statement of intent. ==== Events carry state at a point in time - typically when the event was generated. Asynchronous distributed systems can distribute commands, request/response semantics etc. these are not events (IMO). == Wrapping Up The high availability and loose coupling provided by this pattern are very attractive capabilities. These capabilities come with a cost - much greater complexity. Events carrying state require careful design, Events need to be received in order. Event processing needs to be idempotent or events processed once and only once. System consistency needs to be monitored, managed and controled to provide good experiences in client UI applications. Events change over time making release and maintainance more complex particularly if Event Sourcing is used. Further Reading:: . link:https://martinfowler.com/articles/201701-event-driven.html[Martin Fowler:What do you mean by “Event-Driven”?] ```