July 13, 2024
Journey Platform: A low-code software for creating interactive person workflows | by Arjun Raman | The Airbnb Tech Weblog | Might, 2023
Arjun Raman
The Airbnb Tech Blog

Journey Platform: Low-code notification workflow platform that permits technical and non-technical customers to create complicated workflows by means of a easy drag and drop person interface.

By: Arjun Raman, Ken Snyder, Mengting Li

Efficient communication hinges on delivering the precise message, to the precise viewers, on the proper time. At Airbnb, our objective is to interact our customers — each company and hosts — by delivering inspirational and informational notifications by means of varied channels, corresponding to electronic mail or in-app messages.

Traditionally at Airbnb, complicated notification workflows have been solely managed by engineering groups, with every workflow requiring the deployment of code. As our platform developed, we acknowledged the necessity for a low-code or no-code answer to streamline the creation of those intricate notification workflows. In response, the Advertising Expertise crew developed the Journey Platform, a robust software that permits non-technical customers to construct and ship customized notifications based mostly on our customers’ engagement with Airbnb.

The objectives of the Journey Platform are:

  1. Empower customers to simply create event-driven notification workflows utilizing an intuitive drag and drop interface.
  2. Allow real-time execution of those notification workflows for well timed and related communication.
  3. Provide a unified interface for managing transaction notifications, corresponding to upcoming journey reminders and promotional notifications.
  4. Assure Service Degree Agreements (SLAs) for processing varied varieties of notification workflows, together with transactional and promotional communications.
  5. Cut back the time required to develop complicated notification workflows.

Journey Platform permits customers to iterate quicker by permitting self-serve workflow creation. It has lowered the time taken to help a brand new use-case from 1–2 months to only 1–2 weeks.

Determine 1: Time saved in Journey Platform
Determine 2: Journey Platform structure overview

The important thing parts of the Journey Platform are:

  1. Journey Platform UI: WYSIWYG software permits customers to tug and drop parts and create a workflow. The workflow definition is then transformed to a customized DSL (Domain-specific language) which will be interpreted and executed by the workflow orchestrator.
  2. Workflow Orchestrator: Mind of the system, the workflow orchestrator takes within the workflow definition DSL from the UI. As soon as a workflow is launched, it listens for occasions from the occasion retailer that may begin the execution of a workflow, interprets then parses the DSL to execute workflows on the workflow engine, and depends on the Motion retailer to carry out particular duties.
  3. Platform Retailer:
  • Occasion Retailer: Pre-configured catalog of Kafka occasions which Journey Platform can take heed to and set off new executions of a workflow or go occasions to current workflow execution.
  • Motion retailer: Repository of predefined, specific-purpose capabilities permits customers to carry out varied duties, corresponding to sending emails, push notifications, or emitting Kafka occasions. Customized actions will be outlined and built-in into the software, making them accessible to all Journey Platform customers.
  • Attribute retailer: Central repository for important knowledge, corresponding to person metadata (e.g. person’s geolocation, Airbnb search historical past, and many others.) and contextual info. It helps decision-making in workflow branching processes by exposing these knowledge as a parameter to set circumstances upon by means of the parameter supervisor.
  • Customized shops: Means to create customized motion or attribute shops which aren’t already outlined within the platform.
  • Workflow Orchestrator: Mind of the system, the workflow orchestrator takes within the workflow definition DSL from the UI. As soon as a workflow is launched, it listens for occasions from the occasion retailer that may begin the execution of a workflow, interprets then parses the DSL to execute workflows on the workflow engine, and depends on the Motion retailer to carry out particular duties.
Determine 3: Manipulating and connecting nodes in a Journey Platform workflow

When crafting the UI for the workflow automation system, we aimed to create a well-known and intuitive expertise. Drawing inspiration from stream charts, productiveness instruments with “inspector panels,” and incorporating drag and drop performance, we needed a platform the place customers may begin instantly with out consulting the guide.

We additionally had a objective of utilizing progressive disclosure to incrementally allow the complete depth of the platform capabilities, whereas conserving it easy for customers who solely want a small subset of the options. Through the use of smart defaults, and transferring extra complicated options into tabs and sub-screens, our superior customers may create distinctive options, going past the pre-planned use instances.

To edit the graph, we leveraged React Flow, an open-source library. This enabled us to show the graph, in addition to present primary operations like zooming, panning, transferring, and connecting nodes. On high of this basis, we added our customized node and edge parts, together with drag and drop performance for including new nodes and an inspector panel for enhancing current ones.

Determine 4: The “node inspector” panel can present quite a lot of kind inputs relying on the kind of node.

To create the types displayed within the inspector panel, we applied a schema-based kind system. This technique offers a excessive degree of flexibility, permitting us to declaratively specify the UI for particular node enter/output fields as a part of their sort definitions. The system is in-built a type-safe method, making use of Thrift annotations and Java reflection. Based mostly on the schema info and UI-specific annotations, the interface shows the suitable kind fields, assist textual content, and validation, making certain our UI is mechanically up-to-date with the platform’s capabilities.

Area-specific language

DSL offers a excessive diploma of flexibility and customization, permitting us to outline the construction and habits of the workflow. As a substitute of getting to hardcode a workflow within the workflow engine, we as a substitute have a generic workflow outlined that may execute any DSL-based workflow. Nodes and edges make up a workflow, with nodes representing particular person actions or duties and edges defining the dependencies and relationships between them.

The nodes and edges embody all the mandatory info to outline a workflow corresponding to inputs, outputs, and parameters handed between nodes. The DSL generated by the UI is handed to the workflow orchestrator, the place the DSL parser executes it.

Determine 5: Workflow with the translated DSL
DSL illustration of a workflow

Journey Shops

The occasions, attributes, and actions shops are an integral a part of the backend, as they permit listening to occasions to begin workflow executions, filter customers, and execute duties within the journey. All these parts work collectively seamlessly to create a versatile and customizable backend that may be tailor-made to the particular wants of the platform.

Occasion Retailer

Journey Platform helps listening to completely different Kafka occasions and utilizing them to set off new executions of a workflow, or use the occasion to go indicators to a operating execution. For instance, begin a brand new execution when a visitor books a keep, go a sign to a operating execution when a person receives a push notification, and many others. Just like the motion retailer, as soon as an occasion is on-boarded, all of the groups at Airbnb can use it.

Determine 6: Begin node with occasion set off

Attribute Retailer

The attribute retailer capabilities as a central repository for fetching all crucial knowledge, corresponding to contextual knowledge, person preferences, and system info, which can be utilized to complement the workflow branching course of and enhance decision-making capabilities. These shops are supported by an information storage system that manages varied attributes or traits of entities.

Think about you’ve gotten a brand new person who simply signed up for Airbnb, and also you’re fascinated about figuring out whether or not they’ve carried out any itemizing searches on the platform. If the reply is sure, you’ll ship a personalised message based mostly on their search historical past, and if it’s no, you’ll ship a static message.

This can be a concrete instance of how the Airbnb Journey Platform leverages attributes, corresponding to “itemizing search historical past,” to reinforce the person expertise. These attributes are extracted and outlined as parameters, which can be utilized for varied functions. Every workflow execution has its personal parameter knowledge assortment, which will be accessed within the parameter supervisor. Extra details about parameters might be mentioned within the parameter supervisor.

Determine 7: Setting filter situation utilizing the attribute retailer

Motion Retailer

The motion retailer is used to execute varied duties, corresponding to sending an electronic mail or updating a database document, when a person reaches a selected level within the journey. It’s a widespread library the place every perform will be shared and reused by completely different customers of their workflow.

Determine 8: Instance Actions supported in Journey platform.

Every motion implements a standard interface, together with its metadata required for the UI schema-based types talked about above, and its habits throughout the precise workflow execution.

Interface all of the Actions should implement

Parameter Supervisor

Managing a posh workflow that includes a number of steps with various inputs and outputs is usually a difficult process, particularly if the enter and output parameters change regularly or are completely different for every person. As an example, you may want conditional branching in your workflow or customized communication content material based mostly on person search. That is the place parameterized workflows and parameter managers can show to be invaluable parts.

By specifying inputs and outputs (of attribute node / occasion node / customized node) as parameters, you may reuse them all through your complete workflow execution. A parameter supervisor is a crucial part that may retailer and handle your workflow parameters, streamlining the method of making, storing, retrieving, and modifying them.

Along with offering an environment friendly parameter administration system, a parameter supervisor additionally offers a spread of options corresponding to parameter creation, storage, retrieval, modification, versioning, entry management, and auditing. These options make sure that your workflow is executed reliably and constantly whereas additionally correctly managing and storing your parameters all through your complete workflow.

Determine 9: Including a param from the parameter library

The Workflow Orchestrator executes workflows by decoding the which means of every DSL node and performing the corresponding actions. It manages low-level capabilities corresponding to storing state, interacting with the motion retailer to carry out an motion, listening for callbacks by means of the occasion retailer, and permitting builders to focus on workflow logic fairly than technical particulars. Journey Platform makes use of Temporal because the underlying workflow engine for state upkeep and orchestration. Temporal helps orchestrate workflows by means of Temporal Workers.

DSL interpreter

Builders can incorporate customized performance corresponding to new nodes or edges to broaden platform capabilities, making it less complicated to create workflows that fulfill the platform’s and customers’ distinctive necessities. Moreover, it helps superior options like parallel execution and computerized retries and enhancing platform reliability and efficiency.

Determine 10: Workflow Orchestrator
Determine 11: Multi-tenant system with devoted processing lanes

Making certain SLA for processing several types of workflows (i.e. transactional and promotional) is crucial at scale. Transactional notifications initiated by person motion (e.g. reserving affirmation, visitor/Host messaging, and many others.) have a strict SLA and require greater precedence when in comparison with promotional notifications. To realize this, we have now applied the next at completely different components of the system:

Occasion pre-processing:

  • Pre-filter: As a substitute of passing all of the occasions on to the Workflow Handler, the occasion processor filters out occasions that don’t match the standards. e.g. solely go if the occasion sort is reservation_complete and filter out for all different reservation occasions. This tremendously reduces the QPS feeding into the system.
  • Mixture excessive QPS occasions: Occasions like searches have a excessive QPS. As a substitute of straight processing, we batch and mixture them over a time window. This reduces the QPS by a minimum of a number of orders of magnitude.

Devoted lanes:

  • Now we have devoted lanes for various classes of workflows by means of the system. The occasion listener has completely different shopper teams with built-in throttling. The workflow handler has devoted Temporal namespaces for every class and strict limits on the processing QPS, max QPS to the database, and many others.

The Journey Platform empowers non-technical and technical customers to create complicated stateful workflows by means of a easy drag and drop interface. By leveraging a generic workflow definition DSL, together with motion retailer, occasion retailer, and attribute retailer, the platform facilitates the creation of workflows that reply to real-time occasions, streamlining communication, and enhancing person experiences.

Considering working at Airbnb? Try these open roles.

Because of Balaji Kalaimani, Davis Wamola, Iris Feng, Jesse Garrison, John Bernardo, Kumar Arjunan, Michael Endelman, Priyank Singhal, Steve Krulewitz, Tej Sudha, Victoria Gryn, Xin Tu, and Zhentao Solar for his or her contributions in constructing Journey Platform.

Because of Sagar Naik and Michael Kinoti for his or her management and supporting us on this Journey.

All product names, logos, and types are property of their respective homeowners. All firm, product and repair names used on this web site are for identification functions solely. Use of those names, logos, and types doesn’t indicate endorsement.