This post talks about some useful patterns for dataflow architecture in NoFlo web applications. We’re using these concepts to build Flowhub, the flow-based programming IDE.

Flux is an application architecture for web applications published by Facebook back in 2014. It uses a unidirectional data flow heavily inspired by flow-based programming concepts — events are sent from views to a dispatcher, which directs them to the appropriate data stores. The stores modify application state based on these events, and send updated state back to the view.

This structure allows us to reason easily about our application in a way that is reminiscent of functional reactive programming, or more specifically data-flow programming or flow-based programming, where data flows through the application in a single direction — there are no two-way bindings. Application state is maintained only in the stores, allowing the different parts of the application to remain highly decoupled. Where dependencies do occur between stores, they are kept in a strict hierarchy, with synchronous updates managed by the dispatcher.

Given its nature, the Flux pattern is quite easy to implement in NoFlo. Here is an example of a simple web-based TODO list using a Flux-esque NoFlo graph communicating with a React component:

In the image above you can see the graph in the middle, with the rendered React application on the right, and on the left an edge inspector showing the packets flowing from the view to the dispatcher.

In this example we decided to use bracket IPs to convey the action type. This allows any payload to be sent as the action, and usage of a standard NoFlo router component for packet dispatching.

Problems with Flux in NoFlo

We’ve been following a very similar Flux-like pattern as in the example above also in Flowhub, a flow-based programming IDE implemented in NoFlo. Over time the flows started becoming messy because:

• Different stores would need access to different parts of application state
• Some stores needed to generate their own actions
• Some actions would need to pass through multiple stores

Since the only way to transmit information between components in NoFlo is to sent them as packets along a connection, these interdependencies cause a lot of wiring back and forth. Visual spaghetti code!

To find a better approach, I sat down couple of months ago with Moritz, a former colleague who has done quite a bit of work with both Flux and Redux. He suggested looking at Redux middleware as a pattern to follow.

Introducing middleware

Redux is a recent refinement on the Flux pattern that has become quite popular. One of the concepts it adds on top of Flux is middleware, something that is more common in server-side programming frameworks like Express:

Redux middleware solves different problems than Express or Koa middleware, but in a conceptually similar way. It provides a third-party extension point between dispatching an action, and the moment it reaches the reducer. People use Redux middleware for logging, crash reporting, talking to an asynchronous API, routing, and more.

Middleware can be chained so that all actions pass through each of them. A middleware that receives an action can either pass it on (maybe logging it on the way), or capture it and send out new actions instead.

Here is how a middleware looks like as a NoFlo component:

Actions arrive at the in port. If the middleware passes them along, it will send them via the pass port, and if it instead generates new actions, these will be sent via the new port.

With this structure, chaining becomes very simple:

The image above is from the main graph of Flowhub. In Flowhub we have both very simple middleware, like the logger that just writes the event details to the developer console, and more complex ones like the UserMiddleware that deals with user information and OAuth, or RuntimeMiddleware that handles communications with FBP runtimes.

The middleware themselves can be generator components, listening for external events and creating new actions based on them. This way for example the UrlMiddleware generates new actions when the application URL changes, allowing the other middleware then load the appropriate content for that particular screen.

Actions and state

In the Flowhub graph, the actions are sent as NoFlo information packets from the view to the middleware chain. The packet flow of an action looks like the following:

< github
< pull
{
  "payload": {
    "repo": "noflo/noflo-ui"
  },
  "state": {
    // The application state when the action was triggered
  }
}
>
>

The brackets surrounding the action payload tell the action type, in this case github:pull. The data packet contains both the actual action payload, and the application state as it was when the action was triggered.

Including the state object means that each middleware can access the parts of the application state is in interested in while dealing with the action, removing interdependencies between them.

It also means that middleware become super easy to test, as you can send any kind of state/action combinations to exercise different flow paths.

Current status

I’ve started introducing middleware quite carefully to the Flowhub code base, so right now we’re running a mix of old-style Fluxified stores and new-style middleware/reducer combinations. The idea is to migrate different parts of the flow to the new pattern subgraph-by-subgraph as we fix bugs and add features.

So far this pattern has felt quite comfortable to work with. It makes testing easier, and fits generally well in how NoFlo does FBP.

If you’d like to try building something with Redux-style middleware and NoFlo, it is a good idea to take a peek at the middleware graphs in the Flowhub repo. And if you have questions or comments, get in touch!