The New Web: Massive Concurrency with Elixir, Phoenix Channels, and Redux

‍

In Engineering Terms, Where We Are and Where We Want To Go

‍
Dropbox Sign been able to build a successful business based on a simple set of technologies: PHP, Symfony, and jQuery. To this day, these technologies serve as core parts of our stack and work quite well. We do not see us ever completely removing these parts of our codebase.
 

‍

Yet we've had some issues in terms of maintainability, performance, and productivity, especially with jQuery. Maintaining complicated imperative jQuery code can be something of a nightmare. Plus there were parts of our architecture that were not giving us the performance our customers needed. This led us to a search for some long-term solutions to these tricky problems. 

‍

Ultimately, we embraced a few approaches to our problems as we started to scale.

For greenfield projects, we settled on a new tech stack consisting of Elixir, Phoenix, React, and Redux. But the tech stack is really an outcome of a shift in philosophy at Dropbox Sign toward functional programming and immutability, and event driven architectures. 

‍

This new approach allowed us to build a massively concurrent, fault-tolerant, real-time application at scale.

‍

Why Elixir?

‍

From Wikipedia: "Elixir is a functional, concurrent, general-purpose programming language that runs on the Erlang virtual machine. Elixir builds on top of Erlang and shares the same abstractions for building distributed, fault-tolerant applications."

So why did we choose Elixir? 

‍

First of all, because it's built on top of the Erlang VM, we get a lot of the benefits of Erlang. Originally created by Ericsson, Erlang was specifically created for telephony systems. That means it needed to have certain features: high availability, massive concurrency, and fault tolerance. Think about it – when was the last time a landline phone didn't work? 

‍

Elixir gives you a few added benefits Erlang does not.

For one, the syntax is a lot nicer. Originally created by core contributor to Rails, Elixir's syntax was intentionally designed to resemble Ruby. Elixir took with it the friendliness and inclusivity of the Ruby community. The tooling is also a lot better.

But Elixir has another benefit: a really great web framework similar to Ruby on Rails but written in Elixir called Phoenix. 

‍

The Phoenix Web Framework

‍

In many ways, Phoenix was inspired by Ruby on Rails, yet avoids a lot of mistakes and magic that Rails is known for. Best of all, it's fast. Very fast. Performance in Phoenix is not measured in milliseconds – it's measured in microseconds. 

‍

Unlike Rails, Phoenix has a powerful real-time story (Action Cable really falls short here) through what it calls Channels. It's just what it sounds like: a way to send and receive messages. Processes can send messages about topics and receivers can subscribe to topics so they can get those messages and handle them accordingly. 

‍

While Elixir is based on message passing, Channels allow senders and receivers of messages to be anything, not just Elixir processes. Phoenix even provides an excellent library for JavaScript clients to easily connect to a channel and send/receive messages on that channel. 

‍

 Valid Phoenix Channel messages require the following keys:

• topic - The string topic or topic:subtopic pair namespace, for example "messages", "messages:123"
• event - The string event name, for example "phx_join"
• payload - The message payload
• ref - The unique string ref

If you're familiar with Flux, this might seem awfully similar to Flux Standard Actions.

‍

The Front-End Architecture

‍

1. React for Views

‍

React is a popular JavaScript view library for building UIs. It's used at a number of companies, including

‍

2. Redux for State Management

‍

One issue we were finding with pure React applications is a great way to separate concerns: How do we separate view logic from application logic? How do we manage application state? We settled on Redux, which is a simple state container designed for React-like applications. 

‍

3. Everything Happens Through Flux Standard Actions / Messages 

‍

One benefit of Redux is that state is updated via Flux Standard Actions and a unidirectional data flow, which as mentioned earlier look a lot like Phoenix Channel messages. Since Redux state is immutable and is updated based on actions, given some initial state you can rewind all the actions to get to the exact state at any given point. 

‍

This means if we have all the user's actions and they leave the app at any point, when they return we can replay all the actions and get to the exact same state as before. 

‍

4. The Saga Pattern and ES6 Generators for "Concurrency" 

‍

It is true that because JavaScript is single-threaded, you don't have true concurrency. But with ES6 Generators, you can essentially emulate concurrency. Generator functions are a lot like normal JavaScript functions except that they can halt execution part-way through the function call.

In normal functions, execution of the function never stops. 

‍

But in a generator function, execution of the function will halt on `yield` statements.

‍

‍

This means we can yield a time consuming / asynchronous call and continue execution once that call is finished. However, how do we manage all these concurrent operations in a way that fits our architecture? 

‍

‍

We decided to adopt the Saga pattern for our JavaScript client code. Originally proposed in a paper by Hector Garcia-Molina and Kenneth Salem, the Saga pattern proposes an alternative mechanism (called sagas) for managing long running distributed transactions. In essence, it's a failure management pattern. There are strategies around how to handle failures, whether to roll back to a prior state or fail forward, depending on the situation. In many ways, this fits with Elixir's "Let it Fail" approach to concurrency and fault-tolerance. 

‍

We use Redux Saga to tap into the Redux "channel" or "stream" of actions, and with ES6 generators we can create de-facto daemon processes that respond to particular actions. We can also spin up sagas that open up a Phoenix channel and dispatch messages to that channel or buffer messages from the channel and dispatch them to Redux store.

‍

Conclusion

‍

These approaches have been a huge success at Dropbox Sign. Phoenix Channels, Websockets, and Redux work so well together because they’re based around the same ideas: functional programming and event-driven architectures. This makes the whole greater than the sum of its parts. 

‍

By now, we have an architecture that we feel can grow and scale with Dropbox Sign for years to come. Elixir, Phoenix, React, and Redux are mature enough to productively build the future of web applications.

‍

Like This Post? You Might Enjoy These Too

‍

• How to Make Your React Apps 10x Faster
• Getting React and Webpack to Run on IE8 (If You Must)
• 10 Mistakes to Avoid When Building an API

‍

Or poke around Dropbox Sign's documentation.