Elixir: a low floor high ceiling language for your projects

2022-04-26

Elixir is a dynamic, functional language for building scalable and maintainable applications.

Elixir is successfully used in web development, embedded software, data ingestion, and multimedia processing, across a wide range of industries.

Elixir leverages the Erlang VM, known for running low-latency, distributed, and fault-tolerant systems.

You may be thinking "isn't this premature optimization? I would like to keep it simple"

If it helps you can see Elixir as the language version of Postgres, a simple technology that allows you to start quick and simple but also scales even beyond what you may need.

In fact Elixir and Postgres may be all you need for a long time, let's see why.

Skin in the game disclaimer: I've been developing and running systems in production with some of this technologies for the last 10 years, my latest product instadeq which makes heavy use of dynamic webhooks and websockets is built using Elixir.

Low Floor

Maybe it's because José Valim the creator of Elixir is a really nice person.

Or maybe because the Elixir community has a big influence from the Ruby community (José used to be a Ruby on Rails Code Team member) where they have the mantra "Matz is Nice and So Are We".

No matter the reason, it's a fact that the Elixir community is really welcoming and cares a lot for the developer experience, this reflects on the tight integration between the language and its tooling:

mix build tool
hex package manager
iex REPL

Even release management is built in the officially maintained workflow.

But the development experience doesn't stop with the Elixir team, probably inspired by Ruby on Rails, the community has created tightly integrated frameworks on top of the language.

Batteries Included

The most known one is the Phoenix Framework that allows you to "Build rich, interactive web applications quickly, with less code and fewer moving parts."

Phoenix doesn't stop at making the development of web applications easier, it also provides observability tools like Phoenix.LiveDashboard which "provides real-time performance monitoring and debugging tools for Phoenix developers"

With libraries like Ecto "A toolkit for data mapping and language integrated query" and Absinthe "The GraphQL toolkit for Elixir" you have everything you need to get started so you can focus on writing the code that's relevant to your project.

Even as your requirements grow you may find the solution to your problem already available without having to add an external service, things like in-memory key-value stores, long running requests, persistent data, background jobs and service crash recovery among others are already supported by the language and its VM keeping your architecture simple as you grow.

Read more here: You may not need Redis with Elixir

High Ceiling

As mentioned above, you can start simple and stay simple even as your requirements and load grow, but how far can it go?

Here are some articles covering the topic:

Proven Track Record

It may seem that Elixir is an unproven technology, at least a new one, but its foundations (covered later in the post) have been running in production for over 3 decades, here are some Elixir success stories:

PaaS with Elixir at Heroku
Postmates: On-demand delivery company based in San Francisco
Podium: Podium is an Interaction Management™ platform that enables 30,000+ businesses with a local presence to communicate more effectively with their customers
Supabase: Listens to changes in a PostgreSQL Database and broadcasts them over WebSockets
DNSimple: Simple, Secure Domain Management
Farmbot: Open-source CNC farming machine
Mux: real-time performance monitoring and analytics for video streaming
nextjournal: An evolving platform for computer-aided research
Ably: A realtime data stream network PaaS
Bleacher Report: Sports journalists and bloggers covering NFL, MLB, NBA, NHL, MMA, college football and basketball, NASCAR, fantasy sports and more
Brex: Software and services engineered for fast-growing companies
Cabify: A safer, ethical and innovative taxi app alternative
Discord: A VoIP, instant messaging and digital distribution platform

Check here for more Elixir Companies

No Javascript, if you are into that

There has been a movement lately to create a stack where the frontend can also be written in Elixir, the most known library for this is called LiveView, you can read the initial announcement here: Phoenix LiveView: Interactive, Real-Time Apps. No Need to Write JavaScript.

Here's a more up-to-date demo: Build a real-time Twitter clone in 15 minutes with LiveView and Phoenix 1.5.

An alternative in the space is Surface UI: A server-side rendering component library for Phoenix.

Stating typing, if you want

If you like static typing there's something called Typespecs, they are similar to typescript, since they are type annotations that can be checked by an external tool but don't stop the language from running if the annotations are wrong or incomplete.

If you would like a "real" statically typed language there's one on the platform that's growing in stability and adoption called Gleam that can be used in your Elixir projects

Path to scalability, if you need it

Let's dream big, you get hockey stick adoption and need to scale horizontally.

You can use libcluster or peerage to form static or dynamic clusters of nodes.

For specific distributed system architectures like the dynamo architecture there are libraries like Riak Core Lite (I'm one of the maintainers :)

Also: Embedded, IoT, Multimedia, Data Science, Machine Learning and more

Lately there has been a lot of activity to support Data Science and Machine Learning workflows completely inside the platform, some examples:

Livebook: Write interactive & collaborative code notebooks in Elixir
Numerical Elixir is an effort to bring Elixir to the world of numerical computing and machine learning. The foundation of this effort is a library called Nx, that brings multi-dimensional arrays (tensors) and just-in-time compilation of numerical Elixir to both CPU and GPU

For data engineering projects you can use tools like Broadway a concurrent and multi-stage data ingestion and data processing toolkit.

IoT has been an active area for a while, with mature projects like:

Nerves: the open-source platform and infrastructure you need to build, deploy, and securely manage your fleet of IoT devices at speed and scale
GRiSP: The GRiSP project makes building internet-connected hardware devices easier with Erlang & Elixir
Kry10 Secure Platform The Kry10 Secure Platform (KSP) is a breakthrough Operating System and Support Service built on the world-class seL4, Erlang, and Elixir technologies

Standing on the Shoulders of Giants

You may read that Elixir runs on top of Erlang/OTP, what is that?

Erlang is a programming language used to build massively scalable soft real-time systems with requirements on high availability. Some of its uses are in telecoms, banking, e-commerce, computer telephony and instant messaging.

Erlang's runtime system has built-in support for concurrency, distribution and fault tolerance.

OTP is a set of Erlang libraries and design principles providing middleware to develop these systems.

Some systems built with Erlang/OTP:

VerneMQ: A high-performance, distributed MQTT broker
EMQX: An Open-Source, Cloud-Native, Distributed MQTT Message Broker for IoT
RabbitMQ: The most widely deployed open source message broker
CouchDB: Seamless multi-master sync, that scales from Big Data to Mobile, with an Intuitive HTTP/JSON API and designed for Reliability
Riak KV: a distributed NoSQL key-value database with advanced local and multi-cluster replication
MongooseIM: A robust and efficient chat (or instant messaging) platform aimed at large installations
EJabberd: Robust, Scalable and Extensible Realtime Platform XMPP Server + MQTT Broker + SIP Service

Some companies using it:

Inside Erlang, The Rare Programming Language Behind WhatsApp’s Success
Klarna: Erlang powers the core of Klarna’s system serving millions of customers in Europe
AdRoll: uses Erlang within the live monitoring of their real-time bidding system. This involves live monitoring everything that could go wrong on a system receiving 500K+ bid requests per second
Flussonic: Erlang is used to capture, record and stream video at 10-20 Gbit/s: TV channels, IP cameras, webinars

Check here for more Erlang Companies

Conclusion

If you are looking for a stack that "Makes easy things easy and hard things possible" without requiring a rewrite or migration you may want to take a look at Elixir.

To get started check the Getting Started Guide, join the Elixir Forum or check some of the available books.

Instadeq Blog: No-code Data Analysis & Interactive Visualizations