Parker Selbert is the creator of Oban, a fully-featured background job processing library for Elixir. In the grand scheme of things, Oban is a relatively new library, with its first commit in January 2019. Development progressed quickly however, and in October of the same year, Oban Web was released under a paid license. Oban Pro followed in June 2020, also under a paid license. Nowadays more often than not, where you find Elixir, there you also find Oban.
I reached out to Parker recently about doing an interview for Code Code Ship, and he graciously accepted. The full interview is reproduced for your reading pleasure below.
Parker, thanks for allowing me to interview you. You started working on Oban in early 2019. What did your situation look like then? Were you still working a full time job? Did you take time off to write Oban?
“Oban was written on nights and weekends”
I was working full-time as a staff engineer at dscout, a company I had contracted with for the previous nine years. Dscout started with a Rails application, and by 2015 we had started porting everything to Elixir. The port started with a GraphQL API, and it was quickly apparent we had to integrate with our legacy background job system, which was running Sidekiq. Oban was written on nights and weekends and released into the open for months before I introduced it to dscout.
How did you arrive at the idea to write Oban? Did you consider doing any other libraries before landing on a “background job processor”?
Oban started as an experimental alternative to Kiq, an Elixir Sidekiq port I had started a year or two prior. The requirements all emerged from pain points running Redis-backed queues—a lack of introspection, zero observable history, and no transactional guarantees. More importantly, we were locked into working with Sidekiq Enterprise for some features that the license precluded me from porting to Elixir. Oban wasn't my first OSS or Elixir library, but it had some novelty that hinted at it being marketable.
Was it your plan from the start to commercialize Oban?
“there were hundreds of beta testers right away”
Somewhat. There was a loose idea there. My wife, who is also my business partner, and I worked out a business plan. Considering the amount of work it had taken to build Kiq previously, and the effort that went into Oban, some level of monetization felt crucial. Selling expertise through a product would scale much more practically than selling time as a consultancy.
Do you recall when you "knew" that Oban would be a success for you, commercially? Thinking back to the first year, how did Oban Web + Pro perform, compared to your expectations?
That moment came when we initially offered Web as a free beta, and there were hundreds of beta testers right away. Of course, most free beta users won't convert to paying customers, but it proved that there was a market there. There weren't any other paid products in the Elixir space to compare against at that point, so any amount of success would have been a pleasant surprise. Considering the size of the Elixir community and the number of well-known Elixir-powered businesses, the success of Oban Web and later Pro has exceeded expectations.
“building a business with dual-licensing is fuzzier”
Oban follows the “open core” model, meaning that the core of the library is maintained as FOSS software, and you charge money for closed-source add-ons (Oban Web + Pro). What were the factors that led you to adopt this model vs a dual-license construction?
An open-core model makes the transaction of goods for money explicit. Whereas building a business with dual-licensing is fuzzier. It requires a large enough pool of potential customers that the business that refuses to run GPL will cover the majority that isn't concerned about licenses.
In your opinion, do you think the open core model, as an innovation, has made dual-licensing obsolete?
Perhaps not obsolete, but it shifts the barrier from obvious "enterprise" size companies to smaller and midsize businesses. There are ample early-stage startups and midsize businesses that would benefit from the features of something like Oban Pro, with little chance that they'd be motivated to pay for a commercial license without an obvious upside.
Are you saying that license enforcement is an issue with dual licensing; that many who should be paying simply do not? And this affects mostly smaller and medium sized businesses? So Open Core helps you tap into the small and medium sized business market?
Yes, that's my understanding of the challenge with dual licensing. There's no way to monitor who is using your software or how broadly they're deploying it. That means you can't reach out to nudge users toward a license sale, and there's little incentive to pay for a license that only lets them check a legal box.
Open Core changes the entire dynamic. Now there are concrete features that are useful to a company of any size, not just established companies with semantic legal concerns.
One question developers have when considering open core is: how do you do feature segmentation? Did you find this challenging? How did you solve this with Oban Web + Pro vs Oban?
There's a tangled history to Oban's feature segmentation. Originally, there was only Oban and Web; Pro didn't exist yet. That meant all features went directly into Web, even when they were clearly more "valuable" for mature applications. Eventually, we decided another product was necessary for Oban to grow as a business, hence Pro.
“There are tremendous opportunities for commercial Elixir libraries.”
How did you come to the realization that Web should be split into Web + Pro? What was the tipping point? Was there a particular feature, or just an increasing realization that there were two distinct feature sets with two distinct audiences, being represented in Web? Was size a factor leading to the decision to split Web into Web + Pro?
It came down to the realization that the type of advanced features people requested for Oban had no place in a Web package and may be valuable enough to warrant a dedicated product. The tipping point was a request for Batches, the ability to run a group of jobs and trigger follow-up jobs based on progress. Oban provided the underlying tools to build batches in your own application, but doing it in an accurate way that would scale was valuable enough to be a selling point for a private package.
The initial Pro release was minimal, with a couple of plugins and batches. At first, we bundled Web + Pro into a single license, but we knew that eventually, Pro would become featureful enough to spin out into a dedicated product where we could sell each package separately and tier the licenses.
How do you decide whether a new feature will be included in the open core, vs in Web + Pro?
With Web being a LiveView-powered dashboard, there aren't many decisions to make—if it's needed for the UI, then it goes in Web.
The decision-making is all between OSS and Pro. Our heuristic is pretty simple: anything necessary for Oban's core goals of reliability, consistency, and observability goes in OSS. That includes functionality like ensuring jobs run smoothly regardless of database issues, rescuing jobs "orphaned" during a shutdown, instrumentation hooks, and node leadership.
On the other hand, advanced functionality that isn't required for reliable operation is a candidate for Pro. Features like the aforementioned batches, workflows, global rate limiting, and batch unique inserts are more valuable to more mature, or complex, applications.
How has the open core of Oban worked out in terms of its advertising power? Have you needed to actively market Oban outside of the open core offering? Has this changed over time?
Having an open-core library was essential to Oban gaining traction as a tool in the community. That adoption and widespread familiarity with Oban as a recognized solution have provided tremendous advertising power. The challenge has been communicating which features are missing from OSS but available in Pro. That's where actively marketing through conference talks, the "Oban Tips" series on Twitter, and other teasers about Pro features have helped. Web kind of markets itself—most people want a UI to manage their jobs, and there's a live demo you can poke at.
Now the rationale for OSS vs Pro comes down to whether something is required for Oban to function or possible to build on your own. There's no "monkey patching" in Elixir, so Pro only builds on what Oban exposes.
“building and maintaining a library takes a tremendous time investment”
FOSS alternatives to Oban Web + Pro have not materialized. Why do you think that is?
Most developers I interact with are supportive of Oban and want to see it succeed. But, more importantly, building and maintaining a library takes a tremendous time investment—especially a library as complex as Web or Pro. Unless it's a passion or you have a vested commercial interest, there's too much startup inertia.
“With a platform like Code Code Ship available, there's virtually no barrier to entry.”
If you look around the Elixir ecosystem, there are not a lot of commercial libraries. Do you see opportunities for more commercial libraries? Would you like to see more developers creating commercial libraries?
A few years ago, when we started selling Oban Web, half-a-dozen developers reached out for guidance on commercializing their own libraries. Sadly, none of those materialized. There are tremendous opportunities for commercial Elixir libraries. Commercialization doesn't have to mean a fully-fledged business, just enough to ease the burden of open-source maintenance. With a platform like Code Code Ship available, there's virtually no barrier to entry.
Would you have used Code Code Ship if it had been available in January 2019?
Absolutely! Building out the tooling to take payment, manage subscriptions, host packages, and serve documentation took a great deal of time away from working on Web or Pro.
Finally, can you tell us a little about what the future holds for Oban? Is Oban 3.0 on the horizon?
Oban 2.0 happened purely because of breaking changes from the Pro split. There's an Oban 3.0 "breaking changes" issue on GitHub, but we're not actively working toward that. Some features have been in the backlog for three years, and we're finally getting around to them. We have enough Web and Pro features and performance improvements to keep us busy for years to come!
