Working at The Linux Foundation on Hyperledger. I’ve been blown away by the amount of interest and support the project has received since the beginning of the year. As things really start to take off, I think it’s important to take a step back to reflect and recapitulate why and what we’re doing with Hyperledger. Simply put, we see Hyperledger as an “umbrella” for software developer communities building open source blockchain and related technologies. In this blog post, I’m going to try to define what we mean by “umbrella,” that is, the rationale behind it and how we expect that model to work towards building a neutral, foundational community.

The Hyperledger Project was initially seeded with various blockchain-supporting commercial members, some of whom had interesting internal or nascent open source efforts that needed the kind of home that the Linux Foundation could provide. It emerged at a time when it was clear that three points needed to be made to the market:

1. Open, transparent governance of the software development process for blockchain technologies matters
2. Intellectual property provenance and safeguards of the software matters
3. Key use cases are driving permissioned or “consortium” chain models

Out of that set of initial principles, the Technical Steering Committee determined that the two initial projects should be delivered into the project in an “incubation” state: Fabric and Sawtooth Lake. These two projects differ significantly in many ways, and thus could perhaps provide different, yet also sometimes overlapping purposes.  In the spirit of putting all the wood behind the tip of the arrow, attention and efforts began to focus on those two.

While still in a pre-alpha state, Fabric is attracting growing developer attention and significant mindshare around its unique approach. We are seeing additional interest in new projects that build directly upon Fabric, but exist separately and may have different release schedules and priorities. For example, the chaincode management tools, or the Hyperledger Explorer (initially). We would expect to see further exploration of work that has already been demonstrated in concept, such as hooking up the Ethereum virtual machine to Fabric. And Sawtooth Lake’s unique approach to consensus – Proof of Elapsed Time – is garnering attention, too.  Both Fabric and STL just cut another developer preview release.

As both Fabric and Sawtooth Lake have picked up new contributions and developer momentum, and the commercial interest has continued to grow, we are now seeing interest in applying this model to additional technology efforts, potentially leading to new projects at Hyperledger. In some cases those will be a useful spin-out of an existing project and community; other times they will bring in a new community of developers, both on the new project and spilling over onto existing projects.

The Rationale: Why an “Umbrella”?

Blockchain and smart contracts are still in the early stages of a 20-year, if not a 50-year, adoption and maturation cycle. Some have compared it to 1994 and the Web (MIT’s Joi Ito sees it as 1989). There are clear examples of efforts that have seen widespread adoption and scale: Bitcoin and Ethereum. There are clear examples of commercial blockchain stacks, running in production. There is clear momentum around Fabric and Sawtooth Lake. Yet by no means is this a mature industry – we are still seeking better consensus mechanisms for both permissioned and permissionless chains, a better range of choices for smart contract platforms, and still exploring the right identity models. We have no idea if one specific package of software can serve all these needs at the same time, or if the approaches are as divergent as, say, ACID-compliant SQL databases and eventually-consistent NoSQL databases. Furthermore, some needs may span multiple existing efforts – a graphical user interface, for example, that could just as easily span Fabric and Sawtooth Lake.

What we do know is that there are no software development resources to spare. There is a global talent shortage for developers who understand not only cryptocurrency and blockchain engineering challenges, but who also understand distributed systems. The guts of these platforms are not unlike the complex balancing acts that operating system kernels or hardcore database efforts can reflect. Debugging multi-threaded applications was a challenge when all we knew were single-threaded applications; now debugging distributed applications is that much harder. Software that wouldn’t benefit from a rewrite doesn’t exist (or fresh thinking on old problems). Given the amount of duplication of effort we see today on the same core functions, we need to constantly be looking for opportunities for developers to be working on common code and roadmaps whenever possible. We’re not only building consensus mechanisms, we need to live them as a development priority.

In this environment, the most valuable role the Hyperledger Project can play is to serve as a trusted source of innovative, quality-driven open source software development community, creating modular, open source components and platforms. The optimal focus of Hyperledger is to advance industry goals of distributed ledger and smart contracts. Hyperledger will forge a brand that will be seen widely to reflect the accepted default “safe” deployment platform for enterprise teams, and be seen as a great home for active collaboration around new technologies, only then our mission will be accomplished.

Another way to think of this is to consider Hyperledger’s potential role in the emerging landscape of public blockchain technologies. Most of today’s open source blockchain efforts outside of Hyperledger are focused on permissionless chains, necessarily implementing a cryptocurrency as a means to fund mining and participation in consensus. This has tremendous challenges, and not all of them are technical, as the debates over the Bitcoin blocksize or the DAO demonstrate. What may appear to be technical debates at first glance often are really about different visions for the roles these platforms should play in society and who should govern them. We’ve crossed this bridge before, though. The Domain Name System was fortunate enough to rise to ubiquity long before anyone outside the early Internet architects realized how important it was – how important having a widespread network of root nameservers all consistently serving up the same answers for domain names would be. To get there, we might have started with a few developers building a better replacement for “/etc/hosts”, but which eventually evolved into a three-part structure: standards bodies (IETF), implementers (sendmail, postfix, qmail, etc), and global governance (ICANN).

Mapping this to today, there is no reason why a particular cryptocurrency needs an entirely novel technology stack. The scaling needs of a particular cryptocurrency has tended to drive the technology roadmap for a given stack, but the code behind ETH and ETC (Ethereum and Ethereum “Classic” tokens) is almost entirely the same. The configuration settings, mining community management, trademark questions, and regulatory agency and law enforcement relationships will likely differ, and are really a matter for the global governance of ETC and ETH as cryptocurrencies.

“The most valuable role the Hyperledger Project can play is to serve as a trusted source of innovative, quality-driven open source software development community; creating modular, open source components and platforms; all focused on distributed ledger and smart contract technologies. If Hyperledger can forge a brand that is seen widely seen as the accepted default ‘safe’ deployment platform for enterprise teams, and be seen as a great home for active collaboration around new technologies, then I think we can say ‘mission accomplished’.”

If Hyperledger could help not only forge common ground between different software development efforts, but also encourage a gradual detachment between standards, implementations, and global governance (whether that’s around currencies or other use cases), then we will also accelerate adoption of blockchain tech widely and further reduce needlessly duplicated engineering and hardening efforts.

Perhaps most importantly, we can directly address what many have observed as a major challenge with the existing open source blockchain efforts – tremendous levels of tribalism amongst developers. While invigorating, it can also make sharing code between efforts, or talking about common challenges and how to meet them, notoriously difficult. This is true even when the payoff would be less duplicated code and more eyes looking for security holes and other issues.  Multiply that rivalry with the effects of holding fungible currency whose value can be tied directly to the software in question, or open source project brands tightly associated with commercial brands in which developers own equity, and incompatible copyright license paradigms, and working together can be nearly impossible.

At Hyperledger we believe we can provide an answer to this.  Let’s bring these different implementation efforts within the same “home”, with a consistent approach to intellectual property, community collaboration standards, overall branding (“Hyperledger ____”) and an encouragement to either work together or usefully differentiate.  If we do this, it will remove barriers to collaboration, encourage developers to find opportunities to work on common code, and address the potential for confusion and wasted duplication of efforts without requiring a top-down single architecture or personality to dominate.

The Model

What does being a project under the Hyperledger banner actually mean? Consider the Apache Software Foundation. Within the ASF, after nearly 20 years of existence as a community of communities, there are nearly 300 different “top-level” software projects. Each has its own charter, developer community, roadmap, development process, etc. They mostly use the same collaboration tools, the same IP framework (the Apache license, and contributor license agreements), they use the same reporting process to the Board of Directors, and they must demonstrate the same sort of community-driven development as established and evaluated by the Apache Incubator. There are several competing projects at the ASF; this is not a problem, as projects that fail to get or sustain a critical level of participation and responsiveness to the needs of its users can simply be retired by the Board when it’s clear they’ve lost it.

There are many parts of this we would not copy verbatim, but it’s a model with proven success at turning new efforts into home runs (see Hadoop, Spark, and countless others), and a clear template understood and trusted by a broad community. In our case we can be a bit different. The Linux Foundation can provide many services that Apache depends upon volunteers to provide, from project coordination and developer collaboration tooling, to hosting and conducting virtual and physical meetings, to worrying about developer contributor agreements and trademarks and other legal issues, and more.

With this as inspiration, and through discussions at the Technical Steering Committee, we’ve come to define a  Hyperledger project as consisting of:

1. An identified set of software developer “maintainers” who are responsible for the development process, culture, and general technical direction of the project, and engaging the public. The developers can vote in new members and people may retire, but any active project needs at least a few who maintain a heartbeat of activity, if not more.
2. A bounded set of artifacts, including one or more Git repositories, a bug tracking/issue database, a wiki, a set of mailing lists, and other developer resources (e.g. forum, mailing lists, IRC/Slack channels, etc) tied together as part of the same particular project, and which the maintainers directly and the broader community indirectly are responsible for keeping updated and active.
3. Dedicated space within Hyperledger to describe the project and the community, with a clear delineation on the relationship and differences with other efforts at Hyperledger (or elsewhere), and an indication that each is equally important to the overall Hyperledger effort.

That’s it. From that, all else can be derived. Ultimately, we want to increase the impact of Hyperledger across the open source blockchain landscape. The increased level of traffic to our community, and the ability to serve many different points across the map, can only enhance our ability to serve the broader blockchain community.