We’re excited to announce OpenRarity, a new rarity protocol we’re building for the NFT community. In collaboration with Curio, icy.tools, OpenSea and PROOF, our goal is to provide a transparent, mathematically sound, and objective rarity calculation that is entirely open-source and reproducible by anyone.

With the explosion of new marketplaces and NFT tooling, there has been a divergence of rarity rankings across platforms which can be misleading or confusing to buyers and sellers. We believe it’s important to find a way to provide a unified, transparent, and consistent set of rarity rankings across all platforms to help build more trust in the industry through transparency.

1. Problem

As we researched the rarity landscape, we came across a number of issues:

1. We conflate the concepts of rarity (mathematical scarcity of attributes), and creator tiers or market value that assign value to attributes or items independent of scarcity. While we believe each of these are important, they should be separated.
2. Rarity rankings today are produced using closed-source code and rankings tend to be divergent across publishers (tools, marketplaces, etc). There is no single source of truth.
3. Creators are often charged for rarity rankings by tool providers which doesn’t provide a fair playing field for lower budget projects.

Consensus and transparency are fundamental to the blockchain and Web3, and so we think there’s an opportunity to create a verifiable, consistent, and mathematically-grounded methodology for producing rarity ranks that can build more trust in the space in the long-term.

This divergence confirms the inconsistency across private rarity providers, and since each provider is using a proprietary, and sometimes bespoke, closed-source methodology, we cannot verify their implementation.

Meta traits contribute to divergence

A prominent source of rank divergence is due to the incorporation of meta traits. “Meta traits” are traits that don’t exist in the “on-chain” metadata, and are created off-chain by rarity providers. For example, “Trait Count” is a synthetic trait that affects rarity ranks, and not all platforms use this. There are more complex meta traits (e.g. Deadfellaz “Rotten Count”) that make reproducing these ranks impossible without custom code, making it harder for developers to produce consistent rarity ranks.

We fully support creator-determined rankings, however, OpenRarity is currently focused on representing the purely mathematical and objective rarity determined solely by on-chain metadata.

2. Proposed Solution

We believe that creating a fully transparent, open-source, mathematically grounded rarity ranking algorithm can create more consistency and lead to a better creator and consumer experience in the long-term.

OpenRarity Principles

The core tenets of the OpenRarity methodology are:

• It must be easy for creators, consumers, and developers to understand
• It must be objective and grounded in mathematical principles (open-source, introspectable)
• It must be easy to recalculate as the dataset changes (new mints, metadata typos, mutable attributes)
• It must provide consistent rarity ranks across all publishers

Methodology

We evaluated several different platforms and collections to understand the methodologies currently being used across different providers. While several collections have some form of customization, we found the most commonly adopted rarity function to be a rarity score that is a sum of the probability of each trait, and normalized by category distribution (Trait Normalization).

The problem here is that summing probabilities is inaccurate. Summing produces the probability of a token having a Green Hat or a Blue Hat, while multiplying produces the probability of a token having a Green Hat and a Blue Hat. We believe that the rarity of any given token is rooted in its set of traits occurring together.

Surprisal Ranking Algorithm

Information content is an alternative way of expressing probabilities that is more well suited for assessing rarity. Think of it as a measure of how surprised someone would be upon discovering a trait combination.

Key advantages:

1. Probabilities of 1 (i.e. every single token has the Trait) convey no rarity and add zero information to the score.
2. As the probability approaches zero (i.e. the Trait becomes rarer), the information content continues to rise with no bound.
3. It is valid to perform linear operations (e.g. addition or arithmetic mean) on information, but not on raw probabilities.

Information content is used to solve lots of problems that involve something being unlikely (i.e. rare or scarce). This video shows how it was used to solve Wordle and also has an explanation of the equations, along with graphics to make it easier to understand. You can skip straight to the part on information theory if you’d like.

To view our implementation, please visit our Github page.

Impact on Ranks

As a group of collectors and builders ourselves, we acknowledge that introducing a new methodology can create further divergence. We also understand the significance of uniformity, transparency, and reproducibility for the long-term viability of the rarity landscape. Our goal is to provide a new, durable approach to rarity rankings that will stand the test of time. We’re still early, and we hope that OpenRarity will become the standard for creators and developers who value objective and transparent rarity calculations.

Creators and developers alike will have a choice to adopt OpenRarity — the control is fully in your hands.

Out of scope

After thorough consideration, there are a few cases that OpenRarity will not initially support that are potential future enhancements:

• Number-based traits; this will be addressed in a later iteration.
• Conditional probabilities of traits (e.g. probability of Red hat given that the color of the shirt is already known)
• Currently, Trait Counts are not planned to count towards rarity in the standard

3. Download the OpenRarity beta

The initial OpenRarity library is a python package, available now to download and try:

If you have any questions regarding the code or the algorithm, reach out to us on the GitHub discussions page or file a bug report with a description of the problem. We’re excited for you to try OpenRarity and look forward to hearing your feedback.

————

About OpenRarity

OpenRarity is an open collaboration between Curio, icy.tools (QuickNode), OpenSea, PROOF and has been vetted with creator teams representing several top-100 collections by volume. We’re excited to expand this group going forward. If you’re interested in becoming a contributor or a launch partner, drop a line on our GitHub discussions page.