In a previous article, we introduced Regen Score, an on-chain reputation system designed to quantify regenerative behaviors—actions that contribute positively to social or ecological systems. At the heart of this innovation is the Regen Score formula that assigns numerical values to a range of tracked metrics. This product and its formula were initially conceived to address the scalability challenges faced by the Optimism (OP) Citizen House, particularly in the distribution of RetroPGF, as outlined in this forum post.

Let’s assess the efficacy of the Regen Score formula. Our goal is to validate whether the formula is behaving as intended by examining three distinct user groups across the "regenerative spectrum." This will enable us to understand how the scores vary among these groups and make informed decisions about the formula's future applications.

How We Did It

While Regen Score is still in its nascent stage, we continue to experiment with different solutions before ultimately deciding whether our current formula or a different scoring model will be used. For this preliminary assessment, we scrutinized the on-chain activities of three distinct groups: Optimism Delegates (62), Trusted Seed Members (361), and a random selection of recent OP users (130), as identified by their transaction history on the Optimism block explorer.

We gathered data for each of the 551 addresses involved in the study, recorded their attributes and calculated their final scores. This data was then meticulously analyzed to understand how different samples compare on individual metrics and cumulatively.

Our Takeaways

The first thing we did to assess if our assumptions were correct was to create a histogram of the scores for each group.

Score distribution. The average score for an Optimism Delegate was notably higher than that of a Trusted Seed member, who in turn scored higher than random OP users.

In our analysis, we employed histograms to visualize the score distributions across the three distinct groups. The horizontal x-axis represents the scores, while the vertical y-axis indicates the frequency of these scores. On top of each histogram bar, we've annotated the percentage that each bar contributes to the total. Additionally, in the top left corner, we've included both the mean score and standard deviation for each sample population.

As observed, the average scores of the three groups have the ordinal ordering we’d expect. Optimism Delegates, who are deeply invested in the Optimism ecosystem, got the highest mean score, approximately 1250. Following them are Trusted Seed Members, who are active contributors in the regenerative web3 space, with an average score of around 870. Lastly, random OP users, primarily utilizing the blockchain's native transfer functions, have an average score of about 450.

The score distributions for all groups are right-skewed, indicating the presence of outliers with notably high Regen scores within the Trusted Seed and Delegate groups. A noteworthy observation is that the minimum scores are consistently higher for the Delegate group compared to the Trusted Seed, and likewise, the Trusted Seed's minimum scores surpass those of the randomly selected OP users.

These findings strongly suggest that Delegates exhibit more regenerative behaviors than Trusted Seed members, both of whom outperform random OP users in this regard. A statistical test comparing the mean scores between these groups would yield statistically significant results at all conventional alpha levels.

Behavioral insights. Both Optimism Delegates and Trusted Seed members exhibit certain behaviors that are notably absent among random OP users.

To gain a more nuanced understanding of each group's "Regenerative Profiles," we delved into the individual "seed behavior" averages. This exercise confirmed the formula's ability to capture regenerative behavior and also identified areas for improvement.

In our data table above, the first column lists the specific "Seed Behaviors," followed by columns representing the average scores for each of our focus groups.

At a macro level, the Optimism Delegates group most frequently recorded the highest averages, leading in 7 out of the 18 seed behaviors examined. The Trusted Seed Members was not far behind, topping the averages in 6 behaviors. In contrast, recent OP Users led in only 2 behaviors. Intriguingly, each group exhibited a unique regenerative profile. Optimism Delegates demonstrated a multifaceted ways to “seed” the ecosystem, Trusted Seed Members were predominantly inclined towards giving, and Recent OP Users were mainly active in utilizing the Optimism blockchain.

Potential for positive impact. Trusted Seed members have a higher propensity to become impactful citizens within the Optimism network.

However, it is worth noting that further verification is needed to fully comprehend the implications of integrating such semi-external groups.

Optimism demonstrates a strong commitment to public goods through its RetroPGF program, which distributes millions of dollars in value each season. This program attracts a considerable number of experts and activists who can contribute to better decision-making. Data from platforms like Gitcoin, Giveth, and Trusted Seed can help pinpoint individuals who possess both the skills and dedication to contribute meaningfully to Optimism's initiatives.

However, as with any system, there's a risk of manipulation or “gaming the system,” making it a challenge to effectively engage these individuals and accurately track their impact. Despite these hurdles, reputation systems like Regen Score offer valuable services to broader regenerative and public goods networks, enabling more efficient data exchange and collaboration.

While our findings affirm that the Regen Score formula is a reliable indicator of regenerative behavior, they also highlight areas for improvement. As we continue to refine our scoring system, we remain committed to fostering a more efficient and collaborative environment for regenerative and public goods networks.

We invite you to explore your on-chain impact and share your feedback with us at https://regenscore.io/.