-
Watching the tide roll in
-
How do we value cultural objects?
-
The wisdom and fallacies of the crowd
-
Examining the experts
-
The double edged sword of personal networks & intuition
-
Building better systems of value
-
*Shangri-La: curatorial governance matters
Watching the tide roll in
On December 25, a global icon was born.
At first subjected to a barrage of criticism and condemnation by many, in time this messianic figure rose above initial scorn to be embraced by a group of ardent supporters all across the world who could often recite his every word.
Individual minds were changed over time, as some of his initial critics became converts who went from publicly ridiculing him to being publicly enraptured by him.
Many worship at the altar of Playboi Carti and his sophomore album Whole Lotta Red. As infectious as Carti’s repetitive hooks and ad-libs may be, WLR has stuck in my brain for reasons that have little to do with how good his music is.
❖
WLR is a fascinating example of public hate turning into public love, and prompts a lot of thought about the nature of curation, criticism, and context in shaping how we collectively value certain cultural objects.
On Christmas Day 2020 when WLR dropped, the album was pilloried on social media by Carti fans and non-fans alike.
Though the constantly shifting nature of social media generally makes it tough to observe what past consensus opinion or digital zeitgeist looks like, in this particular instance we are lucky that there is documentation of the loud panning of the album.
In the midst of the overwhelming sea of negativity, there was a tiny group of diehard fans who saw the potential of the album, but those voices were largely drowned out by the avalanche of jokes being made at Carti’s expense on social media.
Over the next four or five days, as people began to listen to the album more and more, those voices of approval slowly grew, to the point where public opinion moved from largely negative to slightly mixed.
After repeated listens, a growing number of fans began to realize that some of Carti’s abrasive choices – like extreme repetition in the lyrics, an unmixed/unfinished feeling in the production, the exclusion of many previously leaked fan favorites from the final tracklist, and working with a new batch of producers – were all transfixing after multiple listens.
❖
Ultimately what caused a seismic shift in the consensus opinion of WLR more than anything else was the Pitchfork review that crowned the album Best New Music two weeks later. More than any other publication, for the past twenty years Pitchfork has been the go-to source for top-notch music criticism, and its alumni are myriad in the field of cultural analysis.
After Paul A. Thompson’s important and glowing review of the album, fans began to realize that the initial social media reception for a work is not always equivalent to a work’s longer-term value, and began to come around to the idea that WLR would have lasting impact.
In the linked thread where fans discuss the Pitchfork review on the Playboi Carti subreddit, it’s fascinating to watch them — in real time — negotiate the value of criticism vs. larger audience opinion when assigning works value, and what value even means.
Seeing Carti play Rolling Loud NY and LA at the end of 2021, and 40 dates on his King Vamp/Narcissist arena tour from Salt Lake City to Hartford, Connecticut cemented the album’s iconic status, as the stadium guitars and distorted bass throughout the album played perfectly in those venues. Even non-fans of Carti have been forced to acknowledge the work’s merits, as his performances forced casual music observers to pay attention.
Now more than a year later, crowds and critics are aligned in deeming WLR a classic album. They could not have been further apart one year prior.
How do we value cultural objects?
Recently I’ve been asking myself questions similar to those asked by fans in the Playboi Carti subreddit, and thinking about the role of criticism versus larger audience opinions when assigning works value, and what value even means.
Though WLR prompted me to ask these questions recently, I’ve been thinking about these questions for a while as a music obsessive, as music is the most commonly reviewed – 2021’s most popular album has 5x the number of user reviews as the year’s most popular TV show for example – and opined upon art form due to its universality and ease of consumption. I’ve been fortunate to think a little bit about these questions professionally as well during my time at Spotify.
These questions about how we value cultural objects – and how we value information – are essential for understanding the media landscape today, and also for dissecting one’s own values and biases.
These questions are particularly crucial in the era of digital art, where the barriers to entry for media creation have been dramatically lowered, and have resulted in a mass proliferation of cultural objects. On the Internet, this happened earliest with non-fiction writing and music, and now this mass proliferation is coming into the spotlight in visual art as well in the web3 era, though DeviantArt and Behance paved the way before NFTs.
In an age of media overabundance, how we filter information and media is incredibly important for how we actively construct our worldview, as well as for how our worldview is built for us passively. Both the active and passive construction of our worldviews affect how we relate to each other as humans.
❖
The question that seems central to all other questions is:
When determining whether a cultural object has value, is it best to trust:
-
The wisdom of the crowd
-
Experts
-
One’s network or
-
One’s intuition
And, how does one define trusting each of these sources of truth?
The pragmatic next question that follows is: how does one synthesize inputs from each source to get the best possible output when it comes to properly valuing a cultural object?
Different disciplines give different weights to each one of these four sources of information. In fashion, most listen to their intuitions, while a small set of others may read Vanessa Friedman or Tim Blanks. When choosing restaurants, some consult Yelp, while others consult the Infatuation, and others go with what Pete Wells recommends. In fine art, nearly every head curator has some a degree in art history or an MFA, as there is a greater emphasis placed on academic expertise in that field.
This question of which sources of information to trust is a challenge that all media and cultural object distribution companies – in particular social media companies with vast pools of cultural objects to pull from – have to deal with.
As web3 rightfully peels back the curtain on the processes and systems that control our digital universe, curatorial governance feels like a topic that will shape the next decade of digital life. Curatorial governance refers to the formalized set of processes, rules, and principles that are used to assign different amounts of value to different cultural objects.
Curatorial governance is paramount in the web3 world where groups of humans like DAOs and communities are often assigning value directly to cultural objects, yet the definition of curatorial governance also extends to the algorithms that rule our web2 world, and even extends to the structures that underpin art and culture in the analog world as well.
Curatorial governance is applicable to people in a wide variety of disciplines – from music A&Rs and managers to film studio executives and agents to gallerists and museum curators to NFT collectors and many more – where some works are chosen to be supported over others.
Even if you are not a participant in any of these disciplines, this topic also applies to you if you care about what media you or others consume.
Inherent to this conversation is a move away from the hyper-personalization-driven digital media environment. If you believe that there is value in the wisdom of the crowd, experts, or one’s network beyond the value of your own intuition or preferences, then this is a topic for you.
Writing about these topics more than anything else gives logic to my thoughts and clarity to the rough outlines of new ideas, so here goes. Search for the word “skip” down to the end if you want to see how we can collectively construct a better world when it comes to valuing cultural objects.
The wisdom and fallacies of the crowd
To take modern art’s most famous contemporary curator Hans Ulrich Obrist’s words, curating is about “junction-making”, and the most potent junctures are between the past, present, and future.
“The curator is someone who insists on value, and who makes it, whether or not it actually exists,” says David Balzer, author of Curationism: How Curating Took Over the Art World and Everything Else.
Combining these two ideas, a working definition of curation and constructive criticism is to act of realizing the longer-term value of a cultural object in the short-term, closing the gap between the object being undervalued today and more highly valued tomorrow.
Taking this as the goal of curation and criticism, based on the Carti example it’s clear that the wisdom of the crowd at a specific moment in time – particularly at t=0 or shortly thereafter – cannot be used as the sole input when assessing the potential longer-term value of a cultural object.
The Carti example is illustrative of three biases in the wisdom of the crowd: information cascades, binary/directional voting and misaligned incentives, and free market default values.
When analyzing the wisdom of the crowd, it’s easy to forget that opinions are not independent, and that social media is very driven by first mover opinions and “influencers”.
The rise of misinformation on the Internet should show everyone that inaccurate opinions believed by “the crowd” can spread like wildfire. The same phenomenon of information cascades occurs in cultural markets as well.
A second important bias to observe related to, but distinct from information cascades in how “incorrect” value spreads is directional (or binary) voting.
Due to cognitive ease and hardwired psychology, most decisions in life – particularly on the Internet – tend to be reduced to binaries, in part due to our tendency to form in-groups vs. outgroups. These binaries strip opinions of important nuance and subtlety that are key in assessing a work’s longer-term value.
More subtly, when individual members of the crowd “vote” on binary topics on the Internet by voicing their opinions, they have no skin in the game, and are not wagering anything with their votes. Given this, they have less incentive to go against the grain and voice what they believe to be “true”, as the risk is not worth the reward, given that most humans are loss averse, and because the prize for being a heretic is small.
In binary voting systems, people also have little incentive to express the true degree of their opinions. In fact, given the way most social media algorithms reward engagement, people are actually disincentivized to express the true degree of their opinions – assuming by definition that most people don’t have extreme opinions on most topics – and are actually incentivized to be as bombastic in their opinions as possible given there is no finite resource at stake.
If there was a world in which one had to wager $1000 dollars on what the consensus opinion of WLR in 2022 would be before tweeting about it in 2020, the initial wisdom of members of the crowd might be more closely aligned with the longer-term value of the album.
This concept of the staking of limited resources as a measure of how strongly beliefs are held and likely to come true is why prediction markets like sports books or Metaculus are often seen as the best sources of truth for the future. This is also the theory behind quadratic voting, where one has to stake one’s vote by spending a limited, zero-sum resource.
It’s worth pausing to say that even though there are evident flaws in the wisdom of the crowd, there are also clear benefits to surveying the crowd.
The intent of this piece is not to deride public opinion; the goal of this piece is instead to figure out the pros and cons of each of the typical four sources of information (public opinion, experts, personal networks, intuition) one uses to make decisions, in order to create repeatable processes of organizing information that combine the best of each source to exceed the value of any one source alone.
It’s worth paying attention to the crowd to some degree because the market for attention is a de facto market of a limited resource given that attention is relatively finite. In this market of attention, special awareness should be paid to cultural objects that do not immediately appear to possess “free market default values”.
Free market default values are the third and last major force to observe when unpacking what causes the wisdom of the crowd to be unaligned with longer-term value.
Free market default values can be defined as the preprogrammed values of capitalist society or evolutionary biology, and values that are linked closely with dopamine spikes. Though this list is by no means exhaustive, one can think of the following as examples of free market default values:
-
Familiarity
-
Visibility / loudness
-
Fame
-
Power / influence
-
Wealth
-
Prestige
-
Beauty / sex appeal
-
Humor
-
Fun
-
Intelligence
-
Cognitive ease
-
Coherence
-
Congeniality
The ‘free market default values’ are the values we default to in the absence of creating our own.
Biases that affect individuals tend to be more idiosyncratic in terms of what they lead a person to value – selection bias might lead someone to value A, and another person to value B for example – while these free market default values tend to be relatively universal at the subconscious if not conscious level, as we all have been primed to believe that these values are strong predictors of future success.
Though these free market default values are all fine as second-order or second-tier values, the goal of curation is closely linked to proposing new first-order / first tier values outside of the free market default values, and challenging our core assumptions.
While trusting the wisdom of the crowd one hundred percent of the time might not be the best idea, the benefit of the wisdom of the crowd is that you are not as directly subject to one of the many, many types of biases that any one individual or even group of individuals may have.
Every one of the other three main sources of information we use to make decisions – experts, one’s network, and one’s intuition – is subject to the many types of individual biases in a much greater way due to having a much smaller “n” or sample size.
Examining the experts
If we’re not taking the wisdom of the crowd as total gospel, at the other end of the spectrum are the pitfalls and benefits of expert opinion.
We can define an expert as someone who has spent a significant amount of time studying a related field, and as such it’s reasonable to assume that experts are slightly better at avoiding the problems of information cascades, binary voting, and free market default values.
This is because the time afforded to experts to think allows them the space and social proof necessary to be a critical thinker free from the very short, reactionary moments in which information cascades are triggered.
The time spent by experts crystallizing thoughts about a given issue in advance of particular moments and objects also allows them to have more nuanced opinions on new objects, and more time to think about what is valuable with intentionality.
If we value labor and time spent when it comes to solving problems, then we should value the opinion of someone who has shown the care and commitment to putting in the work to understand as much as possible about a given field.
However, this greater time spent thinking also causes experts to be more rigid and dogmatic in their thinking. This combined with the ivory tower that experts can sometimes live in can result in them completely overlooking important, left-field new ideas. In the history of music criticism, one can look at the flaws of “rockism” as a classic example:
Rockism means idolizing the authentic old legend (or underground hero) while mocking the latest pop star; lionizing punk while barely tolerating disco; loving the live show and hating the music video; extolling the growling performer while hating the lip-syncher.
Over the past decades, these tendencies have congealed into an ugly sort of common sense.
One can never understand everything there is to know, yet to develop expertise runs the risk of believing one knows everything – or absolutely nothing – instead of just knowing something about something. There is a lot of research on the biases of “smart” people, and these biases are also true for “experts”. There are also major biases in who gets appointed to be “experts” in the first place, and those institutional biases color what is deemed to be valuable by those experts.
When relying on the opinions of experts, there is also the practical challenge that you tend to beholden to the process that the expert has for forming an opinion. Patrick Collison’s work around fast grants is illustrative of the fact that sometimes one needs information from experts more quickly than they can give the answer, which is attendant to the fact given that spending extended amounts of time on a problem is crucial to what makes a good expert.
The double-edged sword of personal networks & intuition
It’s faster to get information on the value of a cultural object by leveraging one’s immediate network, or relying on one’s own intuition. Using sources of information with small “n” is a double-edged sword though.
The advantage of these sources of information with minuscule sample sizes is that they are likely to surface cultural objects at the earliest point in time, when the difference between short-term value and longer-term value is often greatest.
Using one’s own network and intuition is also most likely to surface weird, outlier cultural objects that are polarizing and challenging, and most resonant to those who find the object valuable.
Even if most people don’t have a practiced intuition or diverse networks to produce strange objects, listening to one’s own network or intuition is much more likely to voice heterodoxical opinions on the value of a cultural object compared to the wisdom of the crowd or experts as there is less noise.
Not only does using one’s intuition or sources of information, network, or other sources with small sample sizes allow for idiosyncratic opinions, both sources allow for visceral, inexplicable, and subconscious feelings about a cultural object to be captured and not be lost, which are some of the most important points of data about a work. Using data sources with small “n” allows for the quirks of our personalities and perspectives to be recognized and valued, and sharp pangs of emotion to be captured.
The beautiful quirks of humanity can also be dangerous though, as these quirks go hand-in-hand with individual biases. Choosing whose idiosyncrasies and unique opinions to highlight and whose to ignore is also tough in the age of media and opinion overabundance.
Once the pros and cons of the various sources of information that one uses to determine the value of cultural objects have been analyzed, improvements to the status quo can be made.
It’s important not to lose sight of that goal, which is to craft a draft of a solution that seeks to repeatedly (A) realize the longer-term value of cultural object in the short-term, closing the gap between the object being undervalued today and highly valued tomorrow and (B) do so better than tapping any singular source of information or systems that exist today.
It is possible to craft a better, more systematic solution to how we value cultural objects by synthesizing the pros of and improvements to each source of the four main sources of information.
If done right, a world that has shed the baggage and biases of the past – and bends closer toward true fairness and egalitarianism when it comes to the valuing of cultural objects – can be built.
Skip here if you care more about the solution than analysis of the problem.
Let’s start with capturing the best of the wisdom of the crowd to construct a fairer world.
The current paradigm on major social media uses algorithms to distill the wisdom of the crowd, and these algorithms all share the same problems as the wisdom of the crowd overall.
In order to build a system that is better tuned towards the goal of curation as mentioned above and avoids the intrinsic biases in the wisdom of the crowd, the following processes and principles are valuable:
-
Augmenting all quantitative data on cultural objects with human contextualization is crucial to fill in the gaps on why something might be valuable.
When creating a system where cultural objects are flagged by automated systems once they certain threshold, it’s worth quickly soliciting three different people to write written cases for why (1) a cultural object is valuable, the counterposition on why (2) a cultural object is not valuable, and (3) why the artist or creator has the potential to create something valuable in the future regardless of the value of the particular cultural object in question.
Think of this as similar to Letterboxd reviews for all cultural objects, where a curatorial system is constantly soliciting new reviews and new conversations about value as new objects are created. Creating incentives for keeping the reviews as thoughtful and nuanced is key in this system.
-
Instead of using aggregated engagement data on consumers’ revealed preferences, it's actually more valuable to disaggregate “the crowd” into specific communities and clusters, and to pick cultural objects that spike in particular communities or are at the intersection of two to three communities, rather than the objects that are broadly liked. To some degree, while this task of disaggregation into specific communities sounds exhausting and computationally taxing, it already exists in music and other forms of media.
Communities oriented around specific missions, values, or genres are likely have more intentional and precise definitions of what is valuable, while what’s broadly liked tends to be consumed more passively and less intentionally. This intentionality is key when determining what will have longer-term value.
This is where the beauty in the wisdom of the crowd really shines, in uncovering niches that are underground yet super beloved and fast growing. A rough measure of success in the disaggregation of the wisdom of the crowd is whether this type of system surfaces the best of hyper regional or specific communities. The movement of hyperpop from an algorithmically identified microgenre on Spotify by “data alchemist” Glenn McDonald to official Spotify playlist and then to a full-fledged music scene and critically respected genre is a great example of the potential of the disaggregated wisdom of the crowd.
-
In addition to disaggregating aggregated data, it’s also essential to use different metrics to measure the value of media than the ones used today, and metrics that rely less on engagement and more on network influence.
A good point (though far from perfect) of inspiration is academia’s h-index, which seeks to define value based on how influential a work is downstream, rather than solely on its quantity of consumption. Obscure work A that influences work B that influences very valuable work C is as valuable if not more so than C, as without A C would not exist.
With these three solutions, the best of the wisdom of the crowd can be distilled while minimizing the downsides.
In order to take the best of experts into consideration, it’s important to find a way of defining what an expert is that allows for the minting of new experts over time and to avoid the pitfalls of the ivory tower.
To do so, it’s worth:
-
Defining an expert as a T-shaped learner that has broad curiosity (both about the field and the world) and specific infatuations, or as omnivores with obsession
-
Observing conversation and on-the-record (on-chain) data from interested participants to accurately acknowledge someone putting in the work to sharpen their curiosity and obsessions
The hope is that one day blockchain-based social media and community tools can create a portable resume or data set that will allow one to take one’s curiosity and obsession data from platform to platform; for now though, creating holistic cross-platform portraits of experts is key. Allowing active discourse participants / future experts to share their opinions in the same place that they ascribe value to cultural objects is also really important for this purpose.
-
In addition to minting new experts, it’s also important to have the weight of the opinions of a given expert be dynamic over time, and specific to the object at hand, so some sort of relevance score in addition to a success score is also crucial.
The goal of this work is to create a system that gives as much weight to the opinions of the young kid obsessed and curious about her favorite artistic medium and the world as it does to the weight of the right TikTok or Twitter reviewer or traditional media institution critic or curator in any given cultural discipline.
Once the best of the wisdom of the crowd and experts has been taken as inputs, one can then focus on how to harness the best of one’s network and one's intuition.
When it comes to pulling on the best of our own networks and intuitions, the solution is relatively easy in theory and hard in practice, and comes from gaining awareness of all of the different types of bias and actively countering them at every step.
Defining the curatorial process as a five step process of (1) setting the right goal or set of values, (2) sourcing the right artist, (3) sourcing the right work, (4) evaluating the right work, (5) contextualizing the right work appropriately, and (6) creating the right set of works, it’s important that one listens to one’s network and intuition along the way while also gut checking both at all six points. Given the content differentiation models of video streaming services, one can see this six-step process today in how services like Netflix, HBO, or MUBI choose shows or films to acquire.
Biases in one’s network and intuition occur at every step of the curatorial process, yet are most likely to become evident in the last step when creating the set of works to express the goal or set of values, so it’s imperative to account for biases there.
Once one has taken the best of each of the main sources of information and polished the inputs, it’s important to figure out the best way of weighing each.
Separating out sourcing from evaluating is key. For sourcing decisions, there needs to be lower-stakes quadratic voting to be as exhaustive as possible while minimizing collective time wasted.
For evaluation decisions, there need to be higher stakes quadratic voting with even more precious resources given the even more precious resources at play than collective time. For example, if only ten works fit on a gallery’s walls or on a member’s home screen for example, or that there is a limited budget for funding objects, voting becomes even more important, and the limited nature of the points system should reflect the limited nature of the resource in question.
Our attention is precious. Today’s digital media systems have been designed to focus our attention on cultural objects that have short-term value, not longer-term value. Most of us strive to marry who we are in the short-term with whom we want to be in the longer-term. As such, it behooves us to focus on creating systems that better align the longer-term with the short-term when it comes to the cultural objects we value.
This piece was compiled using my intuition and the feedback of my network, relying on the opinions of experts, and now is being turned over to the crowd. I hope it is even better than it otherwise would have been as a result.
Thank you to Peter Boyce, Jose Mejia, Brian Watson, Gaby Goldberg for the feedback, and Andy Weissman, Tom Windish, Derek Kaneko for the words of encouragement :)