Notes from the studio of Holly Herndon & Mathew Dryhurst 🦾

![Robert Doisneau: Woman Registering Shock at a Painting of a Nude in Paris Shop Window, 1948](https://images.mirror-media.xyz/publication-images/X67xhZ1066RraI5MWjtWT.png?height=620&width=897)

**7:17PM CET update:** People requested to collect this piece of writing so I made it collectible. That is very nice and compliments the piece. <3


---

The "Web 3 arbitrarily introduces scarcity to digital media" critique seems to never go away and I think it is fundamentally flawed.

Digital media has always been scarce, because there is a rights holder conferred by law unless that rights holder renounces those rights with a license.

In plain terms, this means that at least in the jurisdictions where I am able to comprehend twitter arguments, when you take a digital photograph or create a piece of digital art, the copyright of that art is automatically assigned to the creator. It is scarce. 1 of 1. In the case of infringement of your rights as copyright holder, [it can be advantageous to have formally registered the work](https://www.artbusiness.com/register_and_copyright_art_for_artists.html), however the fact remains that digital media is scarce at the point of creation.

That these critiques often come from corners who mistakenly characterize crypto as a fraudulent fiction is ironic, as the premise that the digital media we enjoy on previous internets is not scarcely owned by anyone is fictitious. No scarcity is being *imposed* on media, existing scarcity is being *surfaced* and experimented with.

This is the brilliance of the NFT experiment. NFTs represent an elegant proposal to attempt what I would describe as **feasible abundance**. I.e how can we make free media fair for the people creating it?

Decades deep into the internet experiment we had not yet found a way to reconcile our desire to have as many people as possible enjoy media, while also making life financially feasible for the creators of that work. NFTs create games and value exchange around ownership and collection abstracted above access to the content itself, which ought to be available to everyone. As [Jacob.eth articulates](https://cryptomedia.wtf/), the experiment proposes that the more a work is freely available, shared and memed, the more valuable ownership of that work becomes. As such, it is almost impossible to find a piece of NFT art locked behind a token gated wall. In real terms, *meaningful scarcity* would mean withholding that access.

This premise also explains why many of the most celebrated early NFT auctions from within the Ethereum community were held to retroactively compensate popular memes, from [Nyan Cat](https://foundation.app/@NyanCat/foundation/219) to [Disaster Girl](https://foundation.app/@DisasterGirl/foundation/25046). The values put forward by many corners of the NFT community are that media ought to be freely available, and that free media ought to be valuable. Information is now free *and* expensive. To achieve both states to the satisfaction of all parties at this immature stage is a minor miracle and to be celebrated.

This accomplishment is made possible through experimentation with previously hidden dynamics, and foregrounding them can be startling to those who were not previously aware of their existence. I call this phenomenon **the shock of the nude**.

Many critics of crypto and NFTs mistakenly suggest that crypto is introducing financialization, inequity or scarcity to our digital lives. This is false. Public blockchains, through making visible latent forces such as financing, unequal returns, or scarce and valuable ownership, are bringing long existing dynamics to the surface to be scrutinized. **These forces are not *new*, they are *nude*.**

Our previous internet lives are financialized daily on interfaces most of us do not have access to. Web 2 has concentrated wealth and attention to a handful of players at the top (who often most dutifully serve the objectives and accept the universal terms of a handful of platforms), but few other than professional insiders *have seen* the full extent to which this is true.

What often emerges on twitter is people with a casual, idealistic, fan-side relationship with media condescending those with a longstanding commitment and understanding of their respective industries. This gulf in knowledge becomes apparent when critics are invited to propose their own alternatives to the current state of the web for creators, with highly RTable but practically infeasible ideas ranging from [laborious illegible key exchanges on a private database](https://blog.mollywhite.net/digital-artists-post-bubble-hopes-for-nfts-dont-need-a-blockchain/) (i.e like a blockchain but way worse) to [vague allusions to revive failed state based internets from 50 years ago](https://reallifemag.com/reconnected/). These ideas sound nice to an equally inexperienced crowd, but are unserious and do not stand to help anyone but their authors. If only things were so easy!

An unsurprising byproduct of a web modeled to facilitate advertising, it is often quite difficult to discern between solid information and promotional misdirection on Web 2.  There is a significant discrepancy between the statistics the public has access to and the hard financial and industrial realities obscured by them.

In the arts, the followers and stream numbers we see are a very distorted indicator of how well the person in question is doing financially, and say little of how people actually make a living. Due to tired dynamics of self promotion and brand construction encouraged by web 2 platforms, there is little incentive for anyone to disclose that they are struggling or in a precarious position. Crypto’s public ledgers shatter illusions and hold potential of unveiling sobering truths. **Web 2 is the grift.** **Web 3 is a gift**. It is not morally sound to accept that many people are committing their 20s and 30s to participating in a fiction while silently struggling, adrift of access to good information.

Only once the warts-and-all truth is made visible do we have any hope of addressing how the world actually works. The shock of the nude may be a cold shower, but a dose of reality is far more useful to practicing artists or internet denizens than a fictitious understanding of  the web and creative industries. Access to more accurate information outside of the promotional mirage of likes and follows is an uncontroversial good for anyone aspiring to make a living.

That said, while public ledgers make it quite difficult for many to pretend that things are going better than they actually are, there is no shortage of illusion creation on public blockchains too. For a minority who have the means, it is possible to distort perceptions by illicitly pumping bags, making anonymous acquisitions and the like. **Public ledgers are no panacea**, however I find it far preferable that such coordinated distortions be legible and open to scrutiny. As discussed [in our podcast with Context.app](https://interdependence.fm/episodes/making-web3-human-legible-constructive-speculation-and-the-shock-of-the-nude-with-context), more tools are being built to make on-chain activity human legible.

It will become harder and harder over time to get away with as new forms of journalism are made possible through the ability to track foul play, even retroactively. As [Taylor Monahan recently suggested on Twitter](https://twitter.com/tayvano_/status/1532085029857284101?s=20&t=UeymocOPtyFVEUOwwmhR8A), so far often the best policing of foul play in crypto has come from within crypto itself, and that accountability will spread as more learn how to read on-chain activity.

You can follow me [@matdryhurst](http://twitter.com/matdryhurst) on twitter

We discuss matters like this with wonderful guests weekly [on the Interdependence podcast](https://interdependence.fm/)

D̶i̶g̶i̶t̶a̶l̶ ̶S̶c̶a̶r̶c̶i̶t̶y̶ Feasible Abundance and the Shock of the Nude

We have had the great pleasure to be working with OpenAI InPaint and [DALL·E 1](https://en.wikipedia.org/wiki/DALL-E) for many months, and thought it would be constructive to document some techniques, ideas and reflections that have been raised through exploring these remarkable tools in honor of the announcement and warranted public excitement around [DALL·E 2](https://openai.com/dall-e-2/).

![Holly Herndon & Mathew Dryhurst (DALL·E 1 test image)](https://images.mirror-media.xyz/publication-images/G2IuLWbXAogYQ4g6Sg7W8.png?height=3226&width=3970&&&&&&&&&size=medium)

### Backstory

If you are new to this area, it is worth establishing a brief timeline of developments from the past year. In January last year OpenAI announced [DALL·E](https://openai.com/blog/dall-e/), a transformer model capable of generating convincing artworks from textual descriptions. This was shortly followed by the release of [OpenAI CLIP](https://openai.com/blog/clip/) (Contrastive Language Image Pre-Training), a neural network trained on image/description pairs released for public testing.

What followed was an explosion of experimentation beginning in Spring 2021, with artists such as (but not limited to) [Ryan Murdock](https://twitter.com/advadnoun) and [Rivers Have Wings](https://twitter.com/RiversHaveWings) releasing free and open tools for the public to play with generating artworks by connecting CLIPs capacity to discern textual prompts with open image generation techniques such as Taming Transformer’s [VQGAN](https://compvis.github.io/taming-transformers/), and OpenAI’s own novel [Guided Diffusion](https://github.com/openai/guided-diffusion) method.

In lay terms, to create artwork one types what one wants to see, and this combination of generator (Diffusion, VQGAN) and discriminator (CLIP) produces an image that satisfies a desired prompt. In Ryan Murdock’s Latent Visions discord group, the artist [Johannez](https://twitter.com/_johannezz) (who has been publishing artworks and tutorials around machine learning imagery and music online for years) coined the term “Promptism” for this astounding new capacity to conjure artworks from telling a neural network what one would like to see.

While this development is but the latest advancement in a legacy of machine learning in art too long to do justice to in a blog post, this act of conjuring artworks from language *feels very very new*. Feeling is an important dimension to the act of creating an artwork, as while we have for some years had the capacity to generate art from the laborious process of training GANs, often waiting overnight for results that invite the observer to squint and imagine a future of abundant possibility, a perfect storm in the past 18 months has led to a present in which the promise of co-creation with a machine is realized. It *feels* like jamming, giving and receiving feedback while refining an idea with an inhuman collaborator, seamlessly *art-ing*. It intuitively *feels* like an art making tool.

Analogies are imperfect, but this resembles the leap from the early electronic music period of manually stitching together pieces of tape to collage together a composition to the introduction of the wired synthesizer studio, or Digital Audio Workstation. In the 20th century this progress in musical tools-at-hand took place over half a century. In contrast, this leap in machine learning generated imaging tools-at-hand has gathered steam over a 3-5 year period.

This dizzying pace of development encourages reflection!

### Reflections

Since Alexander Mordvintsev released the [Deep Dream](https://en.wikipedia.org/wiki/DeepDream) project in 2015, machine learning imagery is often associated with surrealism. It is no doubt surreal to find oneself co-creating art by probing a disembodied cognitive system trained on billions of datapoints, and the images produced can be confounding and psychedelic. However these are characteristics often ascribed to new experience. New experiences are weird, until they are no longer.

At the advent of DALL·E 2 we find it quite useful to think instead to the history of the [Pictorialism](https://en.wikipedia.org/wiki/Pictorialism#Overview) movement.

![Edward Steichen: Rodin, Le Penseur (1902)](https://images.mirror-media.xyz/publication-images/gW2JFPpDtCNgNT6L_7tvH.jpg?height=803&width=1000&&&&&&&&&size=medium)

In the early to mid 19th century, photography was the domain of a select group of engineers and enthusiasts, as it required a great deal of technical knowledge to realize photographs. As such, the focus of these efforts and gauge of the success of the medium was to create images that most accurately reflected the reality they intended to capture.

The Pictorialists emerged in the late 19th century as a movement intent not on using photographic techniques to most accurately depict reality, instead opting to use photography as a medium of communicating subjective beauty. This movement progressed photography from a scientific to an expressive medium in parallel with increasing access to cameras that required no technical expertise (the first amateur camera, [the Kodak, was released in 1888](https://americanhistory.si.edu/collections/search/object/nmah_760118)).

![Léonard Misonne: By The Mill (1910s)](https://images.mirror-media.xyz/publication-images/KWRNtSYai5BP3YC5qtNUs.jpeg?height=787&width=1070&&&&&&&&&size=medium)

Debates raged about whether or not photography could in fact be *art*, a conversation inevitably dismissed once artists were able to affordably integrate cameras into their practices. The Pictorialists extolled the artistic potential of the camera by perverting its purported function of objectively capturing the world, staging scenes and experimenting with practical effects to create subjective and expressive works. Painters soon began integrating the camera into their workflow, and now most art forms make use of sensor based imaging in one form or another.

Larry Tesler’s pithy description of AI, “Artificial intelligence is whatever hasn’t been done yet.”, is equally applicable looking back to the birth of photography. Photography had also *not been done yet*, and was then (a phenomenon described as [the AI effect](https://en.wikipedia.org/wiki/AI_effect)) soon integrated into most artistic and industrial practices. Sensor based imaging remains a discrete focus of research pursuing things that have not been done yet.

The parallels with this time period are clear. In machine learning imagery, the laborious task to this point has been to legitimize the medium by attempting to accurately reflect the reality of training material. Great efforts have been made to realize a convincing and novel *dog* picture from training neural networks on *dogs*. This problem has been solved, and sets the foundation upon which we can begin to be expressive.

As happened with the Pictorialists, prompt based systems like DALL·E are democratizing the means by which anyone can create AI facilitated subjective art with next to no technical expertise necessary, and we assume that in the coming decades these techniques will be integrated into artistic and industrial practices to a great degree. AI will remain a discrete field of research exploring what has not been done yet, and in 2 years we will think nothing of painters using tools like DALL·E to audition concepts, or companies using tools like DALL·E to audition furniture for their office.

DALL·E represents a shift from attempts to reflect objective reality to subjective play. Language is the lens by which we reveal the objective reality known to the neural network being explored. It is a latent(hidden) camera, uncovering snapshots of a vast and complex latent space.

The same debates will rage about whether or not prompt based AI imagery can be considered *Art*, and will just as inevitably be relegated to history once everyone makes use of these tools to better share what is on their mind.

The ever evolving pursuit of art is greatly benefitted from reducing any friction in sharing what is on your mind. The observer is the ultimate discriminator, and as with any technological development that makes achieving a particular outcome more frictionless, creating great art that speaks to people in the time that it is made remains an elusive and magical odyssey.

The easier it is to generate artworks, the more challenging it will be to generate distinction and meaning, as it ever was. Great Art, like AI, is very often what hasn’t been explored yet.

### DALL·E, reflections and infinite images

The first time we really got excited about DALL·E was in discovering its capacity to produce internally coherent images. Internal coherence can best be described as the ability to create convincing relationships between objects generated within an image.

To test this capacity, we began to use InPaint to extend images that we uploaded to the system without contributing a linguistic prompt. To achieve this, we would manually shift the viewfinder of InPaint in either direction to extend the scene.

One particularly successful test involved extending the scene of symbolist painter Charles Guilloux’s work - *L'allée d'eau* (1895)

![Charles Guilloux - L'allée d'eau (1895)](https://images.mirror-media.xyz/publication-images/ELIKrPrGZCDeos4Xj9qol.jpg?height=733&width=960&&&&&&&&&size=medium)

![Charles Guilloux - L'allée d'eau (Extended with DALL·E by Holly Herndon & Mathew Dryhurst)](https://images.mirror-media.xyz/publication-images/uHWIrvvT-eNCNMW4f2-BS.png?height=2331&width=3569&&&&size=medium)

As you can see, DALL·E is capable of comprehending the style and subject of the scene, and extending it coherently in all directions. Perhaps most notable is its capacity to produce reflections consistent with objects present within the image. The curvature of the trees and river is successfully reflected in the new water being generated.

This capacity for internal coherence could only happen within the (then) 512x512 pixel viewfinder of InPaint, however successfully demonstrated a capacity to produce images of potentially infinite scale by producing incremental coherent *patches* of an image.

We then extended this technique to produce very large pieces under the same reflective motif. The wall sized artworks below (produced with DALL·E 1 and InPaint) are we believe the largest compositions ever produced with machine learning at their time of creation.

![Holly Herndon & Mathew Dryhurst (DALL·E 1 test image)](https://images.mirror-media.xyz/publication-images/U-bzdprWFntyErCfsO9qH.png?height=1851&width=5977&&&&&&&&&size=large)

![ Holly Herndon & Mathew Dryhurst (DALL·E 1 test image)](https://images.mirror-media.xyz/publication-images/cfbWc1wLPWZyA2eSrmCIR.png?height=2154&width=4190&&&&&&&&&size=large)

This process of creating a patchwork of internally coherent images to form a larger composition was very challenging, quite like attempting to paint a wall sized work from the vantage point of a magnifying glass, and with no master guide to follow. As such, we used techniques like horizon lines to retain coherence.

There is something poetic to composing in this way. Attempting to extrapolate a bigger picture with only access to a small piece of it at a time feels appropriate in the broader context of AI.

We have not yet been able to experiment with DALL-E 2 and InPaint together in the same way, however assume that even more coherent and vivid images can be produced using a similar patchwork technique.

One can imagine efforts being made to increase the coherence of a larger composition by analyzing elements contained within an image outside of the scope of the InPaint viewfinder. With these developments in mind, we expect that these tools will soon contain all the elements necessary to produce limitless resolution compositions guided by language and stylistic prompts.

### Co-authoring narrative worlds

We began to make a series of infinite images extending horizontally, which allude to the potential for these tools to tell stories in the tradition of tapestries or graphic novels. This sequence below is large enough in resolution to be printed well, and is a narrative that could be extended to infinity.

![Holly Herndon & Mathew Dryhurst (DALL·E 1 test narrative)](https://images.mirror-media.xyz/publication-images/uOF6BJERGqfUE9J81U6R-.png?height=639&width=5668&&&&&&&&size=large)

To return to the original idea of extending a painting to reveal more of the scene, what might it mean to be able to produce infinite worlds from a single painting or photograph? This significantly augments the capacity of what we understand of generative art, when a coherent world, or narrative, can be spawned from a single stylistic or linguistic prompt.

![Holly Herndon & Mathew Dryhurst (DALL·E 1 test narrative)](https://images.mirror-media.xyz/publication-images/j48RjuhWHn3_biDka4-uT.png?height=4096&width=17352&&&&&&&&size=large)

One can imagine narrative art forms like graphic novels or cinema being impacted from such a development. This is a particularly exciting prospect when considering the more recent development of prompt art bots being used in active Discord communities. The first of these we encountered was developed by [Wolf Bear Studio](http://wolfbearstudio.com/) for their Halloween themed art project [Ghouls ‘n GANs](https://www.ghoulsngans.io/), where discord users were invited to generate artworks via an in-thread bot, a concept more recently being experimented with by [Midjourney](https://twitter.com/midjourney) and the artist Glassface’s [iDreamer](https://idreamer.xyz/) project (also in collaboration with Wolf Bear).

Bots such as these, in combination with ideas such as Simon De La Rouviere’s [Untitled Frontier](https://www.untitledfrontier.studio/) experiment in narrative co-authorship, augur a future of co-creation not only with artificial intelligence systems, but also more frictionlessly with one another.

This speaks to the crux of what DALL·E *feels like*, a tool for jamming, rapid iteration and potentially co-authored social experiences. One can imagine group storytelling sessions online and IRL that produce vivid narrative art to be replayed later or viewed from afar.

Once the friction to share what is on your mind has been eliminated, the ability to co-create social narrative art experiences at the dinner table or the theatre seems conceivable and exciting!

### DALL·E 2 and Spawning

We have only been working with DALL·E 2 for a short time, however what is clear is that the system has exponentially improved in terms of generated convincing and internally coherent images guided by language and image prompts.

We plan to publish more later on what we discover, however our initial experiments have involved further experimentation with the “Holly Herndon” embedding present within OpenAI CLIP. In lay terms, Holly meets the threshold of notoriety online to have characteristic elements of her image be understood by the CLIP language/image pairing network, something we explored last year with our [CLASSIFIED](https://foundation.app/collection/clsfd) series of self portraits created to reveal exactly what/who CLIP understands “Holly Herndon” to be.

We propose a term for this process, **Spawning**, a 21st century corollary to the 20th century process of sampling. If sampling afforded artists the ability to manipulate the artwork of others to collage together something new, spawning affords artists the ability to create entirely new artworks in the style of other people from AI systems trained on their work or likeness. As Holly recently communicated at the TED conference, this opens up the possibility for a new and mind-bending IP era of *Identity Play*, the ability to create works *as other people* in a responsible, fair and [permissive IP](https://www.thejaymo.net/2020/11/19/permissive-ips/) environment, something that we are exploring with the [Holly+](https://holly.mirror.xyz/54ds2IiOnvthjGFkokFCoaI4EabytH9xjAYy1irHy94) project.

![Holly Herndon & Mathew Dryhurst (Holly Herndon Clones in a 3D animation style produced with DALL·E 2) ](https://images.mirror-media.xyz/publication-images/8pmVhuYofhaKRBVKzr6Gx.png?height=1360&width=2928&&&&&&&size=large)

Tools like DALL·E 2 and InPaint undoubtedly propel us closer to this eventuality, evidenced by these “Holly Herndon” style memes we were able to generate in early tests. 

Memes feel an appropriate medium for experimentation in this context, as any single meme maintains it’s vitality from its ability to be personalized and perpetually built upon. Memes are cultural embeddings, not dissimilar to the embeddings present within the latent space of a neural network. 

Memes are a distillation of a consensual/archetypical feeling or vibe, in much the way that the “Holly Herndon” embedding with CLIP is a distillation of her characteristic properties (ginger braid and bangs, blue eyes, often photographed with a laptop), or the “Salvador Dali” embedding is a distillation of his unique artistic style.

![Holly Herndon & Mathew Dryhurst (Personalized Memes with DALL·E 2)](https://images.mirror-media.xyz/publication-images/bt-AC_skTZP07HvRHI8h9.png?height=1024&width=3072&&&&&&&size=large)

We find personalized applications like this pretty exciting, as while DALL·E 2 (and it’s stunning variations feature we will cover at a later time) unlocks the ability to produce convincing images in art historical styles familiar to it’s training set, we feel that there could be a misconception that its utility is limited to solely recreating artistic expressions of the past.

Like the introduction of the personal camera, it is easy to imagine a near future scenario in which all amateur and professional workflows across creative industries are augmented by these tools, helping people to more clearly depict what is in their mind through common language and an ever expanding training corpus.

The 21st century is going to be *wild*. We will share more as we learn more! 🦾

![Holly Herndon & Mathew Dryhurst ("A building that looks like Holly Herndon", made with DALL·E 2)](https://images.mirror-media.xyz/publication-images/IjS8ukonTqxpre1SehFYs.png?height=1024&width=1024&size=medium)

