The CryptoPunks are 32x32x4 images where the 4 element array at each point in the 2D image represents a pixel encoded in the Red, Green, Blue, Transparent (RGBT) encoding scheme. Simplifying the source material is a common method for building and intuition for a dataset. Let's imagine the punks were created with black and white pixels exclusively. By building a set of black and white punks, and then training our model on those punks, we can learn something about the classifier’s performance.

Generating new Training Data using CPUNKS-10K

We are going to generate a new set of Cryptopunks that only contain black, white and transparent pixel values. First, load the CPUNKS-10K datasets.

import cpunks.cpunks10k as cpunks10k
cp = cpunks10k.cpunks10k()
(X_train, Y_train), (X_test, Y_test) = cp.load_data()

The 8 rarest CryptoPunks are the 4 male and 4 female punks that have no additional attributes. We can display them with the following command.

cp.show([3307, 281, 510, 741, 6487, 641, 1050, 2204]

We see that each of the male and female punks use one of four skin tones. A quick inspection of the colors used in these punks suggests an algorithm for transforming them into black and white versions:

1. Keep the transparency channel transparent
2. Swap the skin tones for white.
3. Swap everything else with black.

The CPUNKS-10K API includes a ColorMap class that makes it easy to swap out colors in CryptoPunks images.

cm = ColorMap()
plt.imshow(cm.img_to_black_white(X_train[0]))

That looks pretty good. We can generate the full set of 10,000 Black and White Cryptopunks using the following command.

X = np.concatenate((X_train, X_test), axis=0)
bw_punks = np.array([cm.img_to_black_white(img) for img in X])
cp.show(bw_punks, nrows=10, ncols=10)

Training the Neural Net in Black and White.

We might expect our model to do a little worse when trained in black and white. Afterall, we’ve thrown out a significant amount of information, for example, hair color, lipstick, or eye makeup, that may have been useful in model performance. Let’s see what happens.

The lines below swap out our new black and white punks with the colored versions.

X_train = bw_punks[0:9000]
X_test = bw_punks[9000:10000]
model.fit(X_train,
          Y_train,
          batch_size = 32,
          epochs = 10,
          shuffle = True)
model.evaluate(X_test, Y_test)

32/32 [==============================] - 0s 2ms/step - loss: 0.0045 - accuracy: 0.9980

Not much change! The new model was able to learn punk gender without the color information it had before.

Understanding the Model

To understand what is happening here, we need to revisit the 8 origin punks in black and white.

First, we observe that the four males and four females are identical. However, there are some clear differences: head size, shadow size, nose size and neck alignment. If we dig a little deeper, we realize that the vector spaces inhabitated by male and female punks do not intersect, that is, a male punk will never have black pixels along the 8th column and vice versa, a female punk will never have a long vertical chain along the 6th column.

Towards Generative Art

This intention of the book is not to dig too deep into model interpret-ability but instead, to spark curiosity. After all, in trying to understand the model, we’ve created our first piece of generative art, the Black and White Punks. While we are certainly not the first to stumble on this idea we may be the first to have generated all 10,000 black and white punks in a repeatable and systematic method that retains the lineage of original artwork. These qualities of process, reproducibility and scale are some of the tenents of generative art that we will see persist throughout this work.