Reflecting on our mint site

This is the first of what will become a collection of blog posts covering the more technical aspects of Starcatchers. Our goal is to help explain more nuanced aspects to help enable other projects to make proper technical decisions. Whether we had a miss and had a learning experience, or hit the nail on the head, we want to contribute back to the community with what we learned.

To kick things off I’m giving a more thorough and complete post mortem on our minting website for Starcatchers. This is an elaboration of the Twitter thread that I put out shortly after mint completed.

Reliability

Before launching I ran a multitude of different load test scenarios to determine maximum load our application could handle, and better decide on auto-scaling thresholds.

loader.io load test example
loader.io load test example

Most importantly, our website did not go down during mint. We did not succumb to any DDOS attacks, or fatal optimization issues prior to launch.

Our specific numbers during launch to highlight:

  • Over 2 million hits
  • Over 80,000 unique visitors
  • Over half a terabyte of data served

Below is the break-down of request latencies with common percentiles.

p50 latencies
p50 latencies
p95 latencies
p95 latencies

Generally speaking, our average request time was 60ms. On the high end requests took around 300ms to complete. There are a few factors that contributed to the spread in request latency.

  • Our service scaling up, and fresh instances taking longer to respond.
  • Geographical distance from the user and where our application was hosted.
Application backend locations
Application backend locations

We hosted our application with Google Cloud in Taiwan, Finland, and Iowa in the US. These locations were picked to have a well spread out infrastructure, without having to pay too high of fees for hosting the site.

Request distribution by geographical location
Request distribution by geographical location

This was not perfect however, as we had over a hundred thousand requests from Australia, and tens of thousands of requests from both South America and Africa. We certainly could have sacrificed a little bit more cost to give a better experience to these locations in hindsight.

Even still, we didn’t drop a single request with our setup, and saved a lot of money handling requests due to the caching provided for free by Cloudflare. Which is a great segway into:

Security

I am a huge fan of Cloudflare. The amount of time and hassle both securing and caching requests that is saved by using Cloudflare is significant.

Cloudflare Blocked Request Stats
Cloudflare Blocked Request Stats

We only utilized the free tier of Cloudflare, but were able to block the obvious bot requests, and remove any concerns around DDOS attacks. I really can’t overstate this feeling enough, as many other projects have felt the pain of being targeted during their launch.

Another aspect worth calling out here is that our application is stateless and doesn’t have any secrets. The attack vector here is quite small, as our application is also scaling up and down to meet demand. Even if an attacker had limited access they would not have a good time trying to compromise the already running stateless application, that may die at any moment. This is also built on top of the fact that a cloud load balancer separates our application from the internet.

Another tangential piece is that we were able to easily control access to the developer section of the site using Google’s Identity-Aware Proxy. We could quickly iterate and test changes live, without actually impacting the production version of the site until we were ready.

Starcatchers IAP Dev Site
Starcatchers IAP Dev Site

This simple login made it so that only our limited team had access to view the pending changes before they were made live.

In Conclusion

We are very pleased with how the mint turned out. With the help of the Google developer discount and Cloudflare’s great free-tier perks we spent $0 total on these critical tools.

GCP Costs for February
GCP Costs for February

If you were to take out the discounts we received, then we would have spent a grand total of $50 in February to have a load-tested website, an adjacent developer site, and a highly scalable production site.

We were able to provide a great experience for the community, and will continue to strive for the same experience going forward. I hope this article was able to shed some light on key considerations to make while creating an NFT minting site, and that you enjoyed reading.

If you have any feedback, please reach out!

Subscribe to Starcatchers
Receive the latest updates directly to your inbox.
Verification
This entry has been permanently stored onchain and signed by its creator.