In Search of an Infrastructure Holy Grail for DexGuru: Mastering Cost-Effectiveness and Performance

TL;DR

DexGuru's transition from cloud-based infrastructure to an on-premises setup marks a strategic shift towards unparalleled cost-effectiveness and control. This move not only aligns with our commitment to full data supply chain control but also enables us to deliver custom-tailored, high-performance solutions, especially for data-intensive applications like OLAP databases.

Unrivaled Cost-Effectiveness

We started our MVP in AWS (Got to $170k a month check), then decided to move towards Hetzner Cloud/Dedicated till we’ve got banned from there as we are “Crypto” company:

So we had no other choice, but build our own cloud:

Sorry, another one:

DexGuru's Infrastructure At A Glance

Our meticulously planned infrastructure includes:

  • High-Performance Computing (HPC): Customized servers and networking equipment designed for high-speed data processing.

  • Advanced Storage Solutions: Optimized storage configurations for quick data retrieval and efficient data management.

  • Sophisticated Virtualization Techniques: Enables flexible and efficient utilization of hardware resources.

Our setup includes:

  • Kubernetes (K8s): For managing containerized applications.

  • Elasticsearch: For powerful search and analytics capabilities.

  • Clickhouse: An efficient column-oriented DBMS.

  • Redis: Used for caching and as a message broker.

  • Blockchain Nodes: The backbone of our data pipeline.

  • Other Services: Supporting the broader infrastructure.

| Service          | CPU | RAM  | Storage |
|------------------|-----|------|---------|
| K8s              | 580 | 1760 | 8000    |
| Elasticsearch    | 480 | 2250 | 60000   |
| Clickhouse       | 440 | 1800 | 35000   |
| Redis            | 180 | 1000 | 1500    |
| Blockchain Nodes | 460 | 1460 | 160000  |
| Other            | 90  | 380  | 10000   |
| Total	           | 1480 | 9620 | 442000 |

Benefits of The Setup

Our on-premises infrastructure now provides us with significant cost advantages:

  • Substantial Savings: Overall we’ve invested $100k into equipment and setup (3 months payments to Hetzner) and now we are facing $3k a month bill for rack/power/internet instead of ~$35k

  • Predictable Expenses: On-premises setup offers predictable, fixed costs, unlike variable cloud pricing.

  • Long-Term Investment: Initial investments in hardware pay off quickly, transforming into lasting assets.

Complete Control Over Resources

By managing our infrastructure, we've gained total control over our resources, allowing us to:

  • Optimize Resource Allocation: Precisely tailor resource distribution to meet specific application needs.

  • Enhance Performance: Fine-tune systems for maximum efficiency, particularly crucial for data analytics and OLAP database applications.

  • Improve Scalability: Scale resources as needed without the constraints or cost penalties often associated with cloud services.

Tailored High-Performance Setups

Our focus on custom setups is especially beneficial for data analytics:

  • Optimized for OLAP Databases: Our infrastructure is specifically configured to excel in handling complex OLAP operations.

  • Enhanced Data Processing Speeds: By fine-tuning our hardware and software, we achieve faster data processing and query response times.

  • High Availability and Reliability: Ensures consistent performance and uptime, critical for analytics and real-time data processing.

The TCO and TEO Advantage

Our focus on Total Cost of Ownership (TCO) and Total Effort of Ownership (TEO) highlights our strategic approach:

  • Reduced TCO: Our on-premises infrastructure results in lower long-term costs, outweighing initial investment expenses.

  • Simplified TEO: Contrary to popular belief, our in-house management of infrastructure has streamlined operations, reducing the effort and complexity traditionally associated with large-scale data processing.

Embracing Decentralization and Regionality

Our commitment to decentralization enhances our system's resilience, while our focus on regionality ensures that we bring services closer to our customers, reducing latency and improving user experience.

Operational Excellence

From hypervisor setup to blockchain node deployment, our operational blueprint is a testament to our commitment to excellence:

  1. Efficient VM Management: Using cutting-edge tools for rapid deployment and configuration.

  2. Robust Blockchain Node Operations: Tailored setups for optimal blockchain interaction and data processing.

  3. Comprehensive Monitoring Systems: For real-time insights and proactive management.

Conclusion.

DexGuru's shift to an on-premises infrastructure is more than a cost-saving measure. It's a strategic move towards achieving peak performance and control in data processing, positioning us as a leader in the field of data analytics and infrastructure management.

Subscribe to evahteev
Receive the latest updates directly to your inbox.
Mint this entry as an NFT to add it to your collection.
Verification
This entry has been permanently stored onchain and signed by its creator.