Heads in the cloud
Posted Nov 5, 2014
We’ve never had a single AWS instance, and never put a bit in a cloud-hosted server. These days that’s a little anachronistic. Surely the cloud is essential to scaling a modern business, cheaper, and oh so much cooler?
There are a few reasons. This morning, at the Web Summit in Dublin, I went through some of these. Let’s recap quickly:
- Applying AWS prices to our current stack, their projected prices are 5x our current running costs.
- Cloud pricing is disassociated from the underlying costs and actual performance.
- Managed services lock you in to proprietary frameworks, often making migration impossible, and hampering your IP development.
You can download the entire deck here, but if you prefer, we’ll go through these key points one by one.
A question of cost and its costs
Let’s start with the top. Our current stack looks something like this:
We’ve built this up gradually over the years from just a handful of servers, keeping a solid scaling schedule and monitoring the traffic coming in to make sure our forecasts are on track. This is a little bit of work, yes, but the rewards are handsome. If we build up a hardware equivalent in AWS’ price calculator, it comes out to a massive $ 2,300,000 - compared to the roughly $ 450,000 we spend every year.
Obviously outsourcing anything will increase your costs compared to doing it in-house, but five times can be said to be pushing it. In fact, we tend to find that cloud costs seem completely disconnected to the underlying costs of infrastructure:
This can also be expected, but when these costs are the size they are, this will distort your internal incentives. In our engineering, we can directly relate performance optimizations in our software to decreased hardware requirements, and thus lower costs. We’ve just completed a year-long scaling project where our IT ops, database engineers, and backend teams worked together to bring down key resource hogs while simultaneously increasing hardware capacity.
If that link is broken - by disconnecting your hardware costs from the underlying resource requirements of the software - your incentives are suddenly distorted. Do you make it run faster and leaner, or do you cut corners to save some money and put that toward more hardware elsewhere? That trade-off is a construct of the cloud business.
The managed service conundrum
What about managed services, like Redshift or BigQuery? Well, when you build your stack into a vendor’s particular system, you wound up building software to interface with those proprietary frameworks and wrappers. This has two major concerns.
First, the dependency you’re constructing is beneficial for your provider but not so much for you. Lock-in is common among many software providers, but the issue is exacerbated when you build your own products on top of proprietary foundations, as opposed to, say, collecting e-mails for your newsletter. Migrating an app designed to interface with a proprietary database, with its own quirks and benefits, is difficult in the first place but a nightmare in production. What if they shut down or force a breaking update?
Second, if you’re running a software company, your product is software. This is what your clients buy into, but perhaps closer to home, that’s what you’re investing in. You’re building intellectual property, and when this IP is leaning on a single provider, so is your entire business.
Our feet on the terra firma
So for us, the question was a no-brainer from the start. In addition to these things, in a business where you are responsible for safe-guarding and warehousing sensitive data, cloud hosting can not provide a guarantee to where and how your data is stored. That’s not just key for our clients, but for their users and the jurisdictions that we operate in.
Some companies have found and will find ways to alleviate the issues we outlined above, and others yet don’t have an issue with this in their model. There is space for cloud hosting in many verticals and it does bring down the barriers to entry to start and run a tech company today - many of our friends, clients and colleagues’ side projects rely on the cloud.
Yet the hype doesn’t really cut through the fog, and there’s still an awful lot of heads in the cloud.