We do a LOT of cool things at LeadSift behind the scenes, which makes Social Lead & ROI Generation possible. Some of these things are innovative, creative, brilliant and they’re mostly all open source. Lots of hours of research, trial and error, head scratching and all night reading of cryptic forums went into creating our system. So we’re going to write about our technology, to share what we do, look for feedback and hopefully help others architect their applications and make them highly scalable without spending all their budget.
Today, I will talk about our hosting and platform. When we started, we were asked by many – why not use AWS? We love Amazon, but it’s expensive. Also, it does not give us as much control as having our own platform. So we set up our “Private Cloud”.
We use ProxMox VE as our virtualization platform, it is fully open and based on Debian GNU/Linux which are two great reasons to use it. ProxMox provides many of the capabilities available with other VE Platforms, like live VM migration, filesystem snapshots of VMs, automated backups of VMs, the ability to use both openvz containers and kvm containers (for when you need full hardware virtualization).
Paying a little over a thousand dollars for 3 hex-core servers, 30TB of bandwidth, 48GB of memory, unlimited IP’s, TB’s of disks configured in Raid6. It takes us all of 15 seconds and 0$ to spin up a new Virtual machine if someone wants to dabble with something new. We have over 50 instances running right now and are always deploying slices, whether for dev, staging or production. The bandwidth seems overwhelming but with our aim of processing the world’s social data – we’re sure to use it all up!
That price also includes a backup server that is sitting in a different co-location, hundreds of miles away from our primary location. The networking setup was a delightful piece of the puzzle. We have our primary servers behind a Cisco ASA and we are communicating with the backup server via the private VLAN provided by our hosting provider to not use up any of our bandwidth. We additionally have an open-vpn setup between our backup server and our primary cluster to add security to it all. The backups on our remote server are pulled weekly in nightly chunks into our office in Halifax, NS. This gives is even more redundancy and disaster recovery capabilities. We have an additional dedicated 5TB of bandwidth allocated to the backup server for the pull back to Halifax.
Did I forget to mention that the backup server is also running ProxMox under the hood (why not!), its a single node virtual cluster, which we can easily expand at any time when we need to. By using ProxMox on the backup server, we are able to have remote replicates of our databases, which we actually use for our backup dumps.
To go with AWS, we’d be paying $$$ and have our hands tied with a lot of the tasks. So guess what, we have our own private cloud which works extremely well for us. If you want to know more details about our to setup or to help with your setup, ping us any time!