Or, "Old Man Yells at Cloud Hosting".Currently I host all of my various web sites and services (business and personal) on a single physical Linux box leased from a small ISP. That's been working out fine for me for many years. However, of late my ISP has become so non-responsive to support requests that I can't interpret that any way other than, "We are no longer interested in being an ISP".
So, sadly, I think I need to find new hosting.
My initial instinct, because I am an unfrozen caveman, and your modern ways frighten and confuse me, is to find someone else to rent me a 1U server. But those appear to be nearly nonexistent these days, because in this modern world, That Is Simply Not Done.
(And though there are a few companies that do this, the fact that there are so few makes me suspect that they won't be in business at all a couple years from now.)
So I'm looking at virtualization options from Amazon and Digital Ocean and whatnot, and it's all very confusing. In particular, figuring out how much my performance, storage and bandwidth requirements are actually going to cost is absurdly complicated.
Here's what I have now, that I need to replace with equivalent-or-better:
- 8 core 2GHz (faster would be nice)
- 32 GB RAM
- 2 TB disk
- a backup of that disk
- Outbound bandwidth: 2 TBytes / month
Right now I'm paying $300 / month for that.
So it looks to me like Digital Ocean and Amazon are the closest replacements for that, but it's hard to tell. I think that the closest options are:
- Digital Ocean: Their $240 option (32 GB RAM, 4 core, 90 GB storage, 7TB transfer) looks close, but that's not nearly enough storage, and they charge $100/TB, so that brings it to $440/month. Also they don't specify how fast those cores are.
- Amazon: The m4.2large option (32 GB RAM, 8 core, no storage) looks close, at $258 (paid a year in advance); plus 2TB storage at $200/month. So that puts us at $458/month already. They also don't specify core speed, and that's for 1Gbps, and if I'm doing my math right, 2TB/month is 6.1Mbps, and I can't even tell what they charge for that.
It blows my mind how much they want to charge for disk. I can buy a physical 2TB drive that will last 5+ years for $80, but these jokers want me to pay $200/month for that. That is a markup of 150x! What the hell!
Options I am not interested in:
- Try to get 2TB of bandwidth into the DNA Lounge building and manage my own high availability data center, for a single server.
- Rewrite 20+ years of code that is in a variety of languages into whatever toy-du-jour Google insists things be written in to take advantage of their hosting option.
- "Virtual hosts are great because when you realize you need more, you can instantly upgrade them and start paying more!" I know this excites many of you, so much, but this feature is completely uninteresting to me because my requirements are static and because I prefer to know what I'm paying ahead of time.
- Also, just to head this off too: I realize that many people are of the opinion, "You should run each web site, and each service, on its own virtual server", and while that sounds sensible to me in principle, it sounds less good when each of those virtual servers is an additional $250/month. At that price, I'll run all of my sites out of the same Apache instance, thanks.
So what should I do?
We colocate our hosting business at discourse.org for $400/month/rack at https://he.net and I can highly recommend them. For that price you get a whole 42u locking cabinet, 15 amps, and gigabit unmetered internet.
Another option, if you like the many (quite amazingly fanatastic for the price, actually) mini-PC boxes on Ali Express per https://blog.codinghorror.com/the-scooter-computer/ you can colocate those for $29/month at https://endoffice.com/ -- I host a few sites of mine that way. The performance specs are not as good as a "real" 1u server but they absolutely destroy what you'd get from Digital Ocean or Amazon EC2 for the same price...
https://meta.discourse.org/t/how-much-is-discourse-affected-by-a-faster-cpu/58407
+1 for he.net. Granted, I just use a cheap $10/mo shared hosting plan, but they've been reliable and responsive. It'd be worth a moment to fill out their colo quote form and see what comes back.
+1 for colo or dedicated server. HE is well known to be a good one. I've also had good experiences with these folks:
https://www.hostdime.com/managed-dedicated-servers/
I remembered Maciej at Pinboard as someone else fiercely intent on co-location; here's a post he wrote about it in 2012:
https://blog.pinboard.in/2012/06/going_colo/
The post mentions Hurricane Electric, which is why I'm posting this as a reply to a comment.
Maciej lives in the Bay Area, and he's somewhat approachable and snarky on Twitter. I think you two would get along.
I like to think that Maciej and JWZ know each other and go bowling on Tuesdays.
I use Dreamhost for dedicated hosting. They're venerable, have great customer service, and are squarely within your price range. https://www.dreamhost.com/hosting/dedicated/#pricing
I would not use Dreamhost. Dreamhost was good a decade ago, now not so much so. I started using them after I lost my free colo box circa 2005. Initially everything I had was on there.
Then they started slipping, and I had tech support exchanges like this -
Me:
Dreamhost:
The problems with this response is:
*) alot is not a word
*) I said a "load average over 2", he says "a load of 2". > != =.
*) his last sentence is complete nonsense
*) my problem was not with a machine with a 2 load average, but a machine with a 33 load average.
Here are some of my nagios monitoring logs of my now very limited Dreamhost setup (Most has been moved to Linode, Rackspace and Ramnode).
Here's my nagios logs from less than two weeks ago:
Host Down[2017-06-11 10:16:25] HOST ALERT: dreamhost;DOWN;SOFT;1;PING CRITICAL - Packet loss = 100%
Service Critical[2017-06-11 12:24:45] SERVICE ALERT: dreamhost;SSH;CRITICAL;HARD;1;CRITICAL - Socket timeout after 10 seconds
Two hour outage. Plus some minor blips since then.
I'm trying to find the time to safely migrate my legacy stuff off Dreamhost. Don't make the mistake of getting on it.
Indeed. I have been a Dreamhost customer for 10+ years but am only still with them due to inertia. These days they are unapologetic about downtime to a degree that is kind of astounding. I've had DH support techs assure me that nothing is down when their own status pages are showing outages. You don't need this kind of headache.
I was referring to their dedicated server offering. You're talking about their shared (VPS) service offering. They're two different products at two different price points.
For dedicated hosting, Dreamhost has a 100% uptime SLA.
Shared hosting is well below VPS service. I agree that one should expect to put up with a lot more on mere shared hosting compared to VPS or dedicated.
avoid dreamhost. when they work, they are fine. when they fail, which is often, they are the most unhelpful and antagonistic fuckers on the face of the earth. they will go down and you will lose customers and they will blame you for it. then they will lose your data. while i would truly enjoy your inevitable anti-dreamhost screeds here in a fabulously public forum, it's not worth it. avoid.
I worry this may not be a ton of help, but can a decent fraction of that 2TB be statically hosted? The AWS S3 product lets you store a ton of data on the cheap (maybe 20% of the cost of EBS?), and buckets can be configured to serve as websites. If so, you might be able to use that to cut out a good chunk of the disk expense..
I don't know what you mean by "statically hosted", but it is the case that only a portion of that 2TB changes frequently. Much of it is write-only, or is only updated rarely. However, if pieces of it are only available to me through weird URLs rather than appearing to be actual files on disk with names and directories of my choosing, that's not gonna work. E.g., all of my http roots have a .git repo in them.
You can mount S3 buckets like disk, but it isn't as fast as disk. I don't know what your performance requirements are.
By static content he means the webserver can just shovel the bits off of disk and serve it up as opposed to having to run some code.
(Images/css/scripts)
You could set it up so those unchanging files are served from outside EBS while still keeping them in git or what not. But it would probably require changes to your work flow so not an option, I'm guessing?
Just to be clear here: what's being explained is that you can point people going to your site directly at S3 for static content, you do not need to serve it up off your own site.
There is no way to switch to using S3 for media without breaking historic URLs. See this thread.
I use Dreamhost. It works for my site's requirements, which are trivial. You might look into their managed hosting:
https://www.dreamhost.com/hosting/dedicated/#pricing
Why not go back to a dedicated server and not go "cloud"? What you want can be had for about $100 monthly in a West Coast datacenter like Los Angeles.
Better yet, get a dedicated server and install a hypervisor like KVM or XenServer, which are both free if you want virtualization.
Your theories and price quotes are useless without links. Dear everybody reading this comment: don't make this comment.
Ugh, I replied to the Michael and not you, see below.
I see that LiquidWeb has a dedicated server option; when I monkeyed with it (remove SSDs, put in single 2TB SATA HD and 2TB SATA HD backup to get your requirements) I came up with $459.00/mo, with some weirdness where you can reduce that rate if you pay them $500 up front to set it up. (They do have a virtual option, but your specs are in the contact-us-for-a-quote category.)
Note that LiquidWeb just ate the VPS hosting company for an organization I'm involved with, and more recently did the transition to one of their VPS setups, and while things aren't bad I get the feeling we got a bit more TLC at the now-defunct company.
I have had a number of personal and professional sites on LiquidWeb servers (both VPS and dedicated) for ten years now. They also offer 2U or 10U colocation services.
https://www.liquidweb.com/dedicated/
Their support is not quite what it once was, but is still solid. I hope to stay with them for another ten years.
Have you looked at Linode? I had my site(s) there for a few years now. Only problem I had with them was when their power feed melted their power equipment, they were back within 24 hours though. Response times are fine.
You basically get your own "Linux VM".
Should have added a link I guess: https://www.linode.com/pricing#all
As far as I can tell, Linode only sells toys, not high availability servers. Also they don't have an option for more than 200GB of storage, period.
Their biggest option has 1.5TB worth of storage attached, though the price is.... hefty $640/month.
Sorry, here you go.
Here is one deal @ https://www.quickpacket.com/billing/cart.php?a=add&pid=90&promocode=2HY4NKSJ47 via https://www.webhostingtalk.com/showthread.php?t=1656112
https://www.webhostingtalk.com/ is your friend, especially the dedicated hosting forum @ https://www.webhostingtalk.com/forumdisplay.php?f=36
+1 for Linode.
48 GB RAM, 12 Cores, 768 GB SSD, 20 TB transfer == $320 / mo
The are great and how they work will fit your "unfrozen caveman" frame of mind.
The REAL value, for you, will be the 20TB of transfer you get each month. That's about $1900 from Amazon.
Linode is actually pretty good - I've had one for more than 10 years, and have had literally zero unscheduled downtime other than the very occasional network hiccup lasting a second or three - and very little scheduled downtime for maintenance/etc.
The 8 core regular plan includes 24GB of memory and 384GB disk, for $160/month. Adding extra memory is easy, and they used to have a standard menu for extra storage IIRC, but I can't find it now. It might be worth a quick contact to their sales team to see if they can give you 2TB at a reasonable price.
As someone else noted, they do have regular plans with up to 1.5TB of SSD storage included, so it's not totally off the wall for them. I don't know if they do spinning rust anymore.
C.
I have been assured that you should not use Linode. Their security track record in the public is less than stellar, and many people have claimed that they are much worse than their track record indicates.
FWIW google allows you to start up a GCE instance which acts pretty much the same as amazon's. And their pricing calculator seems to be a little more sane:
https://cloud.google.com/products/calculator/#id=825edb61-362e-4b99-8c50-c45e213efb23
It comes out to ~ $290 per month for an 8 core 32GB with 2TB of persistent disk running whatever OS you want.
https://www.dropbox.com/s/nwd3sx3w4jbyd1o/Screenshot%202017-06-23%2015.40.22.png?dl=0
I would look at biting the cloud bullet in the short term and then slicing off the costs by:
1) Using a CDN like CloudFlare (which has a free tier. There are other options). That will substantially save on bandwidth costs
2) Moving some of that legacy (the more irregularly used stuff) to something like lambda. You can throttle that to prevent surprises, and there's plenty of ways to wrap legacy code these days.
3) Moving media assets to blob storage (which, to be honest, is where they should be, and you get redundant storage to boot)
Mind you, after decades of tinkering with websites in various capacities I've become that strange new creature called a cloud architect, so I might just be pulling a Dogbert on you:
But I will do free consulting for the one guy who called bullshit on groupware. Your move.
Hey Jamie-- for what you want to do (nothing super special, a lot of php as I recall?) go ahead and grab a box from Hetzner.
A 64GB/2x4TB box runs you 49E per month.
Then, sign up for the Pro Cloudflare Plan and run the website through that. DDOS protection, CDN, web firewall, and they do some really cool things to speed up your http traffic without needing to do a crapload of configuration on your end. The pro plan is $20/month/domain.
Your initial php http requests will go to the Hetzner box, and all of your images/javascript/css/etc will all route through Cloudflare, so the net effect is that the site will have a little bit longer time-to-first-byte (speed of light to Europe) but then the actual DOMContentReady event will fire much, much faster.
Aye, am perfectly happy with Hetzner. The tumbleweeds blowing through the XEmacs web site, together with the rest of my hosting needs, are hosted on a VPS for € 11.90/month (VAT not applicable), and Hetzner’s reliability has been excellent.
This dedicated server is the first one they offer that has more than 1 TB of storage; comes to 69 € per month. But for the west coast of the US you might have to consider Cloudflare on top of that, as Mark says.
Once I followed best practices for email servers (DKIM, SPF, rejecting unknown domains at the MTA level, accepting SMTPS, filtering so I’m relaying as little spam as possible to people who have their accounts forwarded) I haven’t had issues with my mail getting through. No experience with DDoS so far.
How hard a requirement is it that your storage be durable and external to the instance, and do your storage i/o requirements mandate SSD or can you get away with spinning metal?
If you trust your backup solutions, there are a few amazon instance types that seem to meet your requirements or come close. Most notably the i3.2xlarge has 61GB RAM, 1.9TB SSD storage and 8 virtual cores for $290/month if you pay for a year fully up front.
Alternatively, you could seriously economize by using a t2.2xlarge: 32GB RAM, 8 cpus ("burstable" -- basically you build up cpu "credits" when the box is idle and can spent them when it gets hammered) at $183/mo reserved, which would leave quite a bit of overheard for bandwidth and EBS storage.
The hitch with all of the instance types with built-in storage (as opposed to using EBS) is that it's not durable: if the physical machine your VM is on hiccups, it's all gone and will never come back. On the other hand, bandwidth between EC2 and S3/Glacier is really cheap, so if you trust your backup scripts you can usually live without EBS and its fancy snapshots.
Amazon's product pages for their EC2 instance types are, as far as I can tell, deliberately terrible. http://www.ec2instances.info/ is good for quickly comparing prices and configurations.
Since in my mental model of this I only ever have one "instance", I barely understand this question. So the locally-attached storage is basically a RAM dump of a preconfigured post-boot Linux installation or something? I can expect my "instance" to randomly appear to have "rebooted" or even been reinstalled from scratch? What does "boot" even mean for these things? How does init.d work?
An ec2 instance is a VM. You start with an Amazon Machine Instance (ami), and tell it what hardware to run on. The ec2 instance has storage as /dev/sda. The way you are supposed to do it is treat /dev/sda as destroyable. Seperately, you create an EBS volume, which you can think of as like a portatable hard drive. Then you “attach" that EBS volume to your ec2 instance, and you get /dev/xda which you can add to /etc/fstab. EBS volumes are snapshot-able, backup-able, etc.
There is also EFS which like EBS but can be attached to multiple ec2 instances simultaneously.
As others have pointed out, there is also s3 which is significantly cheaper than EBS. So if you have 500GB of images that you are referenced from your site, you could put them into an s3 bucket, turn that s3 bucket into a webserver via aws tools, change your dns so it is img.jwz.org, change your image urls, and you are done.
There are ways to get an s3 bucket into /etc/fstab, but I have never done that. You can mount s3 buckets on your workstation (and probably your phone).
So you create an ec2 instance from an ami and give it the minimum storage the OS needs, like 8GB. All your data goes into EBS or EFS or s3. To upgrade your OS you create a new ec2 instance from an updated ami, “detach” your EBS volume from your old instance, “attach” it to your new instance, and then delete your old instance.
ec2 instances do fail and sometimes get shutdown intentionally by amazon if the underlying hardware has an emergency. As long as your data is in EBS or s3, you are fine.
So if I understand this right, a way to reduce the cost of the 2TB cloud block device acting as backing store for an ext3fs is to instead upload all of my files into this S3 thing, presumably though some custom uploader that is not rsync, and then once I do that, they don't have names or hierarchy any more, so for example the <IMG SRC="https://cdn.jwz.org/images/yellcloud.jpg"> up top has to be rewritten as SRC="https://random37.amazon.com/R3FmvdMxdt7AsW4JsyAZ". Is that right?
No, that is not correct.
If you store yellcloud.jpeg in your "img" s3 bucket, you can turn your "img" bucket into a website that will get a dns name like s3garbage.aws-amazon-garbage.garbage, then you change your dns so that img.jwz.org points to that s3 bucket, then http://img.jwz.org/simpsons/grandpa/yellcloud.jpeg will serve up that file. You can also use their cloudfront dingus as a cdn by checking a box.
Yes, rsync will not work even though there are many attempts at it. There are a million uploaders for s3 that are stable and work fine, it's more or less a fixed public API at this point.
If you need rsync instead of the weird non-filesystem thing that s3 is, use EBS.
Do not store your data on your ec2 instance's storage, use EBS or EFS or s3.
S3 stores objects in something that looks a little like a filesystem. Most static web page parts fit really well in S3, but as you suggest, you may have to either use arcane s3 URLs or sell your soul deeper to Amazon by letting them manage your DNS, too.
Still, it's not a bad way to handle things like images. There is an 's3sync' tool that runs sort of similar to rsync. You should probably ignore any temptation to use layers that convert s3 into a filesystem, though.
EBS is Amazon's version of iSCSI or SAN storage. They carve off a block of bytes of any size you desire, they show up as disks, and you can put whatever filesystem you want on them.
Instance storage actually are disks that live in the host. They can be faster, but they disappear if the hardware dies.
You can actually even keep it in-domain: CNAME images.jwz.org to jwzs-awesome-image-bucket.s3-website-us-east-1.amazonaws.com, activate S3 website hosting on your bucket, and et voila: http://images.jwz.org/foo/bar.jpg will serve whatever is currently stored at s3://jwzs-awesome-image-bucket/foo/bar.jpg
The AWS CLI tool has an "S3 sync" command that functions mooooorrrrrrrre or less like rsync.
Eh, that still means that historic URLs have to change, which is horrible. Yes, I realize that I am one of the few people in the world who remember what the "U" is for.
I expect SSL is a whole other set of fuckery.
You could set up 302s for them on your main server, but yeah: this definitely falls under the heading of Fucking With Shit Which Is Not Itself Broken in order to fit someone else's business model.
Your expectation in re SSL is, of course, completely correct. S3 hosting doesn't support SSL unless you mate it with their Cloudfront CDN, and while this is 100% a thing you can do, I can't imagine that you'd want to.
Having thought about this for a few more hours, my gut recommendation here given your understandable lack of interest in becoming any more of a sysadmin for this stuff than you already are is that you could call one of the hosting companies mentioned here (prmgr gets good reviews from my friends) and see what kind of deal they can cut you.
Hosting on EC2 is better in a lot of ways, but not unless you're prepared to spend some time learning the amazon ecosystem or better yet delegating that task to someone like me who's weird enough to enjoy this shit.
Yes the URL which eventually serves the data has to change. But updating the links you control to img.jwz.org, and adding pattern based permanent redirects for image URLs to the http://www.jwz.org config (pointed at img.jwz.org) to handle external links can paper over most of the disruption.
I too remember URLs as "non disposable". But as someone else said, redirects are the record of changes in design choices. (Serving images and other static content separately is very common in more recently built sites. But uncommon 15 years ago.)
Ewen
PS: a reasonable choice would be to just do S3/redirects for large files, and leave small ones alone. Plus perhaps put new static content on a new domain that can be moved later.
init.d works in the normal way, but local storage in EC2 comes in two distinct flavors:
- instance-local storage: somewhere underneath your instance is an actual racked physical machine with a bunch of RAIDed disks attached to it, running the Xen hypervisor. One or more slices on that RAID are turned into a virtual disk device and attached to your virtual machine. Your root system image (/boot and the bare OS) is (mostly) always on instance-local storage. Some instance types provision more instance-local storage; up to several terabytes spread across several virtual volumes.
- EBS ("Elastic Block Store") storage volumes are virtual block volumes living on some enormous SAN-like backend, and can be attached/detached to your ec2 instance via the amazon console or via the APIs.
EBS volumes have a life entirely separate from your VM: if you destroy the VM, the volume is still there and can be re-attached to a new or different VM. Since they're on a SAN-like thing, they offer copy-on-write snapshots as a feature. The downside is that using EBS is an additional charge (per gb per month) whereas the cost of the instance-local storage is bundled into the instance cost.
Instance-local storage is only persistent for the lifetime of the VM. Just a reboot doesn't kill it -- you can do that all the live-long day -- but if you pause or terminate the VM from amazon's console/API, the local storage is garbage-collected and goes poof. And if a fatal hardware failure happens on whichever unlucky server you happen to be hosted on, there's no guarantee that amazon will be able to preserve or recover any data stored there. This doesn't happen often but it does happen.
I'm a fan of OVH. Their server search seems to indicate that you can get a comparable box in terms of CPU/RAM starting at $85/mo, but I'm not sure about the out the bandwidth out or backups. They're much more affordable than DigitalOcean and AWS, although I can't vouch for their support. Last time I checked they don't offer machines in the U.S, but they've got boxes in Canada.
There are definitely strong "economy of scale" effects going on in the hosting business. Until about a year ago, I ran a hosting business in Taiwan. When Google added a data center in Taiwan, we could not compete with their pricing and shut down our service. So it's tough to be a small guy (and small these days includes people like Rackspace), so it's reasonable to be worried.
We manage a lot of applications for entrepreneurs and startups. Cloud is good for flexibility, but if you are really using your hardware, it gets expensive fast. Dedicated servers are much more cost effective, and we manage a few dozen at various providers. OVH is generally very good value for money. It's a "rack'em and stack'em" service, but they have it well automated and it works fine. I would recommend them for you.
+1 for OVH. Servers in Canada are not usually a problem for American customers. Outbound b/w is rate-limited but flat-ra almost all cases AFAIK.
Incredibly cheap.
Not inherently HA (but you can build it yourself, or use their cloud offering).
Support is OK, not stellar.
Just occurred to me: OVH's Junior brand probably has everything you need... Go check out the pricing at SoYouStart.com.
People, please. When I have ruled something out, there are reasons.
When I say "stop suggesting hosting that is not in the same country as me", among the several reasons are that I have no interest in having the lovely "learning experience" of finding out what kind of technical and legal pitfalls exist when I try to process credit cards on an IP address that is not in my country.
How much you love the company has no relevance to this.
Sorry, I hadn't seen the follow-up post yet when I wrote that.
You don't use the phrase "dedicated server" so maybe you've not realised that's the term the industry mostly uses (and why would you, it's something you'd hopefully only have to shop for every half decade or more).
Something like HOST-128H from https://www.ovh.com/us/dedicated-servers/hosting/ seems to hit all your specs, off the shelf, for $160/mo. OVH just happened to be the first hit that I know are reliable, and that I didn't reject for ethical reasons (hello g*da**y) or because the wanted $$$ for disk because the only advertised SAS (srsly rackspace).
Only caveat is they're hosting in the EU not USA. That might even be a plus for you, but if not then maybe someone can chime in with recommendations for US-hosted, and probably cheaper. DS at scale is still a sizeable, competitive business in the EU, so of course those companies mostly shout about their buzzword-compliant cloudy options, because there's more profit to be made from people who want 'a cloud' than 'a sturdy beige box, wedged in with 40 identically dull machines'.
the hatterOVH has a major facility in Montreal to take advantage of cheap hydro-electric power, so they're not EU only. I've had a server there for three years and while I was concerned that their systems were too cheap to be any good, it's been pretty much flawless. The only down-time I experienced was during the giant DNS DDOS black-outs that affected everyone and even then it was brief.
Their trick is they self-build their servers, they're not using vanity hardware, and when their systems are obsolete they refurb them and shift them to their second and third tier house brands. It's all custom-built since they're buying in units of tens of thousands.
OVH has a data center in Canada now, which we have used. And I think they are opening in the US East coast soon as well.
They're acquiring new datacentres in the US (Oregon and Virginia) to build out there: http://www.newswire.ca/news-releases/ovh-global-expansion-continues-with-new-us-data-center-614606244.html
These aren't for leased hardware, though, they appear to be for other services, at least for now. Given the scale of their Montreal operation (~360K servers) they might have to build a substantial building before they can open up in the US.
I use Gandi for some stuff; don't know if you'd be happy with your box in Europe, though; calc is over at https://www.gandi.net/hosting/iaas/buy?memory=32768&v4_public=1&ip=1&disks_nb=2&disk_size=2048&package=pack_2M&cores=8
Their disk prices are a bit more reasonable and you get 2TB of traffic/mo before charging kicks in. With 2 x 2 TB drives you'd be paying about $450/mo; one drops that to ~$300.
Joe's data center's top tier dedicated hosting option meets or beats all those requirements. $95/mo.
https://joesdatacenter.com/dedicated-servers/
I mainly used their $45/mo mid-tower colo option. Very useful if you want cheap storage online. They were super helpful even when my stuff kept catching on fire, thanks again Joe.
Crap missed the storage backup requirement: my bad sorry. Get two? Still cheaper.
a little hint: https://aws.amazon.com/ec2/sla/
and look at the fine print: " that result from failures of individual instances or volumes not attributable to Region Unavailability". i.e. you need to run at least two instances to get to 99.95% sla uptime. That's the same for all cloud providers, google azure etc. So if you're now running a root server and that's been pretty stable, moving to a cloud and just running a single instance will guarantee you headaches.
The alternative is to use a managed service to run your website (e.g. platform.sh comes to mind) and leave the rest on a VM. ymmv.
H.
So, you're saying that if I run only a single instance, I'm screwed, but I don't even understand how I do what I'm trying to do -- impersonate having a single computer -- while running multiple instances. So, what, now my cron jobs all go off twice, at once? And PHP sessions don't work at all unless I've outsourced my web service to a CDN? And my apache log files go... where?
This all sounds unspeakably horrible.
What's the SLA on your single Linux box?
I thought I was pretty clear about that. My SLA is "we no longer answer our email".
That's sort of my point. You're not any MORE screwed if you run a single EC2 instance, but Amazon also doesn't answer their email and they are unwilling to start cutting checks if they miss 99.95% on that single instance.
There are solutions to running PHP websites and many other things in the way you're describing here, but not a real Linux VM with a shell.
There are, naturally, cloud services answers to all of those issues. Eg, Amazon Elastic Load Balancing, Amazon CloudWatch Events time scheduling, Amazon answer on centralised logging, .... it just depends on how far down the rabbit hole you want to go. Cloud Native (tm) is basically the buzzword for "we're so far down this rabbit hole we have no idea where we came in".
I suspect you probably just want to find another dedicated-server hosting provider that is more responsive. And accept that "box does, everything goes away". AFAICT a single instance cloud VM will be (somewhat) less reliable than a physical box in a rack -- because it has more dependencies on more virtualised things, and because cloud providers tell you to use multiple everything to ensure service uptime, so they're... not going to fight as hard to keep any individual instance running all the time. (Somewhat less reliably might be acceptable if it cost a lot less; but it seems like for your use case "cloud computing" will not cost less without major refactoring of how you do things.)
Ewen
You can run a single instance. No issues.
You obviously already have a backup strategy and can deal with "lightning fried my box" right now. So you just need to continue to do that in AWS.
Your current system probably doesn't have 99% uptime anyway right so you're not losing anything.
(disclosure: I work for the folks in Redmond, so I'm using the Azure terms here, but you will find the same stuff on Google, Amazon, ...)
The short answer: you run two instances, slap a load balancer with session affinity in front (thus sessions will work) and you move anything that needs to keep consistent state into a service (e.g. databases). If you can live with your cron jobs occasionally not firing, leave them on one of the instances. As S3, Azure Blob store etc. are all accessible via HTTP, you can build a "poor mans CDN" by referring the clients directly to storage for static content. Still, this will be about twice the price you pay now. The cost advantage in cloud over hosting typically manifests only if you have a service you can scale down at times.
The long answer. Move the website portion to the hosted web service of your cloud, e.g. azure web apps. There's a linux-hosted version of web apps, so the difference to your current setup should not be big. Move the source code of your website to github (or visual studio team services) and point the web apps to pull the source from there. Move your database to a hosted database service of your cloud provider. Get a sendgrid subscription for outbound e-mail (you know that most cloud ip's are blacklisted ?). Get a DNS service from your cloud provider. For everything that does not fit, run a small single instance VM. That might be going down once a while, but it won't lose data since its VM is kept in a redundant storage array. Use the snapshot mechanism of your storage system and/or the cloud storage provider for your backups (e.g. you can mount Azure Files and just tar there). Slap the CDN in front of your web service, it's the same outbound price you're paying anyway but the CDN nodes are closer to your users. You may pay a bit more for the origin traffic of the CDN that hits your server, but it will be way cheaper that having the traffic hit your web service.
Tune the scale of your web service. Probably you'll end up with less money spent and less hassle to keep it up and running (e.g. the web server runtime and php environment will be updated automatically).
But then again this will cost you a couple of weeks to get everything running smoothly, so maybe it's better to do that as part of an overhaul of the website and run things in parallel for a while.
It's probably a good idea to find some help for the initial setup. There are a lot of tiny issues that aren't well documented and somebody who did this already will probably know them.
H.
Absolutely. It is 100% worth just finding someone you know who's familiar with, e.g., Amazon's offerings and to have them write a Cloudformation script that sets all of this up once, and maybe call them back two weeks later when inevitably the thing you or they didn't think of happens.
If it's a good friend, they might do it for some beers and dinner and maybe a ticket to a show; if it's a less good friend, you should pay them for their time.
I deal with AWS stuff at work, and every time I try to do something new, it's always a minimum of a few hours of reading docs, figuring out where they're not quite correct or misleading, trial and error, etc :/
So, "Rewrite absolutely everything, so that it now all depends on at least half a dozen third party external services, then spend a year optimizing it for this new interconnected maze of failure points. Also break all your URLs. Simple!"
Then take bets on which of those half dozen third parties will have vanished or CATD'd their APIs into incompatibility within three years.
Yup, I think "unspeakably horrible" just about covers it.
There's a vm type (D with premium storage) on Azure where there's a single instance SLA: https://azure.microsoft.com/en-us/blog/announcing-4-tb-for-sap-hana-single-instance-sla-and-hybrid-use-benefit-images/ is the blog post describing it. That will cost you about 1000 USD/Month (D4 V2 VM, P40 disk and 5TB data transfer). Add developer support (8h, e-mail for 29 USD) or standard support (2h, phone for USD 300)
That gives you a 99.9 sla (or 8 h 46 min of downtime per year)
Don't know if AWS/GCE/Digital Ocean has a similar offering.
It is a complete shitshow, but you have to have depend on something, and at least with Amazon, you know they'll never answer your emails going into it.
And why would URLs break?
Because everyone says "you need to put all your media URLs into S3 to save money", but if you do that, you have to move those media to a different sub-domain.
The AWS cli tool has an rsync like feature to import/sync filesystems. No idea how your media is currently laid out but if you have some kind of organization/segregation of your media, then you could create S3 bucket(s) for those directories then create an nginx (other http servers are left as an exercise for the reader) proxy rule that redirects media requests to S3 and you'd not need to change any URLs. The nice thing about this approach is you can migrate at your pace, before you even move ISPs.
There are plenty of nginx/S3 proxy rules for the googling. Here's the first one that came up.
So, you don't actually have to change to a subdomain to put images into S3.
You can point your hostname at Cloudfront, and have it configured with different "origins" - so http://www.jwz.org/images/* is served out of S3, anything that doesn't match one of the patterns hits your default origin, which points to your EC2 instance.
Try giving Monkeybrains a call, their main business used to be colo before they got into WISPing I think. Plus they are awesome humans and staunchly local
Seconded, used to colo with them and they are a tiny company of cool weirdos who treat their customers like human beings. https://www.monkeybrains.net/colocation.php
I agree with Jeff that colocation for your own hardware is still a real and viable thing. These folks bought out my original colo provider and seem to be still going strong:
http://managedway.com/rackmount-server-colocation
FWIW I lack any complaints about their pricing or service, but my needs are modest.
If you are leery of maintaining your own hardware, dedicated hosting is still around. Others have mentioned Dreamhost, so here's a competitor (unfortunately I don't have any experience with this company):
https://www.a2hosting.com/dedicated-server-hosting/unmanaged/compare
Also, how does DNS work on these services? If I ask questions like, "can I get a static IPv4 address" and "will someone run a stealth master for my domain tables" will people just shake their heads and say, "Oh, Caveman..."
You get static IPv4. If you're renting actual hardware (and not a VM) you probably get at least a /29 block with 5 IPs for you to use. Anyplace decent should give you your own IPv6 prefix as well.
I've never done DNS with my web host (outside of old shared hosting). I have my domain registrar handle DNS for me (lots of control over the DNS records) but if you have more interesting requirements, you might want a separate DNS host (or host it yourself, I guess).
For EC2, getting a static IP address is simple and not terribly expensive: they're called "Elastic IPs" and are free as long as you (a) attach no more than one EIP to one EC2 instance, and (b) do not leave any EIPs sitting unattached: https://aws.amazon.com/ec2/pricing/on-demand/#Elastic_IP_Addresses
Amazon will very happily run a DNS server for you (making it a stealth master is entirely a personal configuration choice) for not very much.
Generally speaking, you can get a small block of IPs for dedicated servers just by asking. These days with IP addresses running out, you may need to justify why you need them. OVH has lots of IP addresses, other providers can be really stingy.
The provider will give you reverse DNS. If you need to run your own DNS, I recommend http://www.dnsmadeeasy.com/ or Amazon Route53 DNS. Amazon's DNS has some nice features for people hosting in AWS, e.g. ALIAS records which allow you to point example.com at a load balancer or S3 bucket and have it work in the face of failures.
This is still an option. OVH is a big player already mentioned in other comments, you'd probably be fine with them. I have personal experience renting a box with ReliableSite.net. I know the name sounds a bit cheap but I've rented a box continuously from them since August 2014 and it's been great. The rare support requests I've had to make (last one over a year ago) were answered quickly as well. Your requirements of 8-core 2GHz, 32GB RAM, 10TB/mo, can be met there for about $175/mo (with two 2TB drives in one server for just disk failure backup) or about $209/mo (for one 2TB drive, and 2TB of FTP backup space outside your server). You can, of course, tweak these a bit for a faster processor, or less backup space, or more RAM, or whatever you like.
Sometimes cheaper and easier are Linode/Digital Ocean/etc who sell VPSes. You pay more for the convenience of easy scaling and setup/teardown. I use Linode for a few smaller sites (think $5/$10 per month each) and it's great for that. It sounds like you'd be better served renting dedicated hardware, though.
Amazon just has crazy scary pricing that IMO isn't worth the trouble for most non-Internet-based businesses. Amazon does have a VPS service (like Linode/DO) called Lightsail, but I prefer the former options.
I can't make sense of how Lightsail differs from Amazon's other options... Is it the same thing, just in a bundle instead of a la carte?
Pretty much. Lightsail is the AWS gateway drug to pull people who are familiar with Linode/DO into the AWS ecosystem without immediately having to know how to configure all the AWS parts. Lightsail is how Amazon sells empty baby servers with set cpu/memory/disk together.
I mentioned Lightsail only for completion, in case you (or anyone) really wanted to use Amazon to host your VPS for some reason, but were getting scared off by all the stupid AWS terminology. I still think other VPS hosts are better, though, and I still think your use case probably needs something beefier than any VPS anyway.
No specific colo recommendations though I'd advise to watch out for those that require use of mandatory "sticky" features which interfere with migrating away from the colo at some time in the future.
As for huge markups, my isp wants to charge $2/mo just to host the dns records. That seems excessive especially when a single dns server can effortlessly host thousands of dns records.
I don't know about DigitalOcean but a justification for AWS to charge more for storage is it provides data integrity and durability that cannot be matched by any single drive you buy. AWS can still experience some downtime so your server and data availability won't be 100% but you will never have data lost or corrupted due to a storage hardware failure within AWS (except maybe S3 RRS. The same is true for the other really cloudy providers like Google Cloud and Microsoft Azure. I expect most people running an EC2 instance who need 2TB storage use faster EBS storage for the OS and applications and mount S3 object storage for web files, media content, etc. 2TB of S3 storage is $46/month (plus little charges for read/writing the data).
https://aws.amazon.com/s3/pricing/
Where AWS and the other really cloudy providers will get you is your outgoing data charges. Pretty much all your traffic coming out of their data centers to the Internet is metered. 2TB is $180/month (I think that's for tebibytes, not bits). For web hosting, using CloudFront or another CDN is basically required to avoid those kinds of charges but if you're doing things a CDN can't help with (e.g. your server acting as a reflector for an HD camera stream), the data transfer bill is going to hurt.
https://aws.amazon.com/ec2/pricing/on-demand/ (scroll down to Data Transfer OUT From Amazon EC2 To Internet)
So forget about all that, your unfrozen caveman instinct is right, you don't really want a to use a "true" cloud platform like AWS. Instead, look at the dedicated host options others have provided or a more traditional Virtual Private Sever (VPS); they're likely to give you a big bucket of data transfer as part of the single price. DigitalOcean is more less in the traditional VPS category but there are probably others who can more cheaply meet your relatively large storage needs. I'm another happy DreamHost customer but I only use their shared hosting and lightly at that; as a DNS provider for your host(s), I think they can do everything you'd want.
I currently pay about thirty eurobucks a month for a Hetzner rent-a-wreck (although they boringly call it "serverbidding"). Since your current spend is ten times that, you can comfortably afford to choose your own new hardware spec instead of being tight-fisted like me and taking whatever crusty old crate another customer has abandoned.
For the spec you list, the PX61 (https://www.hetzner.com/dedicated/server/px61) would seem ideal at sixty eurobucks per month (and give you twice as much RAM and disk and way more transfer), but have a look at the full product matrix at https://www.hetzner.com/dedicated/ in case some of the other configurations tempt you more.
To save this sounding like a sales pitch, I'll give some possible downsides: Their IP space can be a bit of a swamp and feature extensively in RBLs, but they'll swap your IP address if it's a problem and I've been running mailservers there for years without bother. There are occasional routing SNAFUs which can cause occasional bursts of 99% packet loss for 5-10 minutes every few months. Also, being in Germany, it's going to be about 200ms away from your customers. This might still be better connectivity than you have at present.
Customer service is very German and thus curt and efficient. I consider this an upside; some may consider it a downside. Yes, their staff have excellent English, which is more than I can say for the staff at some British ISPs I've had the misfortune of dealing with.
As usual with these, you're probably not going to like the answer.
AWS will do everything you want it to do (and probably not even piss you off that much) with the exceptions of reasonable pricing and cost predictability.
In my experience, the "VPS" companies are in the same sinking ship with the ISPs-who-don't-want-to-be-ISPs, so I guess you can either deal with that or hand your credit card to Bezos.
I'm paying about $8/mo for Prgmr.com's VPS hosting. That's only 1 cpu with 2gigs of ram and 24 gigs of hard drive though. they only list up to 4cpu/15gig/180gig hd at $60/mo on their page, but they hang out in #prgmr on freenode last I looked, so you could probably get custom quotes for larger servers or colocation if you ask them. They have racks at he.net as well. I've been with them for years. They do go down for maintenance, but credit it and give good advance warning.
Linode's another option, but the 2TB alone would be $200/mo there, the VPS is $160 for that number of cores, but that leaves you without a backup for your 2tb.
Hey Jamie,
There's a few companies that offer full 42 racks for 400. Most deals in the 1-4U range I've found range from 50-150 monthly. I was looking for 4U of space a while back, but the most obvious answer I think is to find 10 friends who want to pool on a full rack deal, which would get us cheaper prices.
Anyways, find a bunch of people and we're in business. Happy to chat next time I see you.
It would help to hear a little more about the requirements. Do you need 8 cores because of big spikes in traffic occasionally or is it a constant thing? 32GB of ram being used all the time or just for a boatload of concurrent users?
"The amazon way" of doing this is to have a pool of smaller identical virtual machines that gets grown or shrunk based on load. They have some easy tools to set up load balancing and a system to monitor load and add or remove instances as necessary. Doesn't sound like this is going to be reasonable for old legacy code, but if it is then you're in luck.
If you want to know exactly how much your bill is going to be, or cap it at a certain amount per month, amazon is not going to be a good solution. The bill is always going to go up or down based on usage.
With a big EBS volume you will be paying per I/O request, and also for network traffic, and also for DNS queries, etc. It isn't much per request but it does eventually add up with enough traffic.
I have no idea. That is what I have now and it works. When it was less than that, things were bad.
It sounds like maybe you want to do some monitoring to see how the load changes over time, then decide based on that? It's worth it if it saves a few hundred bucks a month right?
Amazon has it set up so they watch the load on your system and when it gets over a certain threshold ( you define all this ) they boot up additional servers ( with the disk images you create ) and add them to the load balancing pool. Then when the load goes down they shut them back off. They bill per hour, so you could end up paying for a small or medium instance most of the time, and then a dozen of them for four hours a week. That would bring your monthly cost down a lot. The load balancer has a feature called "sticky sessions" to make sure any particular user stays on whatever instance they were directed to at first when you have multiple instances running.
There are less well-known VPS providers that provide spinning storage with a reasonable amount of CPU and RAM for less than an arm and a leg monthly. For example I have a 300 gb/2x2ghz/2 gb ram VPS from prometeus.net that I pay $17 euros/month for. Downside is the company is relatively unknown and tiny, and I'd not trust it for my primary server VPS (mine is on Linode for reasons of trust and reliability). And they have no plans that go up to even 1 TB.
backupsy.com will rent 2 TB of spinning disk for $80, but it has a 512 MB ram limit, and restrictions on allowed server processes.
And so on, there does not seem to be any good provider for a beefy VPS for those of us who would rather not remotely manage a physical server.
I forgot about Bytemark's VPS offering: https://www.bytemark.co.uk/cloud-hosting/
There a VPS with your requirements would cost I think £370/month (not sure how many cores come with the 31 gb of ram that is most of that price.. at least 4). Or $480 current rate but who knows with Brexit. Also they seem to not like encryption in the UK these days and really like the internet spying? But, Bytemark is reputable.
RamNode have been good to me the past several years: At http://ramnode.com/vds.php, they offer 32GB RAM / 8 cores (what speed?) / 1 IPv4 / 800 GB storage for USD 160 per month. I'm sure you could get more storage if you needed to.
AWS has a monthly calculator. I plugged in your numbers and got $237/month using all-up front Reserved Instance pricing:
https://calculator.s3.amazonaws.com/index.html#r=IAD&s=EC2&key=calc-046E4CE2-5C15-4C73-9E1F-18782AD023E1
Note:
Instead of a m4.2large, I used a t2.2xlarge. This has 8 VCPU and 32GB RAM. The difference is that the CPUs on the t2 instance family work with CPU credits instead of fully dedicated VCPUs. See here: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/t2-instances.html
This is what you want if your server workload is not very CPU heavy. I suggest trying this instance size in on-demand mode for a few weeks. If it's adequate, then buy the 1 year all up-front Reserved Instance (RI). If it doesn't work (too much or too strong), stop the instance, change the instance size to something higher/lower, and evaluate that. Once you've found the instance size that works for you, then buy the RI. If you buy an RI w/o knowing which instance size is optimal for you, then you may be stuck with that instance size for the term of the RI. See RI rules here:
https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ri-modifying.html
This site is the best for comparing different instance sizes.
http://www.ec2instances.info/
I mistake I made in my estimate is that I used the Virginia region. Since you're on the West coast, use the Oregon region. The N.California region is also an option but it's a bit more expensive.
Also regarding EBS (persistent block storage on AWS): a 2TB volume provides 6000 IOPS and it's transparently replicated across multiple servers, so it's not the equivalent of a 2TB drive that will last 5+ years for $80. See https://aws.amazon.com/ebs/details/
Note that an EBS 2TB gp2 volume provides 6000 IOPS but an instance size must have enough bandwidth to drive those IOs. The T2 instance family may not have adequate bandwidth. But that may not matter based on your disk IO requirements. If your server is bottlenecked on disk IO, resize to a larger instance size. That entails shutting down the instance, changing the size, and restarting. This should take all of 5 mins. This flexibility to vertically scale up/down quickly is one of the benefits of a modern cloud platform.
it may well be that what Amazon is selling is not the same as the $80 disk, but anything it provides above what the $80 disk does is demonstrably shit that I do not need.
I understand that a lot of people think that is really neat, which is why I explicitly said in the original post that I care about this feature exactly not at all. Insofar as I have to care about that, it is a bug, not a feature.
I have used Viviotech for years. They've always been shockingly available for support when I wedge things, and I have yet to have a negative experience with them. Their plans look comparable to your specs:
https://viviotech.net/dedicated-servers.html
6c/12t Xeon E5-2620v3 2.4Ghz32 GB ECC RAM
1 TB SAS RAID 1 Mirror
1 TB Backup Storage
Unmetered 100 Mbps Transfer
Citrix XenServer VM Manager
And if you care about such things, they even offer the ability to carbon offset your energy usage with wind power, blah blah eco-friendly.
I have had servers in some capacity or another continuously since 2009 with Rimuhosting. They seem like they'd be right up your alley - when you need support you deal directly with someone who's very technically savvy, there's no customer service drones to deal with (Just look at the staff list!). Occasionally when I've submitted a mundane support ticket, the founder has been the one to take care of it even though the company's far too large for him to need to. You can read a bit about the company here.
They have a lot of dedicated servers, but one that caught my eye based on your specs is $290/mo, has 64gb RAM, 4x 1TB drives in two RAID-1 volumes, and dual E5-2620 processors. The processor's a bit old, but that's still 12 physical cores (plus HT) at 2.0gHz with a 2.5gHz turbo. I'm not sure if 2x 1TB redundant volumes would be enough space, but they also can do whole-server backups. That server's the second to last one on this page. They definitely are more expensive than a lot of other hosting providers, but you really get your money's worth with the support and uptime.
I don't have any association with the company other than being a very satisfied long-time customer.
I've been using AWS in a professional capacity for years. I seriously doubt you need that much horsepower to host what is essentially a static website. There are the instance sizes that give you burstable performance and a substantially reduced cost. I doubt that your (mostly) static website needs as much horsepower that is currently given to it.
I can also attest that the AWS cost models can be somewhat inscrutable to even experienced people.
First, find out how much CPU, memory, disk and bandwidth you are using right now. Once you've figured that out, pricing options is going to be much easier.
Secondly, to get the vaunted "but AWS scales for everything" promise, you have to expend work on good build/release/image hygiene. I know you can do this, but you have to invest in it. If you are willing, I bet you can get something reasonably hosted for not a lot of money in AWS.
> AWS cost models can be somewhat inscrutable to even experienced people
They deliberately obfuscate what it costs because nobody is cheaper than anybody else, and Amazon has no advantage.
If you dig far enough, Amazon will give you more detail on instance types, but it's still "SLA" level rather than "you will run on this precise CPU".
The main advice I'd give you for "buying cloud computing" is to ruthlessly question each part of what it is that you need. And also whether some of that can be replaced with a lower cost service. For instance Amazon charges quite a bit less to stream data off S3 versus streaming off an EC2 instance. It may well be if you outsource all the "large data file" streaming to S3 (eg, by having your webserver in EC2 just serve up temporary redirects to S3) that you both need a much smaller EC2 instance and end up paying quite a bit less for data transferred. S3 storage is also quite a bit cheaper than EBS (block storage for EC2) too. FWIW, 8 cores/32GB RAM seems a lot for "a webserver" that doesn't have lots of compute-heavy/RAM-heavy (database heavy) stuff behind it.
The "do it all in one big box" is often a poor fit for cloud computing billing; and if you really do want to "do it all in one box" for other reasons (eg, admin simplicity) you probably still want to rent a dedicated server somewhere at the 8+ cores/32GB+ RAM -- those numbers are a non-trivial fraction of even large physical servers (eg, 1/4-1/8th of reasonably feasible RAM/CPU). Single-instance cloud seems to work best if you can get down into the smaller cost tiers (multi-instance cloud can be useful for easy redundancy/scale out even if the instances have to be fairly big).
FWIW, I've also had pretty good experiences with Gandi for hosting in Europe, and I know they also have US-based data centres (but maybe only East Coast?). I've not used Digital Ocean myself so don't have an opinion on them.
Ewen
PS: Yes, "cloud storage" is expensive. Some of that is "there's a lot more under the hood than a 2TB drive we bought at Best Buy", and some of that is that it's just expensive. Some cloud storage is a lot more expensive than other cloud storage (eg, EBS versus S3 above).
I've been very happy with Virmach, although I've not tried any of their US servers because geography.
$280/month for a 6-core E5-2620 with 64 GB RAM and 10 TB BW (gigabit), a 120GB SSD + a 2 TB HDD. Variety of datacenters. They also give you a full /28 segment if you ask for it, otherwise 5 IPs. Can also configure it with dual CPUs (+$75), 128 GB RAM (+$50), etc.
+1 linode.com. Been with them for years.
The entire hosting industry is careening from break even towards a commoditized negative profit margin, including Amazon. Nobody has any pricing power. Any particular cloud solution will be transient from year to year as providers try outright screwing people as a way to maybe actually make some money. Anything you plan to run for years without constantly worrying about getting shafted might as well be on your own hardware in your own basement.
What? AWS has been a significant reason why Amazon is finally profitable.
Amazon is profitable in roughly the same way Enron was. We are at the end of another business cycle highly analogous to 15 years ago. If you're genuinely curious about the details of the Amazon fraud I suggest shelling out for David Kranzler's report.
Scanning his Twitter feed, way too many references to gold with a smattering of Bitcoin. Plus appears on a YouTube channel called Shadow of Truth. No thanks.
Solid. Thick. Thanks for your refutations. Keep us posted on your further rebuttals to highly educated veteran equity analyst reports.
Er, Enron claimed on paper to be wildly profitable and were completely lying about it in order to pull what amounted to the world's largest pump-and-dump scheme on anyone insane enough to be long on their shares.
Amazon's general lack of quarter-by-quarter profitability is a 100% on-the-record documented fact, and the reasons for it are not mysterious: they plow all of their margin back into R&D, acquisitions and headcount growth. This is their stated plan, and they will continue doing it for as long as their board and their major institutional shareholders continue to think that it looks like a good idea.
Time will tell if this is a good plan, but comparing them to Enron is weird: they're not even slightly secretive about this stuff.
(I hold no shares of AMZN except whatever happens incidentally by virtue of owning some of Vanguard's index funds.)
Nobody has mentioned this yet but keep in mind that amazon won't answer your email either, unless you pony up for business support.
Dunno if google is any better.
Have you checked out Rackspace? We use em at work and dig their service.
Dedicated, canned configs @
https://www.rackspace.com/en-us/dedicated-servers
Closest I see to your specs gets you 6 cores @ 2.5 GHz w/ 32G RAM & 1.5 TB for ~$450/mo. They'll quote custom configs.
Two links for alternative hosting that might be worth your brief consideration:
* Someone above mentioned prgmr without linking to it or explaining why it's neat. They're small, but cater to power nerds by basically giving you bare metal control at shared hosting prices (their tagline is "We don't assume you are stupid" and their logo is rendered from ASCII). Their most powerful plan is not quite enough for you, but at only $60/mo, you could probably split business/personal onto two of them and come out ahead? Also, it seems likely that you could contact them and get special treatment. I really love that they exist (and have existed relatively unchanged for a long time), and have read many first-hand recommendations from people I trust, but I have never actually given them my money. Their ethos seems very much in line with the way you like things.
* Virpus is less small and specialized, but they're doing a lot of things very right, and more importantly, very inexpensively. They have two options you might consider, both coming in a couple hundred less per month than what you're currently considering: Bare Metal Clown (forgive the oxymoronic name, but it is a neat way to have things set up) and a more general VPS with dedicated resources (except for CPU cores--with which they are generous).
Amazon is optimized for project servers (servers you need for a variable amount of time, less than 2 years). So you're going to overpay for your usecase (indefinite hosting).
Lowest I can get for Amazon is $544/month:
$312 - i3.2xlarge, 1yr reserved, no upfront (twice the memory you need, but has 1.9TB SSD included)
$48 - 2048G S3 for backups
$184 - 2048G data transfer out (don't forget about this with Amazon!)
You can go slightly lower by reserving the server 3yrs all upfront. But not close to the $300/month you pay right now.
Better alternative is finding a smaller hosting company. For example:
2x Intel Hexa-Core Xeon E5645, 32GB mem, 2x 2TB disk, 10TB transfer out, $195/month
https://www.leaseweb.com/dedicated-server/configure/26323?configurationToken=0bdd5f4d44a162713d7b19cc
By the way, DigitalOcean aren't metering traffic despite having bandwidth on their pricing page. Don't know how long it'll last though.
You wrote about per-service VMs:
> it sounds less good when each of those virtual servers is an additional $250/month.
That's an invalid argument: if your current server uses 2TB disk and bandwidth, then each of the services will be using only a part of it, and cloud pricing linearly for these two factors.
For example, on Vultr.com you can get 5$/mo instances with 1vCPU, 25GB SSD, 1GB RAM and 1TB traffic.
Block storage costs $0.10/GB/mo, so $200 for 2TB.
So for your current budget you'd get 20 small VMs totalling 500GB disk/20TB traffic + all the block storage you need, or more VMs if you want.
You can then use http://nixos.org with NixOps for mentally stimulating server configuration.
You should never think you're smart enough to label something an invalid argument.
Not even if you're a Nixer, and more so if you think that 20-something small web server VMs are a way to gain reliability over one system. The SLA calculation on that is so far off...
> You should never think you're smart enough to label something an invalid argument.
Ok, true, I should have said something like "I believe that that is not the full story".
As to 20 VM having greater reliability than a single physical server: I did not say that. They are not comparable directly.
I would say however that if you can spread your services over a greater amount of locations and hardware, the chance that all services are offline at the same time becomes very low. On a dedicated host, you don't suffer from misbehaving neighbor VMs, but if something goes wrong it will probably impact all services, and it will be harder to restore service.
Personally, I would pick 20 ok-level VMs over a single beefy-level VM, as long as I can correctly provide the service of course.
$250/month is the price Digital Ocean and Amazon are charging for CPU+RAM, before adding in block storage. So maybe I could decide "if I'm serving 30% of the monthly web traffic, I can do that with 1/3 of the cores and 1/3 of the RAM" but it's not at all clear to me that that logic holds. "More organs means more human!"
CPU+RAM are the factors that price-scale non-linearly.
If you have a couple of websites that all use the same infrastructure (e.g. apache+php), then it might be worth getting several smaller VMs for that service and putting a proxy (maybe a DNS-based one with short TTL) in front. Depends.
Here's a nice set of graphs showing the pricing nonlinearity https://cloudspectator.com/cloud-pricing-comparison-2016-north-american-report/
Sadly they just use VM classes instead of direct CPU performance.
I recently asked a similar question of my friends. They suggested https://prgmr.com/xen/. The price and ethos are right, not sure sure about your storage needs.
Scaleway looks good too (bare meta!), though I'm not sure if their US data center is open yet.
Fellow unfrozen caveman. While I run my blog on Digital Ocean because I can't be arsed to host & maintain WordPress, I would never suggest running anything serious there. I cannot figure out any Amazon setup that wouldn't cost me a computer/month in cash for 1 computer equivalent.
I run a ton of other services on a colocated Mac Mini at
MacMiniColoMacStadium. This is super cheap if you're buying that hardware anyway, but not suitable for everyone.I have in the past colocated Linux servers at Switch, but you have to have a sales rep call you (ugh) to get a price estimate. You load up a 1U server with everything you want, and it shouldn't be too bad.
MonkeyBrains is Rudy Rucker's company, and their plans look reasonable, but I've never had the chance to use them.
Did… the blog eat my comment? Or did I not press the button?
Shorter repost: Colocate, I don't understand cloud pricing for small servers either. I use MacStadium for a Mac Mini, which is reasonably priced if you're buying that hardware anyway.
I've used Switch.com for Linux, but you have to have a sales rep call you (ugh).
Monkeybrains is Rudy Rucker's company, always wanted to use them and they seem well-priced, but I haven't had cause to yet.
OK, this seems to be Safari caching the form-followup page and not updating the original page, which was still in my tabs. Delete the dupe if you want.
I haven't used their colo service, but Raw Bandwidth is a local ISP with terrific service. 200 Paul is in the ghetto, but it's the most reliable data center in SF proper, with many big names ensconced there. Tell Mike Durkin I sent you:
http://www.rawbandwidth.com/services/colocation.html
If you're willing to continue owning your own hardware I would second the "Call Mike" advice. I've had a 1U in Raw Bandwidth's cabinets @ 200 Paul since 2001 or so. Uptime has been fantastic. The downside is it's a one man ISP, the upside is when you call support it's the owner's cell phone.
I use Scaleway for my personal servers. They have dedicated boxes up to "massive DB" scale but I imagine you'll find something to your liking and it's dirt cheap - I run two VMs for less than $5/month.
Disk is a little pricey.
Being hosted in Europe hasn't really impacted my site performance but I get, like, 50 hits/day on some of my sites, soooo...
Anyway, another option for you.
IO cooperative has colocation available in Sunnyvale which is pretty cheap, and they give you direct access to the PDUs and serial terminal servers connected to your box. Bandwidth isn't so cheap, but if you stick Cloudflare in front of it you don't care, and save a lot of money in the process.
If you really want to rent one instead of buy one, I have Proliant G8s which I can rent to you for, oh, $50/month which exceed your specifications of 8 cores + 32 GB + 2 TB disk. They also have built in KVMs and serial-over-SSH on the out of band management card. Buying one is probably cheaper, though, they're going for $500 on ebay with 6+ TB of disk.
hope i am not too late for this show... JWZ: Have a look at https://www.online.net. After the cloud fuck up I had on my personal vps (random reboots, linux kernel crash of KVM box etc), i switched to a dedicated server from them.
They are based in France, not sure if Europe is a drawback for you.
As indicated by others above: you can get something from amazon, but amazon is build on the expectation that you manage the risk of their service breaking by running at least 2 copies, spread over at least 2 'availability zones', and pricing is anyone's guess, but it's more than colo or a single VM.
Colo options I think will still be your best bet, both for $ and expectation of what you get, if they can be found.
If not, you're looking for a VM hosted the 'traditional' way, from back in the day where we were talking about the amount of 9s in the uptime SLA. They exist, locally (EU) I can even get them in 2 ways: with the OS responsibility managed by them, or without (i.e. only virtual hardware is their responsibility). They do the latter by running an OpenStack setup, which may be a good term include in your googling if you want to go this way.
That will be more expensive probably, but you don't have to worry about getting that 1 disk in 100 that dies in 2 years iso 5, and other hardware issues.
If you want to compare pricing, I was looking at https://www.combell.com/en/openstack-cloud-hosting , but of course hosting in Europe makes no sense for your needs and this doesn't exactly match what you need.
If all of this is frustrating you, I believe that the following link should help you understand everything about how you must build your website these days: https://circleci.com/blog/its-the-future/
(and just for good measure, though I'm sure everyone has seen all of these links, but we can all use a good laugh around now: http://howfuckedismydatabase.com/nosql/)
So much this.
Check out OVH dedicated servers, fraction of DO/AWS cost and a decent quality. ~$120 for the base server + "unlimited" transfer which (personally tested) handles 2TB/month. On top of that a few dollars extra for the backup.
Take a look at Rackspace.
They give you a VPS where you can install almost any distro (or even win).
SSH into it and do whatever you want.
For my particular use case. It's WAY cheaper (and faster) than AWS.
And you have complete control over DNS records.
Cons: don't expect much from support (at least tier1)