I've said it in a previous DotCloud post and I'll say it again: If they can pull it off then they'll be golden.
But I'm extremely skeptical they can.
Many of the components they list require intimate domain knowledge. E.g. there are entire companies dedicated to designing and babysitting Hadoop clusters, yet DotCloud just lists it casually alongside other bleeding edge mammoths such as Riak or Cassandra.
DotCloud will need to not only stay on top of things for so many components and provide smooth upgrades for an inherently growing diversity of deployments. They'll also have to support old versions effectively forever and deal with all those little customizations that people need.
Of course all that is doable, given enough man-power. However, after a certain point it's not very automatable anymore.
So, that said, I remain curious if this will really come out as the holy grail that they seem to be shooting for. Or if it will gradually degrade into "just another managed hosting provider" (which isn't a bad thing either, of course).
We get this all the time. Yes, Riak and Cassandra require intimate domain knowledge. The question is: what's the most efficient way for you to consume that knowledge? Hire a full-time expert for each? Or use the battle-tested Riak setup that our Riak expert maintains for thousands of developers?
As you point out, beyond a certain point your needs will be specific enough to justify serious customization by the Basho and Clouderas of the world. We don't replace these guys. In fact we plan on facilitating integration with them, so that you don't have to chose between a well-oiled dotcloud stack and that 1% of customization.
As for how scalable and automatable our business is - don't worry about us. Having been in your shoes before, I can tell you with great certainty that dotcloud is much more than a bunch of custom AMIs, chef scripts and clever role-based provisioning scripts.
Our goal is to make you, an expert at these power tools, happy to use us instead, because it's so much freakin simpler. And don't worry, we will add all the customization hooks you need. We're taking one step at a time.
That was exactly my concern when I first heard about it. I got a chance to sit down and chat with one of the co-founders a few weeks ago and came away very impressed - he demonstrated deep level knowledge of a bunch of software I care about (I particularly quizzed him on Solr and Redis) and managed to convince me that his team is very sharp when it comes to keeping on top of the infrastructure elements they support. I think they might be able to pull it off.
Yes, I didn't mean to criticize the team. I'm just in the same business (making my living with what they are trying to automate) and know the hurdles they are going to meet.
For example, my provisioning system looks nearly identical to theirs. I also have the central config file to map out roles/services/dependencies, network relations, and the CLI tools to 'manifest' a layout in the form of server instances, EBS volumes etc. And finally puppet to beat everything into shape. I can compose and deploy a production-ready runtime environment with many permutations of commonly used components in a single day.
The thing is, every seasoned admin builds that stuff over the course of his career. It cuts down manual labor significantly.
But you eventually reach a point of diminishing returns. Each individual component has corner cases, special optimization potential, and more than one valid operation mode (e.g. cluster vs master/slave).
Automating and maintaining(!) all permutations that customers will request is nothing short of a herculean task. That's why every managed hosting provider limits itself to a tiny subset of components ("We are experts in X"), charges significant fees for their services, and refers to specialized consultants for requirements such as "let's have a hadoop cluster".
Solomon and crew were incredibly helpful in providing hosting for us when we were trying to get into YC this last round. Solomon even took a couple hours with us to sit in a park near his apartment and give us some guidance on the day of the interview. These guys deserve all the success they can.
Congrats to the DotCloud guys—if they can crack the language-agnostic cloud hosting nut, they'll be worth every cent. It's a big problem to tackle though :)
I've been using it religiously for a little while now, and I've deployed a couple of Django instances, a Flask project, Wordpress, a Simplemachines forum, a handful of MySQL databases, 1 postgreSQL instance and memcache.
The only tweaking I had to do for any of it was provide a WSGI script for the Python, which Dotcloud was able to provide in a tutorial.
It might be a big problem to tackle, but they are most of the way there.
Congrats guys. Dotcloud is awesome, the team is amazing! Even though we weren't hosted on Dotcloud, Solomon always was willing to help us out. That's the kind of service that comes from a company that you know will do well =)
Speaking as a current Dotcloud customer, the setup they've created to handle creating and managing the various components of your "stack" is very cool - within a week of starting work on our first project hosted on Dotcloud I was already planning and thinking about how to move ALL of our projects over, as the flexibility and ease of use is awesome. Right now they are still in beta, but so far I'm super impressed.
I really like this DotCloud business model. Although i see a competition directly from Amazon Cloud Formation. What will happen to DotCloud if all the independent vendors make their software Cloud Formation ready?
There is a difference here, DotCloud making the platform ready for to you vs the vendor makes the platform cloud formation ready.(Thats AWS they make others to work for them)
Would love to see how this little, nimble start up being chased by a big, fat, bully AWS, fights back.
So what is THE advantage of using dotcloud over my own AWS account where I can configure my servers using libraries like boto or Chef. Using AWS I can go for Elastic BeanStalk or do the complete configuration myself. Not able to see what is the benefit it provides.
Elastic BeanStalk only does Java so far, or so I guess. Sure, they will expand - Amazon has an awesome R&D so I don't expect them to stay sitting there. And if you're daring, you can probably already run JRuby and Jython on BeanStalk, but YMMV. DotCloud already has Python, Ruby, PHP, Java, MySQL, PostgreSQL, Redis... The CLI is also quite easier to use than Boto. It doesn't need you to learn everything about instances, EBS, ELB, etc. for starters :-) Finally, I like Chef/Puppet, but mastering them takes some time. And most folks prefer pushing your code with git/hg (Ã la Heroku/Djangy/etc) rather than authoring AMIs, I was told :-)
You don't have to do all of that sysadmin work. If you're a super-skilled sysadmin who can set that stuff up (with properly configured backups and monitoring and replication and all that jazz) you might not need DotCloud.
(Unless of course your time to do that is worth more than what their service would end up costing you.)
My concern would be, what do you do if you've built on top of these guys and then they get bought by a company like Salesforce that will shut down the service? What are Heroku customers doing post-acquisition, for instance?
my initial reaction, as a developer, is that most of what I do is in one language, so I want to use a platform that is a specialist and not a generalist.
That said, I'm sure I'll check it out and it does sound intriguing.
It may be that for small projects, this is mostly a non-issue.
Historically, heterogeneity has been a big problem for a long time. Vendors want lock in; customers don't. Much of Oracle's early success was about being hardware agnostic. Similar for Java. All that XML nonsense, that everybody hates but that has incredibly widespread adoption, was also about inter-op: it's crap, but it's ubiquitous crap.
Now, I don't know is this is an issue in the cloud. But vendors will want lock in; customers won't.
DotCloud guys, if you are reading this, I am not sure if this is a typo on your main page. It looks like it is, but not sure.
>“Wow. DotCloud have been completely amazing. Their support is second to none. They've taken this chore that we hated doing—systems administration—and turned into a magical, fully automated system.”
Nothing wrong with the usage.
Of course (the folks at) DotCloud have been .... is how it is supposed to be.
The support being talked of and the attitude of doing the chore of sysadmin is for the people at DotCloud and not the company.
DotCloud is interesting but I think I'd prefer a set of comparable Fabric scripts and an AWS account. But that wouldn't be a very good business I suppose.
Well, yes, but I assume that you know what you're doing and when you need to make updates and changes to your server you'll be able to handle it yourself.
I, however, don't know very much about being a sys admin and I certainly wouldn't want to be put in charge of maintaining any production server.
That's what DotCloud is for. I have a beta account with them and its been working out great so far.
This works well... up to a point when you realize that you're paying someone close to a full-time salary to manage your ever growing Fabric scripts and an AWS account.
That fulltime salary is a rounding error for most companies, especially when you consider that you need that person anyways.
Seriously, I'm making a large part of my living off companies that once thought (or were told) that they could do without a sysadmin.
If the internet-part of your business consists of more than a static website (or a wordpress) then, by all means, pay that fulltime salary. Even if you decide to go fully managed hosting; you absolutely want that to be an educated decision rather than an educated guess.
And it doesn't end there. A never-ending chain of small decisions is waiting on every step of the road. Pull that knowledge inhouse (or to a trusted longterm advisor) as early as you can. Otherwise you will inevitably foot a much higher bill later in the game.
I've told a few people that we run on Heroku and had them respond "Really? Isn't that expensive?" When I explain that we have a whole bag of add-ons, several dynos, workers, etc, and our 6 months of expenses are still roughly a week's worth of man hours, they realize that it's actually quite inexpensive.
But I'm extremely skeptical they can.
Many of the components they list require intimate domain knowledge. E.g. there are entire companies dedicated to designing and babysitting Hadoop clusters, yet DotCloud just lists it casually alongside other bleeding edge mammoths such as Riak or Cassandra.
DotCloud will need to not only stay on top of things for so many components and provide smooth upgrades for an inherently growing diversity of deployments. They'll also have to support old versions effectively forever and deal with all those little customizations that people need.
Of course all that is doable, given enough man-power. However, after a certain point it's not very automatable anymore.
So, that said, I remain curious if this will really come out as the holy grail that they seem to be shooting for. Or if it will gradually degrade into "just another managed hosting provider" (which isn't a bad thing either, of course).