if you’ve got lots of money and you don’t care about how you spend it or translating those savings onto your customers
This is a baseless, prejudiced claim, contrasting buying with some idealized scenario where your in-house crew can only possibly build better solutions cheaper. That there are no other outcomes.
But they might also cost far more in manpower costs than any premium. They might give you an unreliable solution that costs you your entire business. Such an effort might distract from the core competencies of the organization. Operating that storage might end up dwarfing the up-front cost (it's easy to hire admins knowledgeable of EMC. Quite a different matter when it's your own home-brew solution).
I understand the draw, but the "all upside" claim undermines the entire piece. There are enormous downsides, not least the reliability of your data. Companies like EMC and Nimble -- despite "great" editorial quotes added by some random person on Wikipedia -- base their entire existence on reliably serving your data, and things like multipathing and replication are the absolute minimum cost of entry in the market. Now add automatic tiering, thin provisioning, disk-deduplication and streaming hardware compression, etc, and the value starts to become evident.
EDIT: The moderation through this is an abomination. HN shouldn't be overly critical, but nor should it pander patronizingly to some tripe because the author happened by.
Hi there, I'm very open to constructive criticism on my project but I feel that your comment is more of a strawman attack than particularly useful in any way. I had a look back through your comments on other posts on HN and they seemed to have a similar tone. If you feel strongly about the topic one thing I could suggest that would help readers of your comment would be to include some details, facts or technical analysis that are relevant and insightful. Also I might suggest reading the recent hacker news post concerning community feedback: http://blog.ycombinator.com/new-hacker-news-guideline
Edit: by the way, I do have compression, thin provisioning, replication and it also auto provisions storage volumes when new VMs are created. As I said, I didn't post my two month old blog here and there is plenty of detailed information for those that are genuinely interested on its way in the next few weeks.
"Hi there, I'm very open to constructive criticism on my project but I feel that your comment is more of a strawman than particularly useful in any way. I had a look back through your comments on other posts on HN and they seemed to have a similar tone."
So you don't attempt to prejudice only in your blog posts, it appears. This is one of the most sadly defensive responses by a blog author I've yet seen on here.
Where, exactly, is the strawman? Please point it out rather than desperately trying to immunize against my completely valid comment in the most frantic of manners.
You posit an idealized notion about building it yourself based upon accomplishing, to this point, apparently very little. It's one thing to cast a theory and pursue it (e.g. "my attempt at building competitive storage on the cheap"), but you're presenting completely unsupported dogma around it, and then bolstering your own decisions by conclusions you actually don't even remotely have. It's the guy who decided to start working at a standing desk and on day one has a laundry list of comments about why anyone who doesn't is wrong and lesser.
This is a fairly typical, of course, and you see it by people who build their own anything ("don't give all your money to big TP -- six quick tips for making your own!"), and we generally only see it in the before stage, as the after stage is more often than not a littered debris field of failures.
I find it rather incredible that you attempt to incite HN guidelines, as if raw gullibility and boosterism was the direction of the critique. You are making broad claims in the linked article that you have absolutely no basis for making, so criticism is well deserved, and a service for anyone who might buy into this notion, making a fool of themselves in their own organization.
The strawman is that he builds a solution for his exact needs and you attack it for not having features (multipathing, replication, automatic tiering, thin provisioning, disk-deduplication, streaming hardware compression) that neither he nor most people need.
All that stuff becomes interesting when and if you need more than two bricks. In a time where you can fit >250T of spindles or ~42T of SSDs into a single server the audience for such shrinks rapidly.
Is it irony that you're presenting an absurd strawman to demonstrate my purported strawman?
I "attacked" (aka disagreed with) any broadly-targeted, generalist claims of the linked article (which is in stark contrast with, for example, the Backblaze posts where they build very purpose-suited storage and never try to over-extend their claims), which was quite clearly that people who buy so-called enterprise storage are, to paraphrase the gist of the article, suckers. I noted replication and multipathing because they are the absolute minimum cost of entry for critical storage, and even the linked article references it as a requirement.
The rest of the features were clearly "value adds", given that most enterprise storage features a lot of its value in software. Yes, everyone benefits from automatic tiering. Everyone benefits from thin provisioning, disk-deduplication, and compression. There are a vanishingly small number of users who won't see significant benefits from all of those features.
"All that stuff becomes interesting when and if you need more than two bricks."
? Multipathing is absolutely critical for a single storage unit. Replication is absolutely critical if you have any care at all about uptime, because a single storage unit, even with multipathing and redundant power supplies and "controllers", as appropriate, isn't enough. The rest of them have nothing at all do do with the number of storage units -- thin provisioning gives you fantastic storage control. Compression is obvious. Disk-deduplication...again, the name of the game is minimizing the amount of data you're actually dealing with, because even if have conceptually unlimited storage in your unit(s), that's data you have to move around and replication and backup and...
Most enterprise arrays these days are really software dressed up behind the hardware. As a customer, you get to pay for the hardware multiple times. This is really obvious in the backup space if you compare a software-only solution like CommVault to Avatar/DataDomain.
If you make the conscious decison to invest in engineering talent and have a lot of need for midrange storage, you can yield a positive return on that investment. If you're growing fast and go EMC, you'll invest significant capital in administering those solutions and negotiating anyway.
But if you don't have the resources to staff it, or the systems for provisioning/service management to replace the consoles you get from a vendor product, you don't belong in the business.
Most people simply don't need this kind of storage anymore.
But this is exactly the sort of storage the linked article is talking about. So your point is that I attacked a "strawman" because I didn't accept the linked article for being something entirely different from what it is?
but the large majority of companies aren't
The majority of mid to large sized companies run entirely on SANs. I'm not sure where you get your information from, but how shoestring upstarts operate has nothing to do with much of the "real world".
This article is about a home-brew SAN, with home-brew SAN qualities and deficiencies.
Ugh, I managed to fatfinger-delete my parent comment, sorry.
(the essence was what you quoted: I claimed most people don't need these advanced SAN-features anymore because they don't need SANs in the traditional sense anymore)
Anyway, I'll just concede that OP aiming for feature-parity with commercial SAN's is indeed nonsensical.
The majority of mid to large sized companies run entirely on SANs
For large companies who bought into EMC/NetApp that may be true.
For small and mid-sized companies that's definitely not my experience.
Criticism is fine. Gratuitous negativity is not. Your comments routinely have a lot of that, and it's time for it to stop. Please stick to the substance and drop the personal bile from now on.
EMC are expensive and nimble are ok for light VM performance.
However they provide a service. As the OP says its a bit broad to claim that they are a pointless cost. Having dealt with both, I can tell you that its very rare that own brand storage is cheaper in the long term. Unless:
o you run at significant scale
o have enough data to warrant a team of skill storage admins
bear in mind that decent storage admins cost the same as a medium sized array, per year, per person. Opex can quickly over take capex.
Actually we've found quite the opposite, because proprietary vendor management tools are generally hard to automate and integrate with existing frameworks such as puppet - the time it takes to administer them is far greater than any of our more standard systems. I just replied to a similar observation from someone else here: http://smcleod.net/building-a-high-performance-ssd-san/#comm...
but it's wrong to generalize it as a solution for everyone
This is the fundamental flaw in virtually all quick-fix lifehack kind of pieces -- they project personal habits and failings generally, and then make broad claims about how to fix their own issues. It is the alcoholic telling you not to keep alcoholic beverages in your house.
I feel absolutely claustrophobic on a laptop with but a single work surface. And it isn't because I need multiple unrelated things fighting for my attention, but instead because I need maximal information for the singular thing I am working on. The flow of using all of the pieces of information is much higher -- for me -- when I don't need to task shift, shroud other pieces of information, and so on.
Docs/requirements/protocol specs on one screen, IDE on the other at the minimum. Realistically I usually like the IDE itself on multiple screens, inspector/project structure/source management on one, text editor on the other. Sometimes I split the text editor across multiple screens.
But that's personal habit. As mentioned I find working on a laptop absolutely crippling, though I have no doubt that there are people for whom it is ideal.
Note that this story isn't really about vomitoxin, or about the permissible amounts of unpleasant stuff, at least not directly.
While that was the lead in, the actual story was about Kraft leveraging the markets, including products they can't use, to get a better price on the stuff they can use.
And FWIW, while the human limit for vomitoxin is 1ppm, it's 5ppm for pet food, and 10ppm for livestock feed.
My reading of it is that the app on the phone is used to configure the button, and the button from then on operates independently (which would be good in that it would allows others to use it when the original person isn't around). It's ambiguous, though, and could go either way.
Given that it would literally only have to connect to the wifi and do the API calls when the button is pressed, the power usage would be minuscule.
They tell you specifically how much RAM you get, and there is no confusion. They also tell you exactly how much storage you get. In this case they very specifically tell you disk read throughput.
So there are a lot of metrics that are not ambiguous. Imagine if they told you that you get a "moderate" amount of RAM, and maybe it could be 4.5GB, or maybe it's 9GB.
There would be benefits if Amazon gave SLA style promises for these sorts of instances. When they see that they are falling short, they redistribute loads or add hardware to meet the promise. Otherwise it's generally somewhat meaningless because if you can't plan on it, you can't build a plan around it.
The overwhelming majority of those "winners" fail. These funds aren't anointing the king, but rather are just trying to make a bet, and it's a bet they usually lose. As you said, their funding gives the entrant a leg up (usually if there's a significant capital cost), but in this case such funding provides astonishingly little advantage of scale.
This isn't an example of a bubble. It's an example of a really comically silly investment.
"winners" failing is not much of an indication - obviously there is a staggering number of investment entities and often they are competing with other "winners" (look at the payment market, with Stripe, Balanced, Braintree etc, they all raised serious money) and someone usually comes out on top.
Whether or not Magic is a smart investment - I can't really tell, but I suspect that if I were to choose between Magic and Twitter back when they were just ideas, I would bet on the former.
Yeah I have to agree it's actually really hard to fail when backed by one of the bigger VC's, if you're on the road to failure they can just bully one of their former portfolio unicorns to acquire you with post IPO Opm dollars. See Twitter's recent bail out of Foursquare care of Fred Wilson. There are no losers in this game.
Even without doing the OPM arrangement, it's very sobering to see companies that fail on tech, product and people level hoovering up gobs of money due to their investor momentum. At some point, an IPO is the only way out (well, that's the best kind of OPM, isn't it).
Everything is relative. When someone is defensive and self-conscious, it's incredibly hard to give feedback of any sort without them feeling "like an idiot".
Yeah I have a hard time telling what's definitely going on from the OP's description.
I personally find the most difficult people to work with are the people who have trouble filling the gaps in their knowledge themselves AND are evasive when they don't know something.
They are afraid that their criticisms will be magnified and distorted by the press and used as the basis of hit-pieces
Hardly.
Many Apple fans still operate as if it is some tiny, vulnerable niche company, and they're a member of the few. Perhaps this is some sort of traumatic stress of those early days, but it manifests in this feeling of great discomfort if your words are used "against" the hive. Arment, significant in this piece, incredibly claimed that it was a nightmare having his entirely valid, indisputable issues with Apple software lately, spoken of by "the others".
This is bizarre behavior. It is absolutely incredible. Apple is one of the largest corporations on the planet. It is an enormous money machine. And people are desperately fearful that their opinions about Apple might get known? Come on.
The whole "magnified and distorted" and "hit piece" noises is just garbage. It has nothing to do with proportionality or reality, it's just this sense that a community is under attack and they need to be defensive, and it's just bizarre.
Whereas every company is falling over themselves to hire the 10x programmer.
Is this true? I've been involved in a number of hires over the years where we quite specifically did not want a "10x programmer". We knew the position didn't pay enough, and had enough tiresome, non-novel work (e.g. building data processes for client data. Not big enough or interesting enough to be big data or technically challenging) that we simply wanted, in effect, a marginally competent chair warmer.
From seeing hiring and employment practices, this seems to be absolutely common across the industry.
Similarly there is another comment that opines that every programmer thinks they're a 10x programmer. Now maybe it's because I have a work history in places like financial firms and banks and insurance and telecom, rather than pure software or Google-esque, but this is absolutely untrue. I found that the vast majority of developers were simply careerists.
I'm surprised the John Gruber "little birdie" thing is treated as a legitimate source, especially in this case where it's outright absurd. John has retrenched to now Apple "basically" inventing it because wires and stuff.
This is a baseless, prejudiced claim, contrasting buying with some idealized scenario where your in-house crew can only possibly build better solutions cheaper. That there are no other outcomes.
But they might also cost far more in manpower costs than any premium. They might give you an unreliable solution that costs you your entire business. Such an effort might distract from the core competencies of the organization. Operating that storage might end up dwarfing the up-front cost (it's easy to hire admins knowledgeable of EMC. Quite a different matter when it's your own home-brew solution).
I understand the draw, but the "all upside" claim undermines the entire piece. There are enormous downsides, not least the reliability of your data. Companies like EMC and Nimble -- despite "great" editorial quotes added by some random person on Wikipedia -- base their entire existence on reliably serving your data, and things like multipathing and replication are the absolute minimum cost of entry in the market. Now add automatic tiering, thin provisioning, disk-deduplication and streaming hardware compression, etc, and the value starts to become evident.
EDIT: The moderation through this is an abomination. HN shouldn't be overly critical, but nor should it pander patronizingly to some tripe because the author happened by.