Also, since this is HN, let's talk about how we build software, as in, put yourself in the position of the Dropbox developer and instead of just the end user. Most of us, I imagine, build software used by non-developers. I hope your attitude towards these things isn't "fuck it, if they were a competent developer like me, they'd have known better", and instead, "damn, I really need to go bulletproof my software so it doesn't delete people's data like that." In that context, there's plenty to criticize and learn from in Dropbox's mistakes here.

Edit: grammar

Dropbox for teams is specifically marketed towards businesses.

Did you read the article this thread is about?

You could, of course, take the approach that Dropbox should be there to protect everyone from themselves.

Fair enough.

I have, in other comments, given the example of my own usage, which guarantees that no data can ever be lost by Dropbox. It's simple and it works. I've been constructive here. If you implement my approach you should be safe. Still, don't take my word for it, implement and verify. That's engineering.

I have also, in another comment, posted about a feature request that might help insure that this does not happen to casual users:

Dropbox, if possible, should modify their client software such that files can only be copied in and out of Dropbox folders and never moved. For the adventurous, they could make this an option that is set by default but can be disabled. A huge warning about the potential for data loss would accompany the act of disabling this feature.

In that regard, I've been as constructive as one could be in this thread.

In the end, you can't protect everyone from their own mistakes, incompetence or carelessness. And, no, I don't think everyone is stupid. We all make mistakes. I had a very painful data loss event in college, so I learned my lesson very early on. I have never lost valuable data since.

I am not going to blame a service for something that is the responsibility of the user. Engineers, in particular, have no excuse for data loss. A 2TB external hard drive is about $100. Please.

You could also argue that business users in this day and age cannot be computer illiterate. Being "literate" does not mean being able to open a browser and push a mouse around. If a business owner is not capable beyond that they ought to be smart enough to hire a qualified IT service to help them manage and secure their systems. If they do have in-house IT then data backup and security ought to be one of their top functions.

Private users are a little bit of a different story. There's a huge chunk of them that, today, could loose every digital asset they own and have no way to ever recover it. This is still a business opportunity for a startup somewhere. Services like Carbonite and others abound:

https://www.google.com/search?q=online+backup

At one point you have to point your finger at the data's owner and say "You, and only you, are responsible for your data".

I can guarantee you that if you read the TOS of any online data backup service there's a clause there about the potential for accidental loss. There is no way in hell that an attorney would advise anyone not to have that clause there. And, no matter how good of an engineer you might be, there is no way you would risk it all to offer some kind of an absolute no-loss guarantee.

The test is simple: Would you, personally, be willing to be sued into absolute poverty for issuing that guarantee? Probably not.

So, we can sit here in righteous indignation for me daring to suggest that users are ultimately responsible for their data and lie to each other or accept the reality that the reality, developer or not.

Down-vote away, but I am right.

If we go back to the very original article that this thread is about, the programmer who joined the startup team seems to have not had any explicitly-created local backup of the data in question. He got lucky. He said "The client left all the files on my machines, so I didn’t lose any personal data - it wasn’t a catastrophic failure.". That's luck, that is not planning. So, he did OK, he talks about having to re-upload some 2GB of data to a new account. He was off and running after that. I can bet you that, unless he isn't very smart, he now has private local backup of his data in case something like this happens again. And that's the right way to handle it.

Don't trust anyone with your data unless you can independently verify the ruggedness and reliability of their solution. Period. If you did not do that, engineer or "civilian", total data loss is on you, not the service.

Apparently you've never heard of incremental backup? Dropbox may not be a backup but there are many services that backup via the internet. Assuming you don't change backup systems every month, 100 days for the initial sync is perfectly acceptable.

Oh, please.

> Dropbox may not be a backup but there are many services that backup via the internet. Assuming you don't change backup systems every month, 100 days for the initial sync is perfectly acceptable.

Oh, please. Again.

If you don't have local backup, for a third of a year you have no backup whatsoever. Even after that, depending on the nature of your work, even incremental backups could have you unprepared for failure for days.

Not saying at all that remote backup is a bad idea. Not at all.

Remote backup BY ITSELF, without local high-speed backup that you control is a very bad idea.

The best pattern is to have single or redundant backup under your control (yup, use that "incremental" thing I didn't know about) and remote backup. Don't expect anyone backup destination to be 100% reliable, not even local backup. If your data is important you need multiple redundant backup, local and remote. Then you can sleep at night.

My local backup strategy is a collection of external --and these days inexpensive-- hard drives as well as a large rack-mount NAS RAID array. Each of about a dozen systems has it's own local external backup drive right on the desk next to the computer. Some have dual local backup drives. We are talking in the order of $100 for a couple of terabytes today. Then, a number of systems also backup to NAS. Every so often we rotate drives for longer term storage at a fireproof external location. It'd take a lot more than Dropbox or any service having a glitch for me to loose any data.

I really don't understand folks who don't, at the very least, have one external USB backup drive on their system. On OSX you have Time Machine which is ridiculously easy to use. On Windows you can spend a few bucks and get Norton Ghost and you are good to go. All-up, probably not more than $200 per system and maybe half an hour to set it up.

Do that plus my recommendation to host your Dropbox location on a dedicated partition in order to force a copy operation during drag-and-drop (both Windows and OSX) and you will not care less about anything that happens at Dropbox or any other service.

It's about engineering, not hoping for, a system to protect your data.

As far as remote backup is concerned. I'd be interested in a system that might allow me to send them encrypted disk images on physical media for backup while providing some online access to the same.

Even with incremental backup you have to do a full backup every so often. In the case of our Windows systems running Norton Ghost, they are setup to do full backups the first day of the month and incremental backups every day after that. It's dead-easy, reliable and works great. Saved my hide a number of times.

A full backup of about 600GB happens in --I think-- about three or four hours. That's the problem with remote backup, the same full image would require a third of a year on a typical DSL connection available in the US today. Actually, it could take twice as long, two thirds of a year, because you would have to interrupt your backup in order to get your bandwidth back for use during business hours. So, if it takes you nearly a whole year to backup this much data the whole thing is just-about useless as implemented. Your incremental backups are likely to take days and you can't even consider the idea of doing full images every thirty or sixty days. That's what's broken about the concept of remote backups without even looking at the issues with potential software bugs at the various providers that could lead to data loss.

A more usable system would be one that, as I said, would receive my full images in physical media to absorb into their storage arrays for both backup and remote access purposes. If you needed to recover a few files here and there you could easily do so over a decent DSL connection. Full recovery would require physical media being shipped to you at a greater cost. Every x number of days you'd send a new full image set and go incremental after that.

The game changer here will be if we ever get to 100Gb network connectivity to the home and office. That would change the landscape in amazing ways. You could talk to remote storage probably as fast as you talk to local storage. At that point in time, having multiple redundant and geographically separate remote backup locations might very well be the most sensible approach to an organization's backup strategy. Such a system could even talk to a locally installed "backup server" in order to make sure that if connectivity is compromised in some way you still have access to your organization's data during the blackout.

The topic of backup is conceptually very but becomes really complex when you consider the multiple potential points of failure and how to deal with them.

This is why I don't consider any issues at Dropbox to be serious. I obviously don't think of them as backup. And they can't convince me to think that way no matter what they do or say. This isn't to say that I think the service is bad. Not at all. It's because I've been around and I've seen too many failures (some of my own) that I take a very careful and guarded approach to my data. And that's healthy. I use Dropbox for team communications. I almost think of it as a really neat way to "FTP" stuff around. So, instead of setting-up my own FTP server and having to manage my users and storage I can use Dropbox. No data is ever moved to Dropbox. All data is copied to Dropbox. That means that the data remains locally stored and, more importantly, locally backed-up every night. So, through engineering, failures at Dropbox or anywhere in between my DSL connection and their data centers are of no consequence whatsoever.

The thing is, you don't need independent full backups within a single online backup provider, and they would probably deduplicate your backups anyway. Incremental backups work just fine, and will always get your data backed up that night unless you are the kind of person that generates many gigabytes of content in a single day.

I use Dropbox for team communications. I almost think of it as a really neat way to "FTP" stuff around.

Yeah, I can understand that. I was considering replying to one of your other posts with the comparison, but I didn't want to start too many thread at once. The problem is that when you use dropbox as a better FTP, you lose out on a lot of the benefits of syncing. No longer can you go to a different computer and pick up right where you were, if you didn't happen to copy in the files again since the last change. If you're sharing files with a coworker you no longer have any idea where the most recent version is. You don't have a full list of file versions. And your FTP-method of dropbox doesn't really have anything to do with backups. You could move files into the dropbox folder and then set it to be backed up every five minutes if you wanted to.

I just couldn't bring myself to using it that way. Our files are our work product. There is no way I could consider having the only copies of the files connected to a service that could cause total loss of data. It is, in my world at least, an absolute non-starter.

With regards to the idea of sharing files with a coworker and not knowing where the latest files are, well, that's what Git is for, isn't it?

As a matter of principle I have a bad reaction to the idea of calling a data loss event "Another Dropbox horror story". And this has nothing to do with Dropbox or any other service. Dropbox is not responsible for your data. You are. If the data is important enough that total loss would be catastrophic, what are you doing placing all the eggs in one basket? It isn't a Dropbox horror story, it's a story about someone who didn't care enough about their data to safeguard it and then the blame is shifted towards whoever last held the data. That just ain't cool. I own my fuck-ups. It isn't fun, but I stopped blaming others for my crap a long time ago. In the end it works out better that way.

Does Dropbox have issues to fix? Sure. What piece of software doesn't. Knowing that software is imperfect is more of a reason to not, again, place your valuable eggs in one basket.

I have a feeling that there are a lot of young and inexperienced people surfing HN who have never lost anything of significance. They come across someone like me who calls bullshit when he sees it and is willing to stand for what is a solid position and they don't understand it. The only tool they have to express themselves is to mindlessly down-vote cargo-cult style. That's fine, with that approach they'll learn soon enough.

I don't know of anyone --not ONE person-- with say, twenty or thirty years of successful work in computing who would trust the only copy of their projects to any one service --no matter what they claim or how good they might be. Hell, most of these people, just like me, would not trust the only copy of their projects to any single device, local or not. Shit happens. And some of us have seen it happen and have had to clean-up the mess on more than one occasion. Eventually you learn.

I do love Dropbox, but they are not going to come and redo two years worth of work if something goes wrong. And that can happen. And, if that were to happen, I would not blame them. I'd blame myself for being a moron and not having my own backups. Maybe I'll figure out a way to use their active sync technology while still satisfying the requirement for solid redundant backups. Until then, I have enough problems with other stuff to have to worry about having only one copy of our projects stored anywhere, local or not.

I own more than one business and we also do work for other businesses, so, generally speaking, the "Data" drive holds business data partitioned-off into appropriately-named directories.

I also have what I call "support" directories there. For example, I have a "SolidWorks Support" directory with models, macros, tools and other items that are there to support work with that software. The same is true of Photoshop, Altium Designer, Web Design and other segmentable tasks.

Then there's more mundane stuff, like "Digital Photography", "Personal", "Scans", "Temp", "Outlook Data", etc.

Projects started to kind of pollute the data drive in some form. Also, the directory hierarchy started to become a little crazy. For example:

    D:\<company name>\Projects\<project name>\Design\Mechanical\<project work folder>\Suppliers
    D:\<company name>\Clients\<client name\...

At one point I thought that I should segregate active development from finished or shipping product. In other words, make a distinction between "project" and "product". That's why I introduced the "Develpment" drive.

There's another reason. When doing FEA thermal and/or flow simulation for various projects the Data drive started to become clogged-up with simulation data. As you navigate through many iterations of design ideas and simulations you can easily fill-up gigabytes. I didn't think that this belonged anywhere near the "Data" drive.

Finally, some tools (Xilinx ISE back then) did not take kindly to paths with spaces. So I always had to have a separate folder for Xilinx projects which has always bothered me.

This "Development" drive now allows me to create a simple folder with a short path:

    Z:\ProjectA

I also have an "xampp-sites" folder on there for web projects. The projects can live in their own directories in the "Development" drive:

    Z:\TheNextFacebook
    Z:\TheNextGoogle

Once a "project" graduates to being a "product" it is zipped-up and copied or moved into a "Products" folder under the appropriate company directory in the "Data" drive. As an example, if we are talking about an iOS project it turns into a product once it is submitted to the app store and approved.

The jury is still out on whether or not this is a good idea or simply a reflection of being insanely anal about organizing data. I don't know. I am always open to new and interesting ideas on this front. I try to think of the "Development" drive as a dirty or work drive of sorts where I can mess with a lot of things without polluting the "Data" drive. I have this implemented in one machine and have not been compelled to do so in other machines yet. Still thinking about it.