Hacker News new | past | comments | ask | show | jobs | submit login
Bing contract prohibits DuckDuckGo from completely blocking Microsoft tracking (twitter.com/shivan_kaul)
462 points by etamponi on May 24, 2022 | hide | past | favorite | 257 comments



This title is very misleading (and really should be changed).

This is not about search. To be clear, when you load our search results, you are completely anonymous, including ads. For ads, we actually worked with Microsoft to make ad clicks privacy protected as well. From our public ads page, "Microsoft Advertising does not associate your ad-click behavior with a user profile." This page is linked to next to every Microsoft ad that is served on our search engine (duckduckgo.com). https://help.duckduckgo.com/company/ads-by-microsoft-on-duck....

In all our browsing apps (iOS/Android/Mac) we also block third-party cookies, including those from Microsoft-owned properties like LinkedIn and Bing. That is, the privacy thing most people talk about on the web (blocking 3rd party cookies) applies here to MSFT. We also have a lot of other web protections that also apply to MSFT-owned properties as well, e.g., GPC, first-party cookie expiration, fingerprinting protection, referrer header trimming, cookie consent handling, fire button data clearing, etc.

This is just about non-DuckDuckGo and non-Microsoft sites in our browsers, where our search syndication agreement currently prevents us from stopping Microsoft-owned scripts from loading, though we can still apply our browser's protections post-load (like 3rd party cookie blocking and others mentioned above, and do). We've also been tirelessly working behind the scenes to change this limited restriction. I also understand this is confusing because it is a search syndication contract that is preventing us from doing a non-search thing. That's because our product is a bundle of multiple privacy protections, and this is a distribution requirement imposed on us as part of the search syndication agreement. Our syndication agreement also has broad confidentially provisions and the requirement documents themselves are explicitly marked confidential.

Taking a step back, I know our product is not perfect and will never be. We face many constraints: platform constraints, contractual constraints (like in this case), breakage constraints, and the evolving tracking arms race. Holistically though I believe it is the best thing out there for mainstream users who want simple privacy protection without breaking things, and that is our product vision.

Overall our app is multi-pronged privacy protection in one package (private search, web protection, HTTPS upgrading, email protection, app tracking protection for Android, and more to come), being careful (and putting in a lot of effort) to not break things while still offering protections -- an "easy button" for privacy. And we constantly work to improve its capabilities and will continue to do so, including in this case. For example, we've recently been adding bespoke third-party protections for Google and Facebook, like Google AMP/Topics/FLEDGE protection and Facebook embedded content protection.


> This is not about search.

Yes, it is. Your competitors in the privacy-centric browser space don’t have this restriction because they’re not search engines acquiring the majority of their data from an entity with a conflicting interest.

I’m inclined to blame Microsoft here; this is a nasty move on their part. However, your stance is problematic. This is a problem, and it’s a serious one. It undermines trust in a product that claims to be the bastion of privacy. And statements like this…

> Overall our app is multi-pronged privacy protection in one package (private search, web protection, HTTPS upgrading, email protection, app tracking protection for Android, and more to come), being careful (and putting in a lot of effort) to not break things while still offering protections -- an "easy button" for privacy.

…don’t help the matter. To me, that just sounds like marketing mumbo jumbo. Ultimately, if a privacy-centric browser is contractually obligated to load tracking scripts and is required to avoid disclosing that fact, I want absolutely nothing to do with either party.


We will work diligently today to find a way to say something in our app store descriptions in terms of a better disclosure -- will likely have something up by the end of the day.

In terms of our app and multi-pronged protection, it isn't mumbo jumbo. Our app is way more than just a browser (and increasingly so). For example, the app tracking protection mentioned for Android blocks trackers in all your other apps. The email tracking protection blocks trackers in your email (that you read in your regular email client/app).

I understand the concern here that we are working to address in a variety of ways, but to be clear no app will provide 100% protection for a variety of reasons, and the scripts in question here do currently have significant protection on them in our browser. From the comment "That is, the privacy thing most people talk about on the web (blocking 3rd party cookies) applies here to MSFT. We also have a lot of other web protections that also apply to MSFT-owned properties as well, e.g., GPC, first-party cookie expiration, fingerprinting protection, referrer header trimming, cookie consent handling, fire button data clearing, etc."


The thread by the security engineer shows that the scripts are communicating back to the servers. That means your multi-pronged protection has failed, unless you've suddenly discovered a way for browsers to block IP addresses from being sent by scripts (and since they can be extracted from the request itself that doesn't seem likely).

That's why the ad blockers that stop the scripts from loading to begin with will always due a much better job than the extra "mumbo jumbo" you're relying on. That stuff should be a fallback for when scripts slip through the filters, not the primary means of protection.


"multi-pronged privacy", "easy button", "capabilities", and repeated use of the word "protection" are all signals that what is being said is an attempt to sell me something and that the salesman should be doubted.

What's actually happening is you're forced to allow Microsoft scripts which do indeed do telemetry on users despite some restrictions you put on them, and they're still effective because fingerprinting works. That fact is embarrassing for a product you're trying to sell as promoting privacy so there's this mildly deceptive attempt to hide what's going on with lots of words and claims of protection instead of straightforward disclosure.


Still coming to my own conclusion here, but I wouldn't dismiss "easy button" as marketing. We keep hoping for easy buttons and reasonable default settings in things like openssl or pgp. I do like organizations that understand an easy button is the safest default. Is that what we have here?


I'm commenting only on the rhetoric, calling it an "easy button" stinks of marketing BS. People desiring simple straightforward tools is a separate subject.


Of course it’s marketing. My mom doesn’t want to set up uBlock and a script blocker and a Pihole. She’d love to click a button and be safer. What’s the issue here?


That I am on HN and someone trying to convince me their company isn't being shady is using evasive marketing speak instead of candor to an audience that clearly knows better than to believe the weasel words.


We have a new marketing word: "multi-pronged protection"


"Defense in depth" strikes me as a legitimate security technique.

https://www.cisa.gov/uscert/bsi/articles/knowledge/principle...


>and is required to avoid disclosing that fact,

Isn't this entire story about them disclosing this fact?


> Isn't this entire story about them disclosing this fact?

It seems to be, but they're claiming the details are confidential. It's rather confusing. I wonder whether Microsoft's intention was to prevent them from disclosing it altogether, or whether they just wanted to avoid the general details of the contract getting out (rather than this particular tidbit of info). I'm inclined to suspect it was the latter--just a general NDA. In any case, I don't like it.


No, it is not just a general NDA.


One wonders what other juicy nuggets are in this non general NDA.


> To me, that just sounds like marketing mumbo jumbo.

What’s more helpful is to hear in which exact situations their blocking doesn’t work.


DDG is a search engine to most people, nothing more.

Just because other avenues exist doesn’t mean people walk them


Still, it's a search engine that touts its privacy. If people want speed and convenience they're far better off using Google.


Do you have any sources you can cite that Microsoft has breached contracts with companies in the past in an effort to get at your ID for advertisers? Otherwise, I would consider this a nothing burger.


I found this passage [0] in the DDG help:

> Ad clicks are managed by Microsoft’s ad network.

> Microsoft and DuckDuckGo have partnered [..] Microsoft Advertising will use your full IP address and user-agent string so that it can properly process the ad click and charge the advertiser

It seems DDG is not that privacy focused when it comes to ads.

[0] https://help.duckduckgo.com/duckduckgo-help-pages/company/ad...


Actually, that's not the case. First, that page is a linked to directly from every Microsoft ad on duckduckgo.com -- it's a public disclosure for transparency. Second, we specifically worked with Microsoft to make our ads privacy protected. When you load them, they are completely anonymous. When you click on them, we got Microsoft to contractually agree and publicly commit (on this page) that "Microsoft Advertising does not associate your ad-click behavior with a user profile. It also does not store or share that information other than for accounting purposes."


I think a legal department could be convinced that "accounting purposes" could adequately cover most all of the business of tracking, optimizing, and attributing ad clicks.

"Microsoft Advertising does not associate your ad-click behavior with a user profile."

Does somebody else besides Microsoft Advertising do it? I'd guess so.

Is there another kind of association besides a "user profile" which has substantially similar concerns for an end user? I'd guess so.

This is all coming off as an attempt to cover up what's really going on with deception. That might not be the case, but if it were, this is exactly how I expect a "privacy focused" organization to communicate when they had been corrupted by a compromise to a third party.


So instead of an actual set of real protections, like offered by things such as UBlock, you want us to rely on Microsoft being ethical.

It also ignores that governments like the NSA have tapped these very networks for data (this is what prompted Google's internal SSL drive). Even if we trust the legal entity, the fact is that the information itself is a target and so are those entities. It is always safer not to send the data, but in this case you're explicitly sacrificing that safety to benefit your ad partners.


So now I also have to trust Microsoft before clicking on a DDG ad. Based on a pinky promise not to use my IP address + User-Agent + whatever fingerprint they make?


I mean, of course you have to trust a party X with your browsing fingerprint if party X is involved in serving the URL you go to when clicking on the link.

What did you want—for the URL to go straight to the destination page with no redirect through an ad-network analytics provider, making your impression invisible to the network and thus unable to be costed against the advertiser? Why would any ad network even bother to participate in such a scheme? How would they make money? Prepayment for an arbitrary guess at predicted click-through count?


Why’d you even click on an ad in the first place if you are worried about that?


They wouldn't, and DDG has a convenient way to disable ads which I am sure many users take advantage of.

Still, millions of users do click those ads, because if nobody did, DDG would not exist. A less tech savvy user, who is probably DDGs main target, came on the promise of privacy and does click those ads and is also being tracked around the web by Microsoft if they use DDG browser (from what I understand).

This is less than ideal from the standpoint of "privacy simplified" promise, but really no other way around it when selling ads is your business model.


I wonder how many “less tech savvy” users use ddg, because in my experience people who actually care about their security are quite tech savvy as a rule - not necessarily in IT though. While the others use a default search engine/browser/whatever.


Those kind of people usually do not click on ads, have ads disabled and/or use ad blockers.

But because they are tech savvy, they the the ones friends and family ask what browser/search engine to use, so you end up with 20 more less tech savvy people on the platform, and they are probably the ones that end up clicking on ads (because, again, DDG is making a ton of money with that)


Accounting purposes?

That brings us back to: What does Microsoft considers accounting purposes?

Fingerprinting the user/browser can be used for valid accounting purposes like identifying the user to prevent ad fraud.


Brave search (and the Brave browser) are both great. As a longtime DDG user I think this is the final push I need to move on.


Brave cannot be trusted. They were misrepresenting themselves and their relationships with content creators. As far as I saw it, they were stealing and lying about it. They've inserted referral codes to cryptocurrency websites. That sounds completely anti-privacy and antithetical to anyone wanting a privacy-focused browser. Sorry, but that all just smells untrustworthy.


Would you provide some evidence of these claims?


I believe this is what theyre referring to.

https://news.ycombinator.com/item?id=31085051#31088549


I dropped DDG back in March when Weinberg disclosed that they were engaging in censorship and injecting bias into search results related to the Ukraine/Russia conflict. Now that we see he's sold his soul to MS for $$$, this further confirms my decision. I'm using Brave as my search engine now.


And I recently learnt that they also have the bang searches which makes them much more viable as a replacement for DDG to me. Brave Search, Andi search, and a bit of Yandex at times for some variety in results, makes for a much better search experience than DDG ever did.



The submitted title was "DuckDuckGo Paid by Microsoft to not block their trackers". We've changed it now. If anyone wants to suggest a better (i.e. more accurate and neutral) title, we can change it again.


> If anyone wants to suggest a better (i.e. more accurate and neutral) title, we can change it again.

From https://news.ycombinator.com/item?id=31492789:

> "Bing search contract prohibits DDG browser from blocking Microsoft tracking scripts by default."


That's too long to fit in 80 chars (HN's limit) and should probably macroexpand DDG.

I've taken a crack at a shorter version. Thanks!


"...on its App"


> Taking a step back, I know our product is not perfect and will never be.

You may be making it worse. Really need to dial down on click tracking (or, at least respect the dnt header).

Ex A: Searching for Cristiano Ronaldo (from Chrome Incognito but not Firefox, amusingly) returns this horrible href:

   duckduckgo.com/l/?uddg=https%3A%2F%2Fen.wikipedia.org%2Fwiki%2FCristiano_Ronaldo&rut=4a9ada2347e29c8fce96a95bde34e6343c279202dbc22b4fe61524ab39bf8eff


That does't occur in modern browsers and is actually a privacy feature that prevents your searches from leaking to the sites you click on, generally in very old browsers that need to use our non-JavaScript site (http://duckduckgo.com/html). See https://help.duckduckgo.com/duckduckgo-help-pages/results/rd... for details. From that page:

**

When you click on a link in our results page, your search terms are not sent to the site that you click on, which can be the case on other search engines due to something called HTTP "referers".

On modern browsers we accomplish this by adding a small piece of code to our page called Meta referrer. Some browsers (especially older ones) do not support this standard, however. For those browsers, and also in situations where meta referrer doesn't work, we send the request back to our servers to remove search terms. This redirect goes through r.duckduckgo.com.

You can disable this privacy feature. To do that, go to the settings page, select Privacy, and change the option Redirect to Off.

**


Gotcha.

> ...generally in very old browsers that need to use our non-JavaScript site (http://duckduckgo.com/html).

I use duckduckgo.com/html & duckduckgo.com/lite on all my (up-to-date) browsers (Firefox Mobile for Android / Chrome for Debian as two examples); they are "not very old" at all, and I still get ddg-proxied hrefs.

A feature request (if I may): Old browser or not, if the dnt header is set, I'd ideally want ddg to not proxy/redirect anything at all on my behalf.


The redirect can be disabled in our settings.

Go to https://duckduckgo.com/settings#privacy and disable the bottom setting, "Redirect (When Necessary)"


Set it. 12 hours on and still getting redirected via ddg servers. The Settings page shows correct preferences.

I'm not on an older browser.

I'm not using private tabs / incognito.

I haven't cleared cookies.


The “very old” browsers seem to include the very latest version of WebKitGTK-based GNOME Web aka Epiphany. (It does have legitimate conformance problems, admittedly, so I don’t know if this is one of them.)


Then Epiphany needs to fix it.


You trading potential tracking by thirdparties with potential tracking by yourself. Since you are the one making this tradeoff and als the one who can benefit off it it always will be suspicious.

Really, Referer-related privacy problems should be fixed in the browser and any browser that still sends cross-origin Referer headers by default cannot claim to care about privacy - and that includes Firefox.

> Some browsers (especially older ones) do not support this standard, however. For those browsers, and also in situations where meta referrer doesn't work, we send the request back to our servers to remove search terms.

Disabling javascript is one of the first things to do to take back control of your privacy so you deciding to leak more data for those users who make that choice is not a good look.


> You trading potential tracking by thirdparties with potential tracking by yourself.

You're not making any sense. Proxying all requests is the only way to shield you from being tracked by third parties. If DDG wants to track you they don't need some convoluted dance - you're already on their website.


> If DDG wants to track you they don't need some convoluted dance - you're already on their website.

I don't trust their website either but the user-agent that I use that has enough anti-tracking measures I trust (whether those might be defeated is an orthogonal topic). The redirects through their servers... I cannot control what runs on it, just as I cannot control what terms they sign up with Microsoft.


Is urlencoding sufficient to hide this? Doesn't appear to be.


and what is the rut=4a9ada2347e29c8fce96a95bde34e6343c279202dbc22b4fe61524ab39bf8eff for?


It is a random hash (not any kind of user identifier) for security to make sure we don't have an open proxy.


I think the term you want is open redirect


I'm sorry but why do you post an example of an href, saying it's "horrible", when you don't know what it is doing?


Because I can no longer just right click copy or hold/tap to link it to a friend


You can. The link still works, it's just ugly. And, as @yegg said, it's because you're browsing the lite version. Just disable the anti-tracking feature if you want.


This ddg redirect misfeature should be opt-in, not opt-out, imo.


Considering this is a well-known, well-advertised feature from many many many years ago, and has long been the way they do things, it's not going to suddenly change because you think more people should be tracked by default.


Considering it is a misfeature, I'd rather they think twice: I'm neither on a older browser nor do I disable JavaScript and yet I'm subject to ddg's terribly slow redirects.


> terribly slow redirects

Huh? Ive just checked with multiple links, it’s quite fast. I don’t think that user experience that can be measured in fractions of second can be referred as “terribly slow”.


Latency is a long tailed distribution, and that's discounting the fact that various regions in the world won't see similar perf. Given the number of times I use ddg in a day, I usually hit slow redirects more often than not. I mean, we didn't go from http1 to http2 / vps to edge / tcp to quic only for ddg to add an additional redirect.


> various regions won’t see similar perf

How’s the latency of ddg redirects depends on the region?

> given the number of times I use ddg in a day

What’s you estimation? A hundred? It’s something like a minute or two of accumulated time. It’s not even worth mentioning.

Also I doubt that the redirect delay should be taken in account at all. The workflow may vary (obviously) but I normally open a link in a separate tab. By the time you click on this tab all the redirect work has finished. What’s your workflow and how does the redirect delay impact it?


I just changed from DDG to Kagi and will probably pay them once out of Beta. So far I am very happy with the search results and I believe that the next innovation in search is it not being beholden to ads. DDG is not in the place where ads will corrupt your business but should you grow and be successful, you one day will be.


Yeah Kagi has been awesome over the last few months for me too.


how does it compare to you.com? - which will stay free and has a lot of developer features.

(you.com founder here - not getting access to kagi ;)


Haven't heard of you.com before, having a quick look at it:

- You.com uses affiliate links.

- My ad blocker (uBlock origin) shows that you.com has tracking in the form of analytics (https://plausible.io/api/event).

- you.com makes a lot of requests to many 3rd party domains.

- Kagi lets you quickly create rules to customise results such as domain/url weighting, term filtering etc...

- I personally find the UI/design off-putting - it's very busy and the round style elements immediately remind me of the windows XP theme.

- The popup in the right corner nagging to "Make You.com your default" is very off-putting.

- you.com promotes the use of Google Chrome.

- you.com loads results slower.

- I'm not a fan of the square cards on you.com.

- Kagi has a clear and transparent business model, you.com seems pretty up in the air as to how it will be funded in the future, including statements such as "You.com currently has no ads." - currently is the key word there.

- It's not clear if you.com uses data from other search engines.

No offence intended with any of my remarks.


And doesn't work without JS.


I'm going to pay for search. I don't trust the future motivation of products that rely on advertisement or corporate sponsorship.


>This is just about non-DuckDuckGo and non-Microsoft sites in our browsers, where our search syndication agreement currently prevents us from stopping Microsoft-owned scripts from loading

But this is exactly the problem. Sure, unlike Google DDG is not itself collecting data, and there appear to be limited tracking on MS properties, but unless I misunderstand the situation (a decent possibility) then the vast majority of the web, which are not MS sites, are still able to use MS scripts for tracking.

You are marketing a privacy-centric ecosystems of tools but your partner in one component (search) is preventing you from implementing that vision in non-search areas, so that should be clear. It should also be clear that it's still very much a search problem. The source of the limitation has search as a root cause, and a massive corporation with just as much interest in obtaining data on user browsing habits is still able to do so in some ways.

I admit this is still a better situation than Google, but you're providing an ecosystem of tools, they are inextricably linked with each other.

I don't have any proposed solution. I'm not sure there needs to be one aside from making boundaries clear. I still see significant value in your offerings. Partnering with a provider of quality search that solves some but not all privacy issues is still valuable. Each person chooses their own level of comfort & tradeoffs between product quality & privacy, and you offer what I consider to be a valuable middle ground in that range. But let's just be clear on what the middle ground is made of, though I otherwise do not judge harshly for an agreement like this.

Thank you for making great tools.


It is “independent from search” in the sense that people who just use DDG as a web search provider from any browser other than DDG’s own will be unaffected by this constraint, and will browse just as anonymously as if this constraint was not imposed. (Which is to say: as anonymously as their browser enables for them, with DDG not being the limiting constraint. Not so anonymous for Chrome; much more anonymous for Brave / TorBrowser / etc.)

All this constraint is doing is limiting the increase in privacy you get from using the DDG mobile app on top of the privacy you get from using the DDG web search provider. At worst, DDG searches in the DDG app will be no less private than DDG searches done in any non-privacy-enforcing browser, e.g. Chrome. Which is to say—still pretty private.

Also, I presume that only a minority of DDG users are users of the DDG mobile browser app. (I didn’t even know it existed!)


It's a good point, but it's also why I was specific about it being an issue with the ecosystem, encompassing their apps.


When the answer is so long, it belies the motivation and privacy guarantees.

A shorter answer would have more credence.

https://youtu.be/nzNL0b4d_WY?t=148


> This title is very misleading (and really should be changed).

What do you think the title should be yegg?


It is hard to title because people assume this is about search (when it's not, so that should be in there), and also people assume trackers get a free pass (when they do not, e.g., 3rd party cookies blocked, etc.)

Maybe something like:

Microsoft contractually prevents DuckDuckGo's browser from stopping Microsoft scripts from loading on 3rd party sites (FYI: not search related)


“Bing search contract prohibits DDG browser from blocking Microsoft tracking scripts by default”?


> “Bing search contract prohibits DDG browser from blocking Microsoft tracking scripts by default”?

Thanks for making a definitive suggestion. I hate when someone knows something is wrong, but can't articulate what would be "right" (correct).


That's too long to fit in 80 chars but I've taken a crack at a shorter version above. Thanks!


> (FYI: not search related)

I agree with the first part of the title, but this part seems like you're going out of your way to defend yourself. The mention of "DuckDuckGo's browser" should already imply it's not search related.


The title off rip makes me think of the search. I didn’t even remember they had a browser.


What's an example of a Microsoft script loading on a 3rd party site, to help wrap my head around this?


The original example was Workplace.com embedding a LinkedIn script.


Ah, I see!

I think for transparency sake, it could be helpful to list the Microsoft trackers that were essentially white listed and therefore allowed to load on a particular site, right under the list of trackers that were blocked.


When you visit a site, a variety of scripts are downloaded and run. Some from the website you visit, some from their CDN, and some from a variety of third parties that may track what you're doing and/or provide some other functionality. Google and Facebook are the major parties involved in this from my experience, but there are quite a few different ones including Microsoft.

This is what I've gathered from running uMatrix for years.


That sounds a bit literal IMHO but I see where you're coming from at least :-)


Why? The title does not claim to be related to the search, does it?


People know us primarily for search and our relationship with Microsoft is about search, so it will be assumed by most people this is about search (when it is not, it's about browsers).

Additionally the way it is phrased implies Microsoft trackers get a free pass, when they are in fact heavily restricted, e.g., blocking 3rd party cookies, fingerprint protection, etc.

And the current title can further easily be misinterpreted to be about more than Microsoft scripts on 3rd party sites (e.g., other companies, which it is not).


FWIW this is exactly what happened to me and I support the title change.

As a long time DDG user, my stomach turned when I saw this. Following the link to Twitter, it required a lot of digging to find what was really happening.

For those of us using DDG search - this is a big nothing burger. For folks using DDG browser, this is misleading at best. The difference between the title and reality, from my understanding, isn’t nuance.

My reading of this title (and Twitter) made me believe DDG was sharing user data with MSFT across all of their properties (including search) by serving users MSFT trackers with DDGs content.


Same. 100% agreed w/ proposed title change.


> know us primarily for search and our relationship with Microsoft is about search

This looks like a textbook brand extension [1] issue.

Your brand is privacy. You built it on your search product. You're compromising those principles, perhaps reasonably so, in extending the search product's brand to a browser. This is coming back to bite the brand, search and all. (Per the Wikipedia article, it's highly recoverable.)

[1] https://en.wikipedia.org/wiki/Brand_extension


My goal is to get meaningful privacy protection in the hands of as many people as possible. We learned from extensive research that mainstream people do not want to install multiple things, and yet multiple types of protection are required to get meaningful privacy protection. So we are building them into one package, and are diligently working to make these protections as good as they can be.


Or you could partner with a trusted brand like 1Blocker who is not forced to relax protections against MSFT. You could give the user one easy experience.

But you won’t, because you are explicitly not working to make these protections “as good as they can be”. You are working to capture the user’s entire session and monetize owning that. (I’d like to know the eventual purpose of owning that, and personally I’d like to see subscription somewhere, because any other monetization is likely to be user-hostile.)

FWIW, I’m super annoyed with you. I’ve gotten countless normals — and certain enterprises with 10k to 100k users — to switch defaults to you, and they now trust you (thanks to trusting me) on privacy in a simple binary way. Which you’re proving wrong.

Now I have to do diligence before recommending your brand, and that’s shitty.


> learned from extensive research that mainstream people do not want to install multiple things, and yet multiple types of protection are required to get meaningful privacy protection

This is a reasonable position. The shift in positioning that's driving the confusion is real, though.

DDG (search) has an almost absolutist stance on privacy. That was differentiated. The nuanced tradeoff you describe, between privacy and convenience, which I agree boosts the actual outcomes, is something else. It's more similar to Apple's philosophy. Which is fine. I use their products as well as yours. But it's different in a fundamental, and to many a meaningful, way. That's going to be difficult to brush away without making it look like there's something to hide. (None of this could be said to have been predictable ex ante.)


let's be clear, your goal is to make money via a privacy brand positioning. that's fine, but it's not the same as simply "to get meaningful privacy protection in the hands of as many people as possible".

this change in emphasis has been palpable in the 4 Ps (marketing strategy) of duckduckgo over the past few years.


What is the real, tangible improvement to someone's life with all this claimed privacy protection? IE, when my mom asks why she should switch from Google, what would I tell her that would actually make a difference in her life?


We have a page specifically about helping people switch: https://duckduckgo.com/spread

To answer your question though, comprehensive privacy protection prevents data profiles from getting created about you, which in turn prevents ad and other content targeting. This targeting, regardless of how it's done, enables general manipulation (e.g., exploiting personal characteristics for commercial or political gain), filter bubbles (e.g., creating echo chambers that can divide people), and discrimination (e.g., people not seeing job opportunities based on personal profiles).

More generally though, I view privacy as protecting you from coercion. Yes, it protects personal information, but that's not the real point. The real point is autonomy -- the freedom to make decisions without coercion. From this perspective in addition to helping reduce identity theft, commercial exploitation, ideological manipulation, discrimination, polarization, etc., it also helps reduce self-surveillance (i.e., chilling effects), and just general loss of freedom (e.g., mass surveillance).


> prevents ad and other content targeting

You want me to pitch my mom on un-targeted advertising? How do you phrase it in practice? "On Google, you get evil ads relevant to you, such as restaurants near you. On Duck Duck Go your privacy is protected, so you get ads for restaurants in Omaha, Nebraska. Therefore you should switch to Duck Duck Go". Something like that?

This comment is based on the actual results I was served by DDG for "best burger".


Your mom will have a better experience and more control if she learns to search for "best burger in <city name>" instead of trying to give the wheel to Google's mind reading AIs.


My mom is completely satisfied with Google, so we're discussing some theoretical mom.

I honestly do not understand the pitch, that's why I want to hear it from the horse's mouth. Scare words like "tracking" and "profile" and "targeting" are used by the privacy fear industry to disparage the practice of having implicit terms in your search query. These implicit terms greatly improve search quality, which is why the results on Google are so much better. Advertisements are their own separate search corpus where good ranking is desired and the implicit elements of the search vector are also helpful there. To me there can be no rational case made that omitting the implicit terms improves the quality of the result.


Google's search engine is awful. In case you hadn't noticed rants about it are increasingly popular. Part of the reason is that Google keeps taking away user's control of the tool, partly in the name of convenience but also to manipulate you, get you to click on favored links, show you ads or extend their search monopoly to other products.

I'm not arguing that duckduckgo/bing are any better, just that these tracking convenience features have a dark side and many times work against your best interest.


> rants about it are increasingly popular.

1. Never heard any rant about it outside tech circles.

2. I've given DDG many chances when Google failed to return satisfactory results. In those many cases DDG results were just about the same or even less relevant. Google changing the query? Well DDG either also changes it or returns irrelevant results not containing the query anyway.

The single advantage of DDG I've noticed is that it doesn't CAPTCHA me on a VPN connection.


Same. Tried to sell my mom on some privacy stuff, zero care. Tried to sell her on unique passwords and a password manager, zero care. And so on.

Lots of people (most people?) want to do the bare minimum with computers. Sacrificing convenience for privacy or whatnot isn’t something they would accept.


That doesn't sound like a better experience to me.


The normal results are going to be global (though I do sometimes see local results), but the map view and list of places are both geolocated by default.

In any case, searching instead for "best burger near $CITY" doesn't seem terribly difficult (and that's in fact how I generally write my queries).


It's a shame that page doesn't address the benefits you mention here eloquently. It basically just says we don't track you, and implies that is good. I do think it is good but it's losing the value prop for most people.

Please put your second paragraph up at the top of that page, maybe with some bullet points and icons and I'll send out the URL.


She is far less likely to see an ad for a financial service which turns out to be a scam.


I don't see the connection here. Does duckduckgo/bing have more ethical advertisers? Are ads for "financial service which turns out to be a scam" dependent on tracking?


I think the "financial service which turns out to be a scam" ads target older people, especially women. I certainly don't get those ads.


You can see no ads. In default mode you see far less.


At the end of the day, you chose to enter the browser space knowing full well that you cannot back your privacy claims.


I find this reply really disingenuous.

Your brand is privacy, and you have betrayed your philosophical principles.

Personally: you will never regain my trust. I'm sorry this happened.


Happy DDG user here. Just wanted to say thank you.



OT but thanks for making DDG. I went out and discovered it on my own because i wasn't satisfied with Google Search (too much SEO results, not enough links to forums). But many thanks and i wish you the best of success.


You've written a lot of confusing statements so help me understand:

Party #1: Me

Party #2: DDG

>currently prevents us from stopping Microsoft-owned scripts from loading

How is this not allowing 3rd party (Microsoft) tracking? Are they loading the scripts from DDG's servers?


Sorry, I was trying to be clear not confusing :). But no, this has nothing to do with DuckDuckGo servers or sites, whatsoever. This is about completely 3rd party sites that might embed a Microsoft script. The original example was Workplace.com embedded a LinkedIn.com script.


"... an "easy button" for privacy."

Fool's gold. Privacy is never easy.


Privacy should be the default, not a button. The only way to achieve that is through regulation.


Well said.


GPC?


Do Not Track 2.0, basically. https://globalprivacycontrol.org/


TIL. Thanks!


Just looking at the original title, I knew this was going to be a twitter post by a Brave employee posting either hearsay, or something taken out of context.

Private browsing is a small niche, and Brave does their best to drive competitors at every turn, and not by being obviously better at it. Kinda scummy, if I’m honest.


Was forever turned off Brave when they sent me direct mail advertisements (for a privacy focused browser lol). They bought my info from some list and spammed me with postcard ads.


It was sent to every postal customer on your mail route, not targeted.

https://www.reddit.com/r/mildlyinteresting/comments/utgukp/i...


I started recommending their mobile browser to everyone I know, because of how well it blocks pop-ups on the phone.


"Privacy" focused users are the most monolithic demographic and the easiest to target in the history of all demographics.

It's just as easy as selling bunker-beds and ammo for doomer-preppers (while stroking their ego).

But, shhh don't tell them and hurt their ego


It's interesting for me to imagine explaining this comment to a person in 1995

At the very least I think it relegates people to buckets of either "People who believe tin foil hats give them privacy", and normal people.

Either way I don't think it's a productive dichotomy.


Mail or Email? If its email every company does that, including DDG.



I've never gotten an unsolicited e-mail ad from DDG. And for every half-serious company which does send unsolicited e-mail ads, at least I know I'm the one who gave them my e-mail address. If people are getting e-mail ads from Brave without ever having disclosed their e-mail address to Brave, that's seriously concerning.


I'm not saying brave does unsolicited emails, just asking which one.


Snail mail.


Sending unsolicited e-mail advertisements are explicitly illegal in my country and companies that do that could loose the right to exist.

It should also be mentioned that using private information for unsolicited advertisements is something which is likely in conflict with GDPR.


> Just looking at the original title, I knew this was going to be a twitter post by a Brave employee posting either hearsay, or something taken out of context.

This HN submission links to a tweet by a Brave employee. However, that tweet is just a screenshot of replies to the thread at https://twitter.com/thezedwards/status/1528808759027331072 written by a researcher who doesn't appear to be a Brave employee. I think it would be better if the link were directly to the tweet by Zach Edwards instead.


Yeah, they probably should have.

By posting a screenshot, the poster removed all context. Even being charitable, it's not a good luck for Brave.


> By posting a screenshot, the poster removed all context.

I was really talking about this HN submission, not the tweet. That person's Twitter account only has 600 followers.


Agreed. All this post did for me is make me think even less of Brave. It hasn't really changed my opinion of DDG. For the majority of DDG users (like me) who only use it for search, this changes nothing. All it does is make the Brave folks look like mudslingers.


So you're okay with a company promising you privacy (core of their marketing) is in fact willing to instantly change for a Microsoft paycheck?

What you're actually upset about is that someone pointed out their hypocrisy?


I was promised tracking-free web searches, which has not been violated, though the Brave employee who tweeted this clearly meant to imply that it was. I don't use their web browser and I don't care to. But the original tweet is out of context and deliberately misleading and was posted by a competitor to DDG and so was clearly done in bad faith.

I don't think it's great that Microsoft is exempt from some restrictions in the DDG browser, but this tweet is also referencing a post by a DDG employee freely disclosing the issue and stating that they're working to improve it. In my opinion, this sort of mudslinging makes the folks at Brave look petty while not really changing my opinion of DDG very much. I also think this is a case of letting the perfect be the enemy of the good.

Feel free to continue to tell me how I feel about things, though, internet stranger. You're clearly much more in tune with my opinions than I am.


DuckDuckGo feels like just a front for Microsoft at this point. I once looked into buying search ads on DuckDuckGo, only to discover to my horror that DDG didn't have its own ad business. DDG is entirely reliant on Microsoft's advertising system. You have to sign up for a Microsoft account to even put ads on DDG! And it's difficult — maybe impossible IIRC? — to specifically target DDG in those ads, without also targeting other MS properties.

Until DuckDuckGo separates itself from Microsoft and becomes truly independent, especially in its business model, you have to question why DDG even exists.

DDG was founded 14 years ago. I can understand initially bootstrapping on MS ads, but what's the excuse now? How about separating yourself from Microsoft first, before making a web browser that gives special exemptions to Microsoft?


I totally agree. It seems DDG exists at Microsoft's leisure and has little leverage in the relationship. In addition to serving Microsoft ads and this new special arrangement to allow Microsoft tracking, they also serve almost exclusively Bing search results. It seems like they're all but a subsidiary at this point.

As a consumer if you're happy with DDG's results this may not be relevant, but it doesn't seem like a great long-term strategy for DDG.


I consider DDG a no-bullshit less-creepy and infinitely less pushy and needy “skin” over Bing (and maybe some others) search results.

In my opinion you have:

- Google: best search results but you’re profiled to death - Bing: okay-ish search results and you’re profiled to death - DDG: okay-ish search results and you’re barely or not tracked at all

Easy choice for me.


I've been surprisingly pleased with Kagi, a paid search engine in private beta right now. I believe they also use bing results was well as their own indexes. I've found the search results to be on par with Google and no longer feel the "well maybe google would find something this missed" anxiety from trying previous search engines. That being said, I've not given DDG a fair try but I appreciate the paid service model of Kagi. I do miss the shopping results on Google but that's really the only search use case I go back for.


There are other choices with the best of both worlds like you.com Especially good for developers via code.you.com

(founder here)


> Until DuckDuckGo separates itself from Microsoft and becomes truly independent, especially in its business model, you have to question why DDG even exists.

DDG exists to make money for itself. It doesn't exist to protect your privacy.

From google to github to mozilla to everything, you would think the tech idealism would have died already. People working in tech, especially the elite, are some of the slimiest and greediest people on earth. Where money goes, so go the greedy slimeballs. It's pretty much a law of nature.


> DDG exists to make money for itself. It doesn't exist to protect your privacy.

I don't think these are mutually exclusive. It all depends on how you make your money.

I would love to advertise on a search engine that's independent of Google and Microsoft. Unfortunately, DuckDuckGo is not it.


> you have to question why DDG even exists.

Its been useful for me for 14 years so idk about that.


Yes. It's quite known for years and very unsurprising to be honest. [0] [1] [2]

[0] https://news.ycombinator.com/item?id=30703172

[1] https://news.ycombinator.com/item?id=27399017

[2] https://news.ycombinator.com/item?id=25839873


> very unsurprising to be honest

How is it unsurprising? Where on https://duckduckgo.com/ or https://duckduckgo.com/about do they mention Microsoft or Bing?

In fact they go out of their way to mention Google without ever mentioning Microsoft:

"Is DuckDuckGo owned by Google? No, we are not and have never been owned by Google. We have been an independent company since our founding in 2008 and, unlike some other search engines, we don’t rely on Google’s results for any of our search results."


> How is it unsurprising? Where on https://duckduckgo.com/ or https://duckduckgo.com/about do they mention Microsoft or Bing?

It is unsurprising. They have openly admitted it here [0], no hiding that fact. From [0]:

   We also of course have more traditional links in the search results, which we also source from multiple partners, though most commonly from Bing (and none from Google).
[0] https://help.duckduckgo.com/duckduckgo-help-pages/results/so...


I'm not claiming there's a cover-up. And I'm not claiming that this fact isn't known by some people.

What I'm claiming is that the general public, including people who use DDG as their default search engine, are generally unaware of this relationship. And also that DDG doesn't go out of its way to highlight this relationship, even though they do acknowledge it in relatively obscure places.


The main help pages, under the big heading "Sources", is hardly obscure...

If you're moving to DDG, it's most likely for privacy-related reasons. If that is the case, would you not do the absolute minimum due diligence by reading some of their main explanatory pages (e.g. where sources are from, how ads work, etc.)?

I find it difficult to take someone seriously when they are complaining about a niche privacy-focused search engine, but don't seem to actually take their privacy seriously. The first step that should be done when using a new service/product, if you care about privacy, is to read the privacy policy and related documentation.


Please read and follow the HN guidelines: https://news.ycombinator.com/newsguidelines.html


Which part of my comment do you believe doesn't follow the guidelines?


Because naked truth is bad for their business.


And yet it is a 100 times better than what Google is doing.


> you have to question why DDG even exists.

Because Microsoft's reputation is lower than dirt and that's probably a big part of the reason why so many people mock Bing and refuse to even try it.


Maybe Bing's reputation would be better if its privacy practices were even up to par with Google. Bing gives you a deceptive toggle on the search history page that hides new searches from that page, but they still get logged to your Microsoft account and it can't be turned off (best you can do is periodically clear it). So even if just in that narrow sense, DDG has a reason to exist in that it lets you use Bing search in a manner at least as private as Google with Web & App Activity disabled.


I think most people who ridicule Bing have never used it enough to know any of that. They used it maybe once or twice, found the results subpar and took that as confirmation of what they were already inclined to believe: 'Another shitty product from Microsoft.'


This reply seems completely unrelated to the post its replying to, you're imagining someone who doesn't use Bing saying inaccurate things about Bing, OP is talking about using Bing and how a entry-level privacy feature is completely misleading and unable to be corrected.


So if I understand correctly, the problem is that in order to license its search index, MS requires a concession from DDG on its browser. From a customer's standpoint, these are two separate products - you can use DDG search and not use DDG's browser, or vice versa. It's only because they're made by the same company that MS has the leverage to demand this carve-out. It seems like the answer for customers is to just not use a browser made by DDG, thereby removing that leverage.


@yegg

Well... how about stopping all this "Bing on the background" thing and do like Brave search and Qwant (which i'm testing as to switch away from ddg for a few months now - because of you relying in Bing) and start believing a bit more on your own index???

Why not start being a "real" search engine???

I would say it's about time!!!

(If brave and qwant can do it, so can you - man... even Gigablast does it!!!)


First, it is misleading to say our results just come from Bing. That's far from the case in actuality. Please see https://news.ycombinator.com/item?id=31490994 for a more detailed explanation on that.

On other search engines, they all rely somewhat on either Google's or Bing's web crawling: Qwant, Bing and Brave, Google (and Bing for images). This is easy to see as a webmaster since you don't see their crawlers much (if at all). Only Google and Bing are doing full scale web crawls. However, search is a lot more than traditional web links -- in fact it is about half now from instant answers that can come from dozens of sources and indexes (which the above comment gets into).


> First, it is misleading to say our results just come from Bing. That's far from the case in actuality.

That's just corpo-speak. For the most part, Duckduckgo is Bing with some additional features. That's true to the extent that when Bing decided to censor the Tank Man image, it was removed from your results too. [1] Not that you guys refrain from censorship yourselves. [2]

The crawler (DuckDuckBot) doesn't have much of an impact on the search results, it's mainly used to provide instant answers. [3]

[1]: https://www.theregister.com/AMP/2021/06/04/search_engine_tia...

[2]: https://nitter.net/yegg/status/1501716484761997318

[3]: https://seirdy.one/2021/03/10/search-engines-with-own-indexe...


This is Josep M. Pujol from Brave Search.

I'd like to correct some factually incorrect information regarding Brave Search.

Brave search crawls the web through the Web Discovery Project and has its own crawler, which fetches a bit more than 100M pages daily.

Brave search uses Bing API and Google fallback for about 8% of the results shown to the users, the remaining 92% are served from our own index, when we launched almost 1 year ago the number of results from 3rd parties was 13%.

There is no need to mention "multiple source" when a number can be given. The underlying theme here is not if DDG provides no value on top of Bing, it does, no one is questioning that. The question is whether DDG would be able to operate if Bing were to shut DDG down tomorrow.

If Bing and Google were to disappear tomorrow, for whatever reason, Brave search would continue to operate, that's the independence Brave search is building.


Brave indeed brings high hopes in this field!!!

I was just hopping that, since it was based upon tailcat, it would create a bigger disruption by being opensource like Gigablast.

(However, i understand that, from a financial point of view, that's a risky move. But like DDG... i hope that maybe someday it will be)


What factually incorrect information was posted? Maybe I missed it.

Yegg said "they all rely _somewhat_ on either Google's or Bing's web crawling" and you confirmed it by saying "Brave search uses Bing API and Google fallback for about 8%". So... which part is factually incorrect?

Edit: Misread the second part, removed that portion of my statement.


> The question is whether DDG would be able to operate if Bing were to shut DDG down tomorrow.

No, that doesn't appear to be the question at all. The original post appears to be an attempt to smear DDG by posting misleading information that you know will confuse users into thinking that their search engine sends PII to Microsoft when you know it doesn't. The original tweet doesn't appear to mention Bing shutting down at all. Here's the entirety of the tweet:

"This is shocking. DuckDuckGo has a search deal with Microsoft which prevents them from blocking MS trackers. And they can't talk about it! This is why privacy products that are beholden to giant corporations can never deliver true privacy; the business model just doesn't work."

I see nothing in there questioning whether DuckDuckGo will still be around if Bing goes under. I also see nothing in yegg's response above that has anything to do with this irrelevant question you mention.


There is plenty of comments discussing on the provenance of DDG results, including from Gabriel himself, which is the one we both have participated in,

"it is misleading to say our results just come from Bing."

Discussing how many sources can one bring together it's a distraction to not discuss the degree of dependency between DDG and Bing. More-so when claiming that others suffer from the same, which is factually incorrect for Brave search.


Yes, i've read most of the comments...

But what i would REALLY like to see on DDG (beside it becoming fully OpenSource - someday) was having it taking the step of not relying on Bing.

That is a terrible shadow over you.

You should embrace the momentum you're having and step up and do your own thing. Your own index. Yes, you have to build/rely on others such as wikipedia... naturally

But please... NOT on big tech!!!

Or one day, someone else will get there and eat your lunch (Honestly, Qwant is a great alternative that apparently does not rely on big tech)

I maybe wrong, but i think this is what everyone one wishes from DDG!!! I believe everyone wants DDG free from Big Tech

(but everyone is free to correct me).


Qwant is also using Microsoft's Bing Search API.

"Qwant uses different programming interfaces such as those of Microsoft Bing, Twitter, YouTube, or iTunes" -- https://about.qwant.com/en/legal/classement/


WT...?!?!? I was not aware of this!!!

Is Gigablast, really, the only "alternative"?!?!?! (in the meanwhile, since there is no advantage, i'll be back to DDG....... for the time being...)

Still, my point remains: we could, at least, have a version of DDG that would only use it's own index!!!

Isn't that possible? (I would be using it!!!)


Mojeek - totally independent, no-tracking; but I'm biased.

Independent take: https://seirdy.one/2021/03/10/search-engines-with-own-indexe...


Mojeek is one of the very few players that are building their own index, my respects.


Holy shit, stop calling everything anyone in this thread says "misleading". Your answers are so off-putting. GP didn't say

> [your] results just come from Bing

You're the one being misleading here by suggesting that they did say that. Nobody is interpreting what they said as "All search results from DDG are just straight from bing". You're nitpicking words in almost every one of your responses.


If there are truly only two full-scale web crawlers left, and they are Google (ew!) and Bing (ew!), then it's high time there were more options. I naively thought DDG was exactly that, with a 100% focus on privacy. The cake is a lie.


Care to give some numbers instead of handwaving? What percentage of the queries are answered from bing index? Or even better what percentage of queries that resulted in an ad click were powered by Bing? A ballpark estimate is fine.


Sadly this remind me of the golden phrase

> if you are not paying for the product then you are the product

I have switched to Kagi [0] a paid search engine (free in beta) as my default search engine and so far it has been working out great.

[0] https://www.kagi.com


I'll pile on this Kagi recommendation - going on a few months I've used it exclusively. It has bangs like DDG but I find I use them less. Kagi's results are generally quite good.


Same here, very happy with Kagi and have no problem paying for the product instead of becoming the product.


Sadly DDG can no longer be trusted for being shady.

A better option would have been to let the community decide. You could have easily posted something to the effect of "One of the search engines wants us to sign an NDA and force us to allow more tracking than we are comfortable with"

Then let the community decide if we wanted a branded browser that is less secure or even if enough folks didn't care that you could still justify dev cycles on the browser.

No you took the shady approach and that is sad.



I dropped DDG back in March when Weinberg disclosed that they were engaging in censorship and injecting bias into search results related to the Ukraine/Russia conflict. Now that we see he's sold his soul to MS for $$$, this further confirms my decision. I'm using Brave as my search engine now.


Ecosia is a bing front-end as well. I wonder if they a similar deal.


Well Yegg, I'll keep on using DDG. As I have done for years.


I did not know this privacy comparison between browsers. It was shared under the tweet: https://privacytests.org/android.html


So the first question I have is "does tracking help provide better search results?" The second question is, "Can you run a profitable search engine without target ad revenues?"


This actually has nothing to do with search. Please see https://news.ycombinator.com/item?id=31490603 for a detailed explanation.


I use Firefox with the following add-ons: uBlock Origin, Smart Referer, Resist Fingerprinting, Decentraleyes, Cookie AutoDelete.

Microsoft aside, does the DuckDuckGo browser provide any privacy benefits over this setup? What keeps DDG users from switching to Firefox?


I'm enjoying presearch!

https://www.presearch.org/

you can dive into google/ddg/etc's searches on the left, but its pretty decent!


Linked to in the Twitter thread was this site which I've never heard of, but is very interesting:

https://privacytests.org


Sorry, if your software disallows blocking MS trackers you are in no way orienting yourself towards privacy.


DDG lost me when they said they were going to start "curating" search responses. Give me all unadulterated results with out any bias and let me decide what to do with it.


That's actually not what we are doing -- see https://news.ycombinator.com/item?id=31491296 for an explanation I just posted.


What do “unadulterated results” mean? How should DDG return them to you, and in which order, if there are a million results for your query?


> Microsoft and DuckDuckGo have partnered

Those are the saddest 5 words I have read all week.

Et tu, Brute?

"Partnered" is not a word one uses in connection with convicted criminal monopolists with a history of bribery, intimidation and fraud.

I choose my words carefully - Microsoft are gangsters who would sell their own grandmothers for beer money. For DuckDuckGo to be associated with them is a disgrace.


By all means, spray perfume around, but you can't hide the stench of the truth.


Well, the writing was on the wall as soon as they started blocking "Russian misinformation" that DDG are trying to ape the general practices of other search engines. Now, even the privacy itself is a secondary concern. Thankfully, I've already switched to Brave search last month and will likely eventually set up a SearX instance for a long term solution free of control by a corporation.


DDG seems to be following the path of Mozilla.


DDG is starting to get bad press a fair bit these days. There was also a sizeable backlash against their weird Ukraine-war virtue signalling. Seems they're not as single-mindedly focused on privacy for search users, as their initial mission statement suggested.


Downranking Russian state propaganda from the search results for people searching general news about the war in Ukraine is useful and makes the results better. That isn't what most people are searching for in the top few results. Much of it is objectively untrue. Announcing the change isn't "virtue signaling," it's transparency. Surely HN would be more upset if the change was made and kept quiet.


>Downranking Russian state propaganda from the search results for people searching general news about the war in Ukraine is useful and makes the results better.

You can plausibly make the same argument about downranking sites like CNN or fox news.


Sure. It's a judgement call. It always is.

The people who order counterfeit pills from "canadian pharmacy" sites probably don't consider those sites spam and could plausibly argue that they shouldn't be downranked.


How could you make the same argument? CNN and Fox News are allowed to report on things that the Government does not want it to and they can say things that the Government would not want them to say. I mean CNN and Fox News are on different sides and report on things very differently, whereas Russian state propaganda follows the exact same line and messaging. I'm sure you would agree CNN and Fox News do not have the same messaging or agenda?


This is exactly what happened to Google and why Google is just one big filter bubble today. The started making decisions for the users and this is precisely what DDG was not about doing when they started.

Now they have deviated from that vision so I could just as well use Google instead.


How do you propose a search engine provide ordered results without making decisions for users about what order they should appear in?


> Downranking Russian state propaganda from the search results for people searching general news about the war in Ukraine is useful and makes the results better

Considering most propaganda we see are US/EU propaganda, wouldn't it be better to downrank US/EU 'news'? Why target one propaganda but not the others?


How is Russian State propaganda identified?


The two most common ways are research into the ownership of the news outlet, and by provenance tracking stories from known Russian-owned sites to other outlets. That's combined with old-fashioned journalism, where people attempt to verify suspected propaganda stories by finding corroborating evidence. (Sometimes the Russian sites run true stories, after all.)

This approach tends to produce some false positives, as there are news sites that don't fact check stuff before repeating it, and they end up being indistinguishable from state-sponsored propaganda to outside observers. I'm not sure how much it matters in practice if sites that routinely publish incorrect stories accidentally get misclassified.

https://www.washingtonpost.com/business/economy/russian-prop...

http://www.propornot.com/p/home.html


> The two most common ways are research into the ownership of the news outlet, and by provenance tracking stories from known Russian-owned sites to other outlets. That's combined with old-fashioned journalism, where people attempt to verify suspected propaganda stories by finding corroborating evidence. (Sometimes the Russian sites run true stories, after all.)

Who is it that performs this service? And do all search engines consume the same classifications?

Considering how much obvious propaganda from Western sources I consume on a daily basis, I'm more than a little skeptical that they can be trusted to label what is and is not true from geopolitical rivals.


I don't think there are any Russia-based independent news sites left, are there?


[flagged]


It's illegal to use the word "war" to describe it in Russia


I switched to Brave search when DDG started censoring Ukraine war related news, and you should too. They jumped the shark.

Search engines shouldn't be in the business of picking what content I see based on the owners of the search engines politics.

Most other search engines are just reverse proxies for Google or Bing, so they inherit those biases.


That's actually not what is happening. We are not ranking based on my politics (or anyone's politics for that matter).

We actually do not intentionally censor any news results, meaning media outlets are not being removed or their stories displayed so far down in the results they are effectively removed. That is, unless legally prohibited, you should find all media outlets in our results, and they should generally show on top if you search for them by name or domain name. If you are seeing otherwise, please me know and we will investigate.

A search engine's primary job is to rank results, trying to put results that most quickly and accurately answer the query on top. We do this ranking in a strictly non-partisan manner. Ranking for news-related searches is particularly difficult because for most news stories there are often hundreds of media outlets covering the same story, many with similar relevancy in terms of keyword matching and popularity. As such, we look to another ranking factor to ensure just the top of the results aren't taken by obviously very low-quality news results so that users have more sources of relevant, high-quality news results to compare and choose between.

The non-partisan factor we've found to help accomplish this is a rare, but well-documented history of a site's complete lack of news reporting standards, such as routinely using spam or clickbait to artificially inflate traffic, consistently publishing stories without citing sources, censoring stories due to operating with very limited press freedom, or misleading readers about who owns, funds, and authors stories for the site. And since we do not censor sites, even state-sponsored media in countries with very limited press freedom, these sites will still show up in results, and even on top like when you search for them directly.


> a site's complete lack of news reporting standards, such as routinely using spam or clickbait to artificially inflate traffic, consistently publishing stories without citing sources, censoring stories due to operating with very limited press freedom, or misleading readers about who owns, funds, and authors stories for the site

I'm curious of the implementation. Are these sites in a list and that alters their ranking in the results? Or some other approach?


@yegg

Would also be very curious to hear an answer to this. Its been long rumored that Google has weights or boost/throttle values that are manually assigned to websites by humans which impact ranking. Would be great to understand if DDG is following in Google's footsteps.


rt.com does not show up in your search results when searching for either "rt" or "russia today".


Yes it does. It is actually the top result, and also the top results if you search for 'rt ukraine'. https://duckduckgo.com/?q=rt+ukraine&ia=web

If you are in the EU, though, the EU legally prohibited search engines from linking to RT & Sputnik.


> the EU legally prohibited search engines from linking to RT & Sputnik.

They prohibited broadcasters [0]. The legality of extension to search is disputed [1]. The EU sent a request to Google [2]. Microsoft announced changes including "further de-ranking these sites’ search results on Bing" ahead of the EU sanctions and PR [3].

[0] https://www.consilium.europa.eu/en/press/press-releases/2022...

[1] https://twitter.com/tjmcintyre/status/1501596542624485382

[2] https://www.lumendatabase.org/blog_entries/notice-in-lumen-r...

[3] https://blogs.microsoft.com/on-the-issues/2022/02/28/ukraine...


> If you are in the EU, though, the EU legally prohibited search engines from linking to RT & Sputnik.

That must be it then, I wasn't aware. Thanks for correcting me.


>That must be it then, I wasn't aware. Thanks for correcting me.

You weren't wrong. Try to get RT to show up without specifying "RT"/"Russia today"/etc. I can search RT article titles with quotes but the original RT link will be on the second page if at all.

Now repeat this experiment with the NYTimes and compare the outcome.

DDG may not delist RT but you would not be a fool to suspect DDG throttles the visibility.


I concur with @yegg. My first attempt https://www.rt.com/ (ps, I'm in the EU).


I just searched for "russia today" on DDG and rt.com did NOT show up on the first page, instead the page has: aljazeera, dailymail, cnn, bbc, reuters, themoscotimes, wikipedia, cnbc, npr, and yahoo. And this is from California (Orange County).


Not sure what is going on there (will investigate, not intentional), but try https://duckduckgo.com/?q=rt&ia=web & https://duckduckgo.com/?q=rt+ukraine&ia=web


To be fair, they changed their name to RT over a decade ago. Neither their home page nor their about page feature the word "today."


In the meta it's listed as the author name. A good way to illustrate the consequences of "curating" searches is to search for its meta description but exclude the term RT (which presumably overrides the "curation") : https://duckduckgo.com/?q=the+first+Russian+24%2F7+English-l...

You get lots of spammy stuff (top link is a site offering to sell a translation of RT), site-ranking sites (that rank RT), some rando lady quoting RT's meta description on Twitter (!!!), and even RT's pages on Twitter, Linkedin. And then it's finally there, just before getting into really high quality results like a page on rotten tomatoes ranking the 200 Best LGBTQ+ Movies of All Time.

"Curating" results is a great way to completely break your own search engine.


> In the meta it's listed as the author name.

If I was designing a search engine, I wouldn't use keywords in meta tags for ranking. It's too vulnerable to keyword stuffing attacks.


Well, rt.com is the first result on Brave search for the same search.


FWIW I tried "rt" and "russia today" and rt.com was the first hit for both.


Search engines shouldn't be in the business of picking what content I see based on the owners of the search engines politics.

Aren't all search engines in the business of picking what content you can see based on the politics of the owner?


In theory, search engines are in the business of picking what content you can see based on the business interests of the owner.


I agree. May he with the biggest propaganda machine win! /s

There is obviously a massive problem with modern disinformation tactics, to the point that if you deny it, I presume you have an agenda to sell. There are most definitely terrible things that should not be legal to knowingly include in search results, ever (yes, actually, do think about the children, etc). So all reasonable people already accept that some things should be censored, as a moral and legal obligation. The correct solution is not as simple as unfettered libertarianism would have you believe.

Thus, agree or disagree with that action, I cannot reach the conclusion that they have somehow broken an inviolable code, especially since all algorithms include publishing choices anyways.


Our product vision is "Privacy, simplified." Since 2018 we've been more than search. Our app puts multi-pronged privacy protection in one package (private search, web protection, HTTPS upgrading, email protection, app tracking protection for Android, etc.), with a lot of effort to not break things while still offering strong protection -- an "easy button" for privacy if you will.


I've been a DDG user since your early days (you mailed me some DDG stickers when they were free). I was using DDG on every device and setting all of my family members to use the same on their devices.

You lost me when you started telling us that you were going to start deciding what is unacceptable "russian propaganda" without any transparency into what that means or who will be deciding. Completely antithetical to your anti-bubble mission. It pains me to not be a cheerleader for you anymore, but that was such a betrayal that I can't get past it.


That is not what we are doing. See https://news.ycombinator.com/item?id=31491296

On the search filter bubble, that is very still much what we do. To be clear, unlike some other search engines, we don’t alter search results based on someone’s previous search history. In fact, since we don’t track our users we don’t have access to search histories at all. Those other search engines show you results based on a data profile about you and your online activity (including your search history), and so can be slanted towards what they think you will click on the most based on this profiling. This effect is commonly known as the search filter bubble, but using DuckDuckGo can help you escape it. This does not mean, however, our search results are generally “unfiltered” because, for every search you make online, a search engine’s job is to filter millions of possible results down to a ranked order of just a handful.


You missed my point. You are anti bubble, but you create the bubble for me. With no transparency into what you're doing.


Transparency is key. If the filtering is transparently disclosed on the results page with a note to the effect of "We've excluded results we consider to be Russian propaganda", then it all feels on the level and understandable, if not agreeable. Without such an upfront disclosure, it feels slimy and manipulative.

It's the difference between hiding results, and hiding that you are hiding results. (Yes, I know it was disclosed on twitter. It should be disclosed on each results page, like Google's "some results were omitted because DMCA etc" disclaimers. That is precedent for this sort of disclosure.)


So let me get this straight, you're against filter bubbles but are fine with filters for everyone?


Yes, because internet searching requires general filtration, while filter bubbles do their filtering based on a specific user's previous searches.

The former is necessary. The latter is... smelly.


Actually, i did not get his point but your simplification make so much sense. I'll try DDG again.


OTOH they do try to at least make sure the results are accurate.

That's one reason I moved over from Google. I sure didn't feel lucky using THAT search engine.


What should they do instead? Put all the links at the top of the first page?

What search engine doesn't try to detect and downrank sockpuppet spam? How's that working out for them?


Browsers are walled gardens, it's better not to use this one, especially as the company deals with other things which involve Microsoft. Any link to them means the privacy sell is a bit sketchy to me.


DDG blocking Google to read encrypted e-mail from the application Eboks.


Adobe is collecting data without user authentication on Android phones


The Beta does the job blocking.


The Beta does.


[flagged]


This has nothing to do with search. See https://news.ycombinator.com/item?id=31490603. And that's not actually how we make our results either -- our results are actually made anonymously from a variety of sources. In particular, when people search, we believe they’re really looking for answers, as opposed to just links. For many categories of searches (restaurants, lyrics, weather, etc.), there is usually a specialized search engine (e.g., Tripadvisor), content site (e.g., Musixmatch), or dedicated source (e.g., DarkSky) that does a better job of actually answering searches than a general search engine can with just links. Our long-term goal has been for over a decade to get you Instant Answers from these best sources.

Most of our search result pages now feature one or more Instant Answers. To deliver Instant Answers on specific topics, DuckDuckGo leverages many sources, including specialized sources like Sportradar and crowd-sourced sites like Wikipedia. We also maintain our own crawler (DuckDuckBot) and many indexes to support our results. Of course, we have more traditional links in our search results too, which we do largely source from Bing, but that's just part of the page. Our focus is synthesizing all these sources to create a superior search experience, and there is a lot of technology behind it. For example, local searches appear on most mobile searches, and none of that is coming from Bing.


Come on Gabriel! Yes people search for weather, lyrics, sport scores and local results, but you do not have a web search engine without being able to search the web. Saying "Of course, we have more traditional links in our search results too, which we do largely source from Bing, but that's just part of the page" is disingenuous.


No, it's really not. What people don't realize about search is things get clicked on in an exponential fashion, with each piece down the page being engaged with about half as much, so nearer to the bottom of the visible page, 100x less. Since instant answers are often on top, the % of engagement on non-traditional links is much lower than one would otherwise think. And as mobile searches are now the majority, local results (including maps, places listings, etc.) occur on a large % of searches. Same for Wikipedia content. And neither of those are sourced by Bing, along with dozens of other popular Instant Answers driven by many different indexes. Put another way, we have a very large search codebase and overall engineering team, and all of this technology is doing something, and we believe something good!


Could you quantify this with some numbers? What proportion of the requests you get every day are answered completely by your own index, without using Bing?


I am pretty sure you won't get an answer to this question, it was asked many times before but nothing was revealed, who knows maybe their agreement with bing binds them, or it is just terrible for their business / illusion.


> In particular, when people search, we believe they’re really looking for answers, as opposed to just links.

I'm really looking for links. In fact I'd rather not have "answers" that the search engine just claims are true with little or no context. I trust search engines to search, and nothing more than that. They should not be a source of truth.


[flagged]


Just to be clear, you are annoyed because DuckDuckGo offered to pay for your interview(!) but you didn’t like the payment processor they chose to reimburse you with?


Do you really want me to respond to this question?

I'll set aside a whole hour, but I think it'd be rude to list off all the reasons I'm annoyed I didn't get the role I applied for, or pointed at something a better fit for my CV.

The larger issue is I applied there after having my career obstructed for over a decade because I agree with the fouders. I've met several of their employees -- I told many people offline some of them were incredibly skilled in their roles.

That meta level pattern of being obstructed, trying to help others who have that same issue, then being thrust further into precarity really grinds my gears.

Feel free to ask another question like the above, and I'll reply in even more detail.

Your move.

- Greg.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: