> The government initially released the document in a PDF format which renders it like an image, impossible to search. When PDF files are uploaded to the Archive, we automatically run them through an Optical Character Recognition (OCR) process. This turns those images into text, making it much easier to move between sections and search for specific words or phrases. This allows journalists and the public to more easily parse through volumes of information contained within these massive documents.
Shouldn't an accessible, searchable and digital version of government documents be the default form in which they are released?
Why do citizens have to resort to the Internet Archive to see such a basic right fulfilled?
In my country this happens frequently, as well. Too often, documents that should be accessible and widely publicized are hidden somewhere on the internet with practically no incoming links and in a form which is the digital equivalent of a grainy fax.
Sometimes the intention of superficially fulfilling a transparency obligation, without actually communicating anything is palpable.
Much of the policy around classified materials is designed to be idiot proof. I imagine the process for releasing the document was something along the lines of: go into secure room; print out redacted document; review redacted document; take redacted document to non-secure scanner; release scanned version.
Sending digital files across a security boundary is just a much more dangerous operation in general than sending paper. Given the prevelence and quality of OCR (particuarly when run on a good quality print, as was in this case), there is just not enough of a reason to take the risks of sending files across the boundary.
EDIT: Idiot proof is probably not a fair characterization either. The other part of security policies is to be genius proof. What if someone involved is trying to exfiltrate data. There is a lot more places for them to hide it if you give them a digital pathway out then if you make them take it out on pieces of paper that will be reviewed.
One possibly is because they want to make sure the redacted sections are really redacted. You can do a web search and discover that others have tried to redact sections of PDF documents, only to later discover that people can uncover the redacted sections via software tools.
Scanning a redacted document is the simplest way of making sure redacted sections stay redacted as there is no underlying information to reveal.
Releasing the plain text is all that's needed. I fail to see how redacting by overwriting the relevant text is at all reversible - e.g. replace every redacted word with five X's. Replace multiple redactions with ten X's. That actually seems more secure than what's currently done, where you can guess at the words by their length.
I doubt Mueller's team wrote the report in plain text. They probably used Microsoft Word and exported a PDF document.
When I'm president, all government agencies will be forced to use LaTeX. Then we can have both plaintext source, PDFs with actual text, and PDFs that don't leak redacted material.
You nailed it. Aside from that anyone who is serious about search is going to run it through an OCR and get the raw text anyway. Why bother doing more work?
The government releases documents as PDFs, which is the gold standard for document exchange. Here, all that happened is that the OCR text was stripped when the underlying Word document was redacted, and someone forgot to re-run OCR on the post-redaction version. But it’s trivial to fix.
(The article’a belly-aching about the fact someone scanned in the document from paper is misplaced. I have definitely done the exact same thing, because a purely digital conversion keeps all sorts of metadata. Yeah there are ways to strip various layers, but printing to paper and scanning it back in is a sure fire way to strip the metadata and make sure that redactions “stick.”)
> Shouldn't an accessible, searchable and digital version of government documents be the default form in which they are released?
The way it was done was the most sane way from a security standpoint. In my opinion it is the first time I have thought that somebody who knows what they are doing is actually on the case.
I say this because it is just too easy for you to leak out the redacted parts. People have tried, and failed. We don't know if what is under those black blots are really important or not, but if they were, and the leaking of such information could cause somebody harm -- then this is the way to do it.
The government is legally required to create an accessible version of the report for disabled people, and the Justice Department already promised to do so. While I can understand not wanting to delay the report’s release because that version wasn’t available, (1) I don’t know what the law says about eventually fulfilling the requirement, and (2) the Justice Department actually has the original documents and doesn’t need OCR to create the accessible PDF, so I’m surprised they haven’t done so yet.
I suspect the reason they released a scanned version of the redacted document is probably a policy decision to avoid serious screw-ups.[1] Also, organizations of all sizes have known how to photocopy/scan documents for decades so it's a path of least resistance thing. The other thing is that those in charge of releasing this report had no interest in doing anything, including a simple OCR pass[2], on that scanned document that would help disseminate the information in this report faster... it's against their interests to do so.[3] So you're right, a more useful document should have been released. But for the previously mentioned reasons, it wasn't. As long as no critical mass of citizens demands digital competence from their government, their government will not offer it.
[1] There was a story in the last month or two about how someone screwed up redaction in an electronic document in a court case where they effectively only painted black bars over the text but the underlying text was in the release document.
[2] It wouldn't surprise me one bit if lawyers viewed an image-only PDF document dump as a digital form of the 'paper blizzard'. It really isn't (not even close) as there was already an OCR'd version of the document available within the hour of its release and some NLP analysis within the day.
[3] In the event that objections are raised, they can always throw up the 'we were moving as fast as we could to get it released and this was the best we could do given time constraints' defense. Most politicians and judges would likely buy that story since they don't tend to be technologically literate themselves.
> [2] It wouldn't surprise me one bit if lawyers viewed an image-only PDF document dump as a digital form of the 'paper blizzard'.
I highly doubt any federal litigator would think of having to OCR a document as being some sort of road block. Acrobat Pro (which has excellent OCR just a couple of clicks away) is universal in the field. Many federal courts will reject electronic filings if they’re not fully text searchable. It’s almost certain that someone just forgot to run the OCR after redaction.
I really like this OCR version! I did a search for "Manafort" and it not only tells you where in the docs the keyword pops up with nice location markers, but also a snippet of surrounding text to give the keyword context. I do wish there was a low-level cache of commonly used keywords so that some searches would be faster, but I don't know what archive.org's position is on tracking user behavior (probably not a fan).
Thanks to this OCR'ed version, I was able to put an NLP text summarizer of the document into production yesterday. Check it! https://hackernews.hn/item?id=19815506
At this point the whole thing feels like Iraq 2.0 and I wonder if I'm getting suckered by media.
What only worries me as a progressively-disenchanted but firmly and forever a supporter of Bernie Sanders is that even sanders seems to have been wrong on this one unlike Iraq, which I would shout from rooftops that he was the only politician right about. I would (and still do) quote his 'What are you afraid of' video to all who would listen...
Maybe it's because I don't use Twitter, but I find that thread to be incredibly unreadable. Are there any particularly salient points you can point out?
Is Taibbi wrong in his overall point that the media has grossly misrepresented the existence of a Russian conspiracy?
He's said several things about it, and has trashed the Russiagate conspiracy. He emphasizes both that the US interferes constantly in other elections, and also points out that Israeli collusion and interference in the US is orders of magnitude worse than Russian influence:
I think you're misinterpreting him. He doesn't say that it's not a bad thing, or that it's a hoax. He simply says that the US does the same thing abroad and it's hypocritical to not care about that part of it.
No, I don't believe I'm misinterpreting him. He is clearly showing heavy disdain for the assertion that the Russians had any material effect on US elections. He is also pointing out that there is the much greater issue of Israeli collusion that everyone, including yourself, is ignoring when the Russia collusion narrative is a "joke" (his words). Everyone please watch the video for yourselves. It's short.
The search function seems to be broken. I searched for instances of "Russian collusion" and waited for two years for the results and it came back with zero instances of Russian collusion found...
A bunch of people in the White House deleted evidence and used encrypted communications. Not hypocritical at all.
The President probably committed obstruction multiple times, but Mueller refuses to Make a final judgment on that because they believe that is the job of congress.
And the campaign did try to benefit from the Russian influence campaign, and did try to obtain the stolen emails. They wanted Russias help. But there was no evidence of direct coordination (remember the deleted evidence).
The classic issue here is the lack of evidence could mean it didn't happen or that that the evidence is expertly hidden.
However the American spy network is large, well provisioned, has no compunction about spying on Russians and was controlled by Democrats when the alleged conspiracy took place. If there is no conclusive evidence of collusion then it didn't happen.
The idea that the Russians are better at influencing the American people than the Democrat and Republican campaign machines is jaw dropping. It doesn't make any sense. Trump could have found more capable people to collude with if he wanted to.
The Russians were better at spying during the cold war. Why wouldn’t they still be?
Also, the issue isn’t just if there was ‘collusion.’ It was what they tried to do. The Trump campaign tried to work with the Russians but failed because they had no idea what they were doing. They were doing stuff like sending people’s emails to the wrong place.
This Twitter talking point is ignoring the fact that the whole point of this effort was to investigate collusion with the Russians, regardless of any particular crime the investigated are accused of. The report's title is "Report on the Investigation into Russian Interference in the 2016 Presidential Election" no less.
It’s not unreasonable to require that the president of the United States does not commit high crimes while in office. It doesn’t matter why the investigation was initiated. If he attempted to obstruct justice he should be impeached. Period.
The president might be a criminal and you think it doesn’t count because the investigation was started for unrelated reasons?
There’s nothing that can be said over the internet to change the mind of someone who thinks like this, but wow it sure is frustrating to read this garbage.
No one said that he should get away with criminal activity. The problem is that the entire media has been overwhelmed with the Russia collusion narrative for almost 3 years at the exclusion of much more important problems with actual evidence. And now that collusion wasn't found, the Russiagate conspiracy theorists shift attention to petty crimes of little consequence. Most politicians in high office are guilty of similar crimes on both sides of the aisle, and worse. How about: Illegal and unconstitutional NSA surveillance of the entire domestic population; collusion with the medical, financial, and military industries to the point of bankrupting the country; collusion orders of magnitude worse than anything Russia is accused of with Saudi Arabia and Israel, which is backed by explicit evidence, such as the 100 billion arms sale to Saudi, the Yemen war veto, moving of the Israel embassy to Jerusalem, the very existence of AIPAC lobbying, the "giving" of the Golan Heights to Israel, and the unconstitutional anti-Israel boycott laws on the books of many states.
But no, we must waste three years on a phantom conspiracy with no evidence because of political revenge. Consolation prize is we get to complain about obstruction of justice that will never get prosecuted because Pelosi is in on the corruption herself.
From Trump’s perspective it was a “witch-hunt” investigation which hamstrung his Presidency from the start.
And in fact by the end there was not compelling evidence that Trump or his Campaign coordinated with Russia at all.
There were many dirty tricks and bad actors which served to perpetuate the investigation, which by the way cost many many millions for innocent people to defend themselves against. And they are innocent until proven guilty.
Trump knows he did not collude or conspire or cooperate with Russia to defeat Clinton, and yet the investigation grinds on for hundreds of days while he is trying to enact policy to improve the lives for millions of Americans. That’s his mindset.
Flynn, Papadopoulos, and Cohen lied to Congress or investigators, which impeded the investigation, and were charged for it. No such charges were brought against Trump.
We don’t have to speculate about what trump’s mindset was at the start of the investigation. The mueller report gives precise details about trumps state of mind. After learning about the investigation for the first time Trump slumped back in his chair and said to attorney general Jeff sessions “Oh my God. This is terrible. This is the end of my Presidency. I'm fucked. You were supposed to protect me.”
> ”Oh my God. This is terrible. This is the end of my Presidency. I’m fucked.”
> How could you let this happen, Jeff?” The President said the position of Attorney General was his most important appointment and that Sessions had “let [him] down,” contrasting him to Eric Holder and Robert Kennedy.
> Sessions recalled that the President said to him, “you were supposed to protect me,” or words to that effect. The President returned to the consequences of the appointment and said, “Everyone tells me if you get one of these independent counsels it ruins your presidency. It takes years and years and I won’t be able to do anything. This is the worst thing that ever happened to me.”
And I’d say the same thing as a completely innocent man in the same position. Trump truly was fucked, badly, for hundreds of days. Sessions was supposed to protect Trump from a special counsel which did in fact derail his Presidency to pursue a fruitless investigation.
By any account, if the media had their way, it would have been the end of his Presidency.
I’m sort of amazed by his grit in getting through the whole thing intact. Many times I expected he would resign just to try to escape it. I can’t imagine millions of people and almost the entire mass media absolutely convinced that you’re an agent of Russia, Clapper on CNN saying you’re Putin’s asset, all without any evidence.
In fact, the investigation did not establish that there was a “quid pro quo” relationship with Russia.
> “Although the investigation established that the Russian government perceived it would benefit from a Trump
presidency and worked to secure that outcome, and that the Campaign expected it would benefit electorally from information stolen and released through Russian efforts, the investigation did not establish that members of the Trump Campaign conspired or coordinated with the Russian government in its election interference activities.”
It would be more accurate to say that the enemy of my enemy is my friend.
This is not quid pro quo, it’s not collusion, and it’s not cooperation. It is benefiting from blood in the water.
Trump also benefited significantly from Bernie Sanders’ attacks on Clinton and the fact that many of his voters wouldn’t vote for her. Trump did not collude or cooperate with Sanders either.
The FBI "did not establish" that Al Capone had people killed. The court of law "did not establish" that OJ Simpson murdered his ex wife and her friend. Yet they very clearly did those things.
The issue I see with your assertion is that we know from past presidential scandals how enormously difficult it is to elucidate criminal activity; it took years before the Watergate investigations finally uncovered enough evidence to expose Nixon's crimes, and Reagan served a full eight years in spite of his illegal involvement in the Iran-Contra Affair. Yet despite the contents of the Mueller Report, we have a large portion of the citizenry (most of whom haven't even read it) who are already certain that Trump is innocent, after only two years of investigation. The citizenry was certain of Nixon's innocence after two years of Watergate as well; they "did not establish" that he had committed any crimes at that time. But he clearly did.
It appears that you need to be convinced that a crime was not committed. That you have the presumption of guilt and you will stick with that because in your gut you feel like he must have been complicit in the election meddling.
I take the opposite approach. I need to see evidence that the Trump Campaign actually conspired or coordinated with Russia before I will believe or assume that they did.
I also take the position that this accusation was politically motivated from the very start, and has no evidentiary basis. That illegal warrants, illegal leaks, and slanted and straight up false reporting were used in furtherance of those same political goals. And all this makes me doubly determined not to accept anything but evidence before entertaining the notion that there was collusion or coordination or cooperation.
I am definitely not certain that Trump did not collude with Russia. What I am certain of is that there was never any direct evidence of collusion, the media and the democrats knew this, and still decided to run with it to drum up ratings and political capital. And in the end using lies to push a narrative, no matter how good the intention, usually blows up in your face, with the crazy Trump now shored up even more for the next election. I really do hope a progressive wins the next democratic primary. Otherwise we are in for another 4 years.
Users on HN really should not be able to fully erase their posts when they engage in a argument and decide to back out. This kind of attitude fractures the discussion.
It's not really a discussion when everyone on HN thinks the same way politically. There's no point to have political discussions here. If you're not on the liberal train, you get downvoted to oblivion, or flagged. It's just not worth it, noone replying or reading actually wants your perspective, they just want their turn to say their talking point based on some keywords in your comment. That's why I deleted my comments and that's why this is my last political discussion on here.
This is a really good way to shutdown any kind of conversation ever.
Yes there are going to be people that regurgitate their talking points without wanting to listen. But you don't have to respond to them.
If you really have an idea that stand on it's own merits, then it shouldn't matter if it's unpopular unless what you're really seeking is validation from your own camp that agree with you overall, but might nitpick on some small details.
I've had my comments downvoted and flagged in the past. I stand by every single one of them and don't hide behind my opinions being downvoted as an excuse to destroy the rest of the discussion.
Otherwise editing and hiding your posts does nothing but make the rest of the discussion and the people trying to engage you unreadable. What you want is an echo chamber where people must agree with your positions.
"The Mueller report does establish that, in fact, members of the Trump campaign conspired or coordinated with the Russian government in its election interference activities."
Edit: Since the comment I am replying to is no longer present and I got downvotes for my comment: the person claimed there was "no evidence [anyone] conspired with Russia" so I linked to an article in the NYT by a law professor that contradicts this claim in detail. No evidence _is_ a false claim.
At least I believe the NYT isn’t so far gone as to publish such a claim outside of the editorial section, but I may be wrong.
For a different perspective, read what Glenn Greenwald has to say on the subject, in a piece titled “Robert Mueller Did Not Merely Reject the Trump-Russia Conspiracy Theories. He Obliterated Them.” [1]
Edit: I actually took the time to read the editorial. It is a collection, in my opinion, of dubious claims about standards of evidence being too high as the reason Mueller and his massive team could not bring a single solitary charge against Trump or the campaign related to Russian interference in the election.
Please remember how this all started, with a dirty dossier obtained from the Russians by the DNC, with a counter-intelligence operation claiming without evidence the President could be a Russian patsy. And yet here we are, still entirely unable to expose truth to the lie.
>Please remember how this all started, with a dirty dossier
Nope, it began when some foreign diplomats gave us a heads up that George Papadopoulos was bragging abroad that the Russians were going to help Trump by releasing dirt on Clinton. That the investigation was predicated on the dossier is a common misconception. If you are getting news from sources that get this basic fact wrong, you may wish to downgrade those sources' credibility in your estimation.
That happened in “late May 2016” right around the time that Fusion GPS hired Steele.
Note that Steele and Bruce Ohr were already in cahoots for several months by then, and Steele was already established as an CI with the FBI (a cover he later blew).
Papadopoulos met Professor Mifsud in London in March, and later Misfud said he had ties to the Kremlin and he claimed Russia had dirt on Clinton. Papadopolous sent an email two weeks after the London bar meeting saying Misfud introduced him to the Russian Ambassador to England and that the Ambassador might be able to help setup a meeting between Trump and Putin.
It was what Mifsud told Papadopolous that he would later talk about with the Greek or Australian diplomat that would be reported back to the FBI.
Interestingly, there are reports that Mifsud is actually a western intelligence officer, and has no links to the Kremlin whatsoever.
Chris Steele (an agent of a foreign government) was dead-set on preventing Trump from becoming President.
He had high level contacts at the FBI and DOJ which he was in contact with throughout early 2016.
Hopefully one day it will become fully clear what Steele was telling Bruce and Nellie Ohr, up until the very day before the FBI opened their investigation, when Nellie met Chris at the Mayflower Hotel on July 29, 2016.
But we do know at least that it was lies, mostly (Russian?) lies, that Steele was feeding the FBI. And we know certainly those lies had a bearing on the character and direction of the investigation, and on the methods employed to attempt to gather evidence (FISA), even if it is not the publically admitted initial basis for starting the investigation.
Mifsud, for his part, has gone to ground and isn’t taking interviews.
Not sure why you were downvoted, you are correct. It says as much on Page 1 of the Mueller Report:
> In late July 2016, soon after WikiLeaks's first release of stolen documents, a foreign
government contacted the FBI about a May 2016 encounter with Trump Campaign foreign policy
advisor George Papadopoulos. Papadopoulos had suggested to a representative of that foreign
government that the Trump Campaign had received indications from the Russian government that
it could assist the Campaign through the anonymous release of information damaging to
Democratic presidential candidate Hillary Clinton. That information prompted the FBI on July
31, 2016, to open an investigation into whether individuals associated with the Trump Campaign
were coordinating with the Russian government in its interference activities.
I'll stick with the actual quote from the actual report:
"the investigation did not establish that members of the Trump Campaign conspired or coordinated with the Russian government in it's election interference activities."
Regardless of what any opinion piece wants to claim, in America you are innocent until proven guilty. Beyond a reasonable doubt is not just something done just for fun, but an important legal standard:
Beyond a reasonable doubt - The standard that must be met by the prosecution's evidence in a criminal prosecution: that no other logical explanation can be derived from the facts except that the defendant committed the crime, thereby overcoming the presumption that a person is innocent until proven guilty.
We started out trying to determine whether Trump colluded with Russia. This is a very important thing for the American people to know. Trump obstructed the investigation and we weren't able to determine whether that happened.
If obstruction didn't count unless you were convicted of the crime you were obstructing justice on, people could get away with crimes by interfering with the investigation and there'd be no recourse. We don't want people to get away with crimes by interfering with the investigation. That's why obstruction is a crime.
I believe it’s a well established legal theory that a President can not be charged with obstruction for exercising his Constitutional duties.
For example a President can issue pardons. Even for crimes which have not yet been charged or tried. This clearly “obstructs justice” and yet is also very clearly not obstruction of justice in a legal sense.
I am skeptical about any claim the investigation simply failed to find evidence that was there to be found, if not for obstruction by Trump. This was a very expensive, very long, very comprehensive investigation. My understanding is that Trump waives executive privilege and have quite unprecedented levels of access to his cabinet, staff, and even White House counsel.
The report very clearly states that their investigative efforts were stymied by lies from witnesses, and deleted evidence. If you combine this with the President's behavior and the rest of the report, it requires a larger assumptive leap to say that they did not coordinate with Russia to influence the 2016 US elections.
The leap is on persisting to assume that they did, when the special counsel had access to FISA warrants, NSA wiretaps, electronic eavesdropping, interviewed hundreds of witnesses, and generally investigated the hell out of this for hundreds of days, and in the end could not conclude that there was any cooperation or coordination with Russia.
In short, the full force of the USG investigated and came up short. I think the conspiracy is laid bare.
If you expected the report to conclude that the President/Campaign did not coordinate with Russia I’m sorry that was never going to happen. Because you cannot prove a negative.
You will always read a statement to this effect;
> Accordingly, while this report embodies factual and legal determinations that the Office believes to be accurate and complete to the greatest extent possible, given these identified gaps, the Office cannot rule out the possibility that the unavailable information would shed additional light on (or cast in a new light) the events described in the report.“
Because you can always keep trying to investigate a little longer. Interview and re-interview another hundred witnesses. Review another million emails. Listen to about hundred thousand hours of intercepts.
But if there’s one thing we can say and everyone will agree, this investigation was comprehensive. It was extensive. It was extremely professional (plus or minus somewhat gestapo raid tactics).
But at this point the only thing to be said about claims that the Trump campaign coordinated with Russia to influence the election, is that they are truly without evidence.
I'm not saying that we necessarily didn't find evidence because Trump obstructed the investigation. Maybe he was innocent of collusion and obstructed to prevent us from finding out about other crimes or embarrassing things. Maybe he had other reasons.
The point is that we don't know what we weren't able to learn because of his obstruction. Maybe it was just an unsuccessful attempt to cover up the things we did learn about. Those were certainly bad enough, but trying to cover them up could have prevented us from getting enough information to determine that his campaign wasn't colluding with Russia.
The strongest case that Mueller’s team could find to establish a charge of obstruction was Trump’s conversation with Comey where Comey said that Trump told him, “I hope you can see your way clear to letting this go, to letting Flynn go.”
You can argue if that is obstruction or not, but I don’t see how it impedes the investigation to hope someone is not charged with a process crime.
> EDIT: to repliers, the person I replied to said "we weren't able to determine whether that happened" that's why I asked how he obstructed a report that was finished. I'm not debating whether or not Trump obstructed justice.
He obstructed the investigation leading up to the report. No one is saying he obstructed the report after its release.
> EDIT: to daveFNbuck, in America, you're innocent until proven guilty. If someone can't conclude you committed the crime beyond reasonable doubt, you're innocent.
That's why we've made obstruction of justice a separate crime regardless of whether you can be found guilty of anything else. Obstruction is a separate offense that you can be found guilty of.
Lack of evidence is not a conclusion. It's what you'd expect if there were successful obstruction of justice. Even Barr's summary quoted that the report "does not conclude that the President committed a crime, it also does not exonerate him."
The matter of why 'collusion" wasn't directly addressed ... is directly addressed, in a titled section of the Introduction to Volume I, on page 2:
In evaluating whether evidence about collective action of multiple individuals constituted a crime, we applied the framework of conspiracy law, not the concept of “collusion.” In so doing, the Office recognized that the word “collud[e]” was used in communications with the Acting Attorney General confirming certain aspects of the investigation’s scope and that the term has frequently been invoked in public reporting about the investigation. But collusion is not a specific offense or theory of liability found in the United States Code, nor is it a term of art in federal criminal law. For those reasons, the Office’s focus in analyzing questions of joint criminal liability was on conspiracy as defined in federal law.
Again: "collusion" itself is not a legal term. Conspiracy and obstruction, among many othrs, are.
Actually that's an interesting research idea. Calculate the total hours of airtime and/or news articles written. I bet it would be close to a continuous month if not more.
Shouldn't an accessible, searchable and digital version of government documents be the default form in which they are released?
Why do citizens have to resort to the Internet Archive to see such a basic right fulfilled?
In my country this happens frequently, as well. Too often, documents that should be accessible and widely publicized are hidden somewhere on the internet with practically no incoming links and in a form which is the digital equivalent of a grainy fax.
Sometimes the intention of superficially fulfilling a transparency obligation, without actually communicating anything is palpable.