This book enabled me to think better from first principles.
e.g. How might I go about optimizing a redshift query? Well, now that I have an idea about how data is laid out on disk, because redshift is a columnar store, if I try to optimize X query, here's how I imagine the index to be so that sequential reads would be faster.
I could find a reference on how to optimize redshift queries, but this book answers the WHY and not just the immediate how.
I've read so many books that were practical, yet became so much less useful over time. (e.g. reading a book about the specifics of the Angular API, whereas now I write mostly React.)
I keep returning back to this book for understanding a top-level view of the fundamentals of distributed systems, specifically data stores.
I hope you give the book a second look at some point.
For the longest time I believed that Kubrick knew exactly what he wanted before he began shooting. The finished film existed in his mind, and set pieces and actors simply had to materialize and adhere to that vision. I don't buy into that idea anymore.
An example would be that he's known for making actors do a hundred takes. Apparently he made Tom Cruise walk through a door for a hundred different takes! One interpretation of that would be that Cruise couldn't deliver what Kubrick had in his vision. The other would be that Kubrick was trying to tire Cruise out, so that Cruise be too tired to keep "acting", and bring out something neither of them expected, something "real" and "raw" and not rehearsed.
For a while, I was also obsessed with filmmakers who appeared to not adhere to a strict vision, and "discovered" their film, rather than try to construct the film to fit their rigid vision. The likes of Wong Kar Wai or Godard or Wim Wenders, who would write snippets of dialogue the same day they would shoot and improvise with actors. In particular, Wong Kar Wai was known for shooting for years on end, and discarding 95% of the film he shot. He couldn't have envisioned the final product at the beginning.
I find common in all of their approaches to include exploration. Filmmaking is expensive. It's not cheap like sketching with pen on paper, which is why I think most filmmakers "sketch" with the script, and less so while shooting on set, where every minute they are burning cash.
When I was a young teen, I'd look at the art of movies like Star Wars and I couldn't ever achieve anything like that on my own. What I didn't realize was that that art required iteration. Of course it's impossible to come up with a grand idea on the first try on a large canvas! Those concept artists didn't start by creating the final piece, but they drew hundreds of little thumbnail-sized sketches, playing with elements, this curve they like, or that feature.
I think of how much pre-visualization work went into the likes of Star Wars Episode 1, where you CAN'T stop the train once it's been moving, with hundreds of visual effects workers working on something that Lucas has decided on much earlier. In this, I think that the means of production of how people work have outsized effects on the final artistic product.
I've watched 2001 twice in my life. Once on my laptop screen, and once in a theater. Wildly different experiences.
On a laptop, it was maddeningly slow. I couldn't bear that scene where the spaceship docks on the space station while the Blue Danube Waltz played. Excruciating. I wish there was more explanation, as my mind was wandering watching this dull film.
In a theater, I experienced a completely different film.On the big screen, I could imagine the scale of this achievement, of a man-made spaceship landing on a man-made space station, all this coordination, against the Blue Danube waltz, and it was all I could see in my field of vision. Not distracted by other things as I was on my laptop. I didn't need explanation. I just felt it. I finally felt that I had "gotten" the movie. It was an experience. Not just plot and explosions and super heroes, like most movies are today.
A powerful film is expressed in images! Film. It's called a motion picture. The more a film can be expressed in images, the more "pure" it is to its medium! And most of the film was silent.
I'm glad Clarke got the chance to make it a novel, and told it in the form most fitting for him.
Kubrick was a filmmaker and Clarke a writer. I think it is natural that they would disagree on the telling of this story. Different mediums call for different ways to tell the story.
We forget in the age of constant exposure to video that big screen cinema is not the same thing and that some films use the large screen fully as part of the art.
One of my favorite old movies is "The Stunt Man". One of my most disappointing viewing experiences was re-watching "The Stunt Man" on a friends small screen. Most of the emotional and dramatic tension in the film is driven by the sense/fear that the Peter O'Toole character is omniscient, that no matter what the other characters try he will somehow be ahead of them. This works because the viewer also ends up believing this. But a lot of the work to sell this is done in the viewers peripheral vision and on a small screen it simply doesn't happen which leaves the film sadly deflated.
Kubrick shot it using Super Panavision 70 film and it's supposed to be seen in theatre for the immersive experience. Sadly there are only a handful of theaters that can screen 70mm.
Lawrence of Arabia is supposed to be an immersive, "culturally, historically, and aesthetically significant" epic historical drama film which won Best Cinematography. But I couldn't avoid constantly glancing at the time and my notifications, when I watched it on a smart watch screen that also displayed a bar across the top showing the time and my notifications.
I had a similar experience when watching this through the terminal (img to ascii conversion) on my headless server. It was too easy to get distracted by syslog events that were overwriting stdout from the video.
When I watched the movie via a video player made from Redstone, I didn't realize it at first, but the inexact color reproduction really subtracted nuance from the mood, and the fact that I had to run away from monsters every night really prevented me from mentally investing in the movie. This was all really subtle, but later on when I watched the movie in a big theater, I was amazed at the difference!
I grew up around Washington DC which had some fine large screen theaters. One had a curved screen (the Uptown, IIRC, movies were all in 70mm format) so if you sat in the first few rows, the screen almost wrapped around you. IMHO, the only way to appreciate some of these great movies (2001, Lawrence of Arabia, How the West Was Won) is to see them on a large screen. IMHO, you just don't get to see how spectacular the cinematography is when you watch on a TV or a small screen theater. Unfortunately, most of the large screen theaters have disappeared.
He combines words and images in a style that inspired Wired magazine. The book is about the influence of technology on how and what humans communicate and think.
Your guests will be flipping the book upside-down and looking at the reversed image in a mirror at times. It's entertaining as well as informative!
While not bad (I have it) this is not Marshall Mcluhan's work. It is some other guy's. The text is ripped from Marshall Mcluhan's work of the same name which is a for more exploratory work and not fit for a 2017 coffee table.
It took 33 years before it was publicly announced that GCHQ succeeded in deciphering Enigma codes.
Every single time they used intercepted intelligence to their benefit, they made sure to leave false trails of other ways they could've known so the Germans wouldn't suspect.
Well, from what I've read, that's why they didn't share information about the 9-11 team. And during WWII, they let some convoys go down to protect their methods. It's a tough call. One thread of Stephenson's Cryptonomicon covers this.
Yes, he is one of those authors. Every block of every thread ends with a cliff-hanger, and you need to read two more blocks before you get resolution. But then, you've encountered three more cliff-hangers ;) So after the first read, I bookmark heavily, and read each thread independently. And sometimes I just reread my favorite threads. Such as Bobby's thread in this book. Also his ancestor in Baroque Cycle, "King of the Vagabonds".
Neal Stephenson's next book is The Rise and Fall of D.O.D.O., co-written with Nicole Galland, currently to be released June 13, 2017. It seems to be something in a little different vein than his other works, but I'll read it as I do all his books.
Thanks! I don't know Ms. Galland's work, but it's not much of a stretch from Anathem ;) And consider that collaboration in Mongoliad. So hey, I know what I'll be reading in June.
I can't answer that at all, I'm just saying that if I ever get to the end of that 1100+ paper brick.. I sohuld probably travel to his town to express how good a storyteller he is.
If you haven't yet, read The Baroque Cycle too. It's set in the same universe, approximately 300 years earlier, with an ensemble cast of historical and fictional characters. Very long, very dense at times, and yet a masterful and very rewarding work in my opinion. The audiobooks read by Simon Prebble are also quite good if you don't have the time to read it in print but do have time where you could listen to it.
Yes! It's very cool to get the back-story of the Shaftoes and Waterhouses. And the enigmatic Enoch Root, who arguably appears as well in The Mongoliad trilogy. Maybe he's even mentioned in Anathem, but I may be stretching for that.
ENIGMA is easy compared to modern crypto; it did not have a sound mathematical analysis. Hell there was no such thing as proper cryptanalysis back then.
Academia is much more advanced now; all the tools we have for reasoning about our cryptography were developed in academia, with no evidence that the NSA has shown any interest in that kind of stuff. (In fact the NSA has expressed disdain for academic cryptography).
"There are hundreds of mathematicians in the NSA, far more than in academic settings, so readership for a typical paper is wider than in the world at large ..." [1]
I am still surprised that the Japanese and Germans did not figure out their codes had been broken. The disaster of the U-boot campaign was pretty good evidence of that, if nothing else.
Besides, expecting a widely used and deployed cryptosystem to be uncompromised for years is absurd. They should have assumed it would be broken, and developed regular replacements.
I am still surprised that the Japanese and Germans did not figure out their codes had been broken.
There must have been a lot of people who had suspicions.
But consider: for many years, US citizens who talked about ECHELON were considered crazies. Later, Bush's enormous surveillance expansion was mostly denied or dismissed. The 2016 Russian hacks of the DNC and the propaganda machine were brought up on national television during the debates. Yet there was denial, dismissal, and very little concern.
Without a plan for responding or reacting, denial is a very appealing way to deal with upsetting news. The Germans and the Japanese who were in a position to suspect that their communications had been compromised were also embedded in a totalitarian military chain of command, more focused on preserving the relative power of the people at the top than anything else. Questioning the efficacy of the system is easily cast as disloyalty. What could anyone do?
They organized airplane flyovers that "saw" the U-boats. The Germans did not know how many aircrafts were patrolling and whether it was a high or low probability of being spotted.
If the British could not organize a parallel construction they simply let it go. They knew the plan for Crete invasion but they could not create a story on how they learned it so they preferred to lose naval control of the large part of the eastern Mediterranean sea. [0]
Let's imagine you've figured out that the codes were broken in Hitler's Germany. The only solution is replacing an expensive encryption system with another, equally expensive system, including all the training that goes along with it.
Who do you tell? And who is the guy that going to go to Hitler to tell him that their unbreakable system is broken?
You go to Admiral Doenitz, who already suspected it was broken, and was talked into not changing it by underlings, not Hitler.
BTW, my reading books about it suggests that one was not executed in the military for questioning orders. One reason the German military was so effective is much discretion was allowed by underlings, as well as listening to them.
I'm not well versed on the subject, but I assume it was just another of those large-scale intelligence failures, like the https://en.wikipedia.org/wiki/Englandspiel only with the boot on the other foot. Groupthink in action again. Also, given the large number of important ciphers which were broken during the war, I'd guess wildly that the pre-war crypto communities (such as they were) were generally much too complacent about the risks from cryptanalysis, likely because ciphers had never been subjected to state attack on a Manhattan Project scale before. Comparable to the long time it apparently took for people to become generally aware of C buffer overflows as a serious security problem, maybe.
That's what we actually did, however RADAR was a new thing allowing a small number of British aircraft to regularly intercept Bombers. Without any evidence it must have seemed probable for something similar to be locating subs.
Detection worked just fine at night. The problem was: "Although the RAF control stations were aware of the location of the bombers, there was little they could do about them unless fighter pilots made visual contact." https://en.wikipedia.org/wiki/Radar_in_World_War_II
The codes were changed regularly but the system was compromised. Naval codes were harder to break and often the allies had long periods of being in the dark.
The British did significant amounts of data analysis and traffic analysis. e.g. estimating German tank production by looking at the serial numbers of captured / destroyed German tanks.
I don't recall anything about the Germans doing the same thing.
Another factor (according to the excellent Battle of Wits by Budiansky) was that the Germans were overconfident that Enigma was unbreakable. Turned out their confidence in the hard computation that would need to be done to decode Enigma was wrong.
This was not a trivial system for it's time. Do you change your ssh keys and certificates every day?
The enigma had a new encryption code for everyday distributed on paper and torn off and destroyed once used. The were different codes and machines used in different branches of the army/navy and the system was updated through the war.
The British didn't get to see the machines or it's method for many years. There were 159 quintillion possible keys and even a 1 million guesses/second it would take 5 million years to guess a code - and don't forget they changed everyday. Also, remember there were no computers to do this, let alone one that that could even remotely approach 1 million operations a second.
So you ought to able to see at the time people were pretty confident it couldn't be broken, and if they hadn't made some mistake in it's use e.g. distributing weather reports, it might have not been.
We are arguably much more complacent than they were vs. their time. It was only recently that perfect forward security became a thing in HTTPS for example (i.e. different key for each connection).
Bit unfair, dumping on people who had seventy or eighty years less experience, and profit from widely published literature and history on the topic.
I mean, it's not like they could even go read the Wikipedia piece on the German tank serial numbers info leak. Might have been a feature of a certain seminal strategy game - "you have defeated A[213 of 330]" ;-)
They did have extensive experience with spying in general, and compartmentalization of it. The compartmentalization was not applied to encryption. They also knew that losing an enigma machine to the enemy could compromise it (and did), but they just apparently assumed that no U-boot lost its enigma machine to the enemy.
By the time statistical evidence could have grown strong enough to shine through the careful layers of deception, they were far to busy not noticing that they were losing the war to notice that they lost the encryption battle. In a world of believers, only traitors quantify bad news.
Oh, they noticed all right. From "U-Boat Ace" by Jordan Vause pg. 103:
"retrieved a working Enigma machine along with the documents and code keys for three months. Not surprisingly, U-Bootwaffe fortunes declined in the following months, and from that point on Doenitz remained in doubt about the Enigma cyphers his boats were using. But the experts reassured him over and over again that they were sound, and so he retained them until the end of the war."
I haven't found anything on the subject, but he would have been alive when Enigma become public. It would be great to know his thoughts (and those of people like Speer).
e.g. How might I go about optimizing a redshift query? Well, now that I have an idea about how data is laid out on disk, because redshift is a columnar store, if I try to optimize X query, here's how I imagine the index to be so that sequential reads would be faster.
I could find a reference on how to optimize redshift queries, but this book answers the WHY and not just the immediate how.
I've read so many books that were practical, yet became so much less useful over time. (e.g. reading a book about the specifics of the Angular API, whereas now I write mostly React.)
I keep returning back to this book for understanding a top-level view of the fundamentals of distributed systems, specifically data stores.
I hope you give the book a second look at some point.