I feel like I'm on one extreme of this spectrum which is: I 'need' to develop a deep understanding of systems from the bottom-up and only then can I feel comfortable building on top of it/abstracting it away/being productive with it.
When I started learning socket programming I didn't feel completely comfortable with using it until I had a complete understanding of the networking stack from the PHY upward. I even needed to write my own simple ARQ implementation to feel like I could fully appreciate and have a "feel" for what two machines are "actually doing" when they're sending packets back and forth.
I had the same obsessive desire to completely understand from the bottom-up how database engines were implemented because I just didn't feel comfortable with just the knowledge that "selects on tables with indexes on the field being queried are 'fast'". I just _had_ to know what was making them fast. Funny thing, when I found out that databases and database indexes were just disk-persisted/in-memory self balancing search trees or hash tables, I thought to myself, "that's it? why didn't anyone just tell me that instead of just repeating "queries are fast/optimized by the db"?"
I don't know if my brain just works differently because it seems most people I've worked with seem to get frustrated/annoyed with these details and not even have any desire to learn a system in-depth like this. They seem to only want to reason about a system at its highest level of abstraction, whereas to me, with the database example, once I learned that B+ trees were one of the primary data structures that backed database systems, I felt like the system as a whole just became that much more transparent and easy to reason about because now I could leverage all of my intuitions+past experiences relating to performance of operations on tree structures, having an intuition of perf implications of using linked data structures, reasoning about issues like paging chunks of data in and out of disk, etc.
You just have an interest in different parts of the stack. Someone could write the same thing, but instead talk about why they needed to understand how the computer hardware was implemented to run the programs and even needed to build a small computer themselves to get a feeling for what the computer was "actually doing".
Alternately, at a systems design level, the constraint may not actually be the implementation of the components, but the organization and production of the system. This could be a simple CRUD app, where the business domain is more important, or the manufacturing of a new toy, where cost retooling and speed of production matters significantly more than trying to make the toy of a slightly higher quality.
There is a LOT of complexity out there in the world, and most people (on hacker news especially) gravitate towards parts of that complexity that interests them. A lot of other people gravitate towards different problems, which is a good thing, because it takes all sorts of specialties to get something nifty off the ground.
That's a really good point. I've also done what you mentioned in your first paragraph but stopped digging deeper once I hit the "now I have to understand physics to actually understand why electricity behaves the way it does in order to flow through transistors/logic gates" barrier. My reasoning at the time was a combination of asking myself how much time I actually had to dive into that as well, plus how much of a potential return I thought I could get from having resolved those questions.
Both this and your above comment might just be the most uninvited and self-absorbed accounts of internal reasoning I've encountered in a neutral space online in a long time. Maybe you can do a deep dive on your presentation and determine the abstract reason why you think colleagues aren't interested in the same things as you (hint: it's not the topics).
Seems like I struck a nerve here... I didn't realize I'm not allowed to use HN to start discussions with other readers about topics they bring up :)
I like to post about these kinds of thoughts on HN because I feel the crowd is the type to offer useful insights I don't find elsewhere on the internet or even IRL. Do I have a tendency to ramble? Sure, look at my post history...
If you have any insights about what I mentioned (a thing I've been curious about for a long time), I'm all ears - but I don't see how being salty is contributing.
I can understand why my comment reads "salty" - I apologize for that. In all honesty, based on what you said, I actually think you and I have much in common (having undergone similar technical exercises for the pure sake of understanding).
My issue was/is your delivery (albeit typed and perhaps it is just the medium, again, apologies) - you didn't start a discussion, even when the parent responded you hardly even acknowledged what they said before you simply doubled down on speaking "at" them/us about yourself and your interests in affirmative statements (that in all honesty read like you're "one-upping" everyone for some reason) that leave open very little room for an equitable response.
I guess, what I'm trying to say, is that I think human nature is such that people want to feel comfortable with another human before diving into a technical topic with them (even on HN!). That rapport is subtle and relies on so much more than just factual knowledge or a common project; it needs genuine, mutual interest in other people, and if I could make any suggestion (advice I try to take myself) it would be a more polished delivery. Or, as my mom always told me, "it's not enough to be right all the time - you have to be kind, too."
Somewhat related; I've always been very curious why a lot of developers I've worked with seem to think testing isn't worth the effort, and I've been keeping a mental checklist of all the reasons/excuses I hear so that I can reflect on them as well against my own experiences - sort of a way to challenge myself and ensure I'm not just cargo-culting methodologies.
Recently I decided to go through `Growing Object-Oriented Software - Freeman, Pryce` (and actually finish working through it this time) with the goal of understanding what "proper TDD" is supposed to look like. Something interesting I noticed is almost all the complaints I've seen as reasons against TDD/certain testing methods all seem to be examples the authors use as how not to use TDD/testing. It seems to me that a lot of developers have just learned _of_ these techniques by name, and haven't really put a lot of time/effort into practicing their application and instead just seem to write them off on face value or by the literal interpretation of their names.
One example, a lot of people I speak with seem to think TDD is very literal "write a _unit-test_* for everything before you write the implementation OR ELSE...", but of course there's a lot more nuance than that depending on the situation. The book actually puts a huge emphasis on having your initial test(s) be end-to-end tests that slice through the system as a whole, with the idea being to create nested feedback loops of varying granularity/abstraction to allow you to iterate without fear. This was something I never heard emphasized at all when I started learning about TDD, or hell even in a paid course my employer put us through by a "TDD expert".
I should also clarify, I only consider these claims from developers that have a proven track record of working on large/complex systems since, well, those who don't probably haven't cultivated their ideal workflows/approaches yet (or just haven't been given the opportunity to showcase them, as is common in large companies).
* I emphasize unit-test because a lot of code bases I've worked in rarely have anything other than just unit-tests...
I agree. Everyone should consider the costs and benefits of testing to arrive at a system that works for them.
Early on in a system design, I prefer going overboard with integration tests.
The architecture is hard to predict early on, so I don't want to get paralyzed on figuring it out. Just get the tests passing without overthinking.
Once I find a better way to do things, I can rip apart and restructure the internals with the safety net of end to end tests.
I rarely write unit tests at this stage because the effort of writing and rewriting them adds friction to getting the software working and restructuring the code.
If every time I redraw a boundary, I need to rewrite a bunch of unit tests, I'll be less likely to improve the code quality.
I had to investigate why our docker-ized dev environment was so slow on macOS at my last job and this was the root cause. One project's test suite ran in about 5-10 mins in our linux CI/CD environment, and about 50+ mins in macOS with docker.
There was a very in-depth thread on the docker forums where the devs explained why there was such a huge performance penalty. IIRC it was due to all the extra bookkeeping that had to be done to ensure strong consistency and correct propagation of file system events between the virtualized docker for mac environment and the host file system.
The test suite would run integration tests that performed a lot of npm/yarn operations which meant lots of disk IO.
Curious, does one need to use nixOS for an optimal experience?
I've been using debian for a long time and would prefer not to have to switch OSs but the idea of using nix for having full control over my package graph is very tempting.
I've been somewhat procrastinating on trying nix as I've heard GNU Guix has a similar feature set and haven't been able to decide on which one to dive into...
My ideal setup would be to just be able to run a single shell script that configures a new machine to the exact state of all my other dev machines. I have a shell script that somewhat does this but it's not completely unattended and still requires a lot of manual config for certain steps.
Aside from that my main use case is being able to easily share a dev/build environment with others for ensuring that they can compile a certain project exactly as I do. For now I just use docker but it's frustrating not having explicit control over the layer cache and being able to tell it what to cache and what not to.
> Curious, does one need to use nixOS for an optimal experience?
My experience: no. I've been very happily using Nix on debian for years. Debian gives me my boring desktop apps (I have very very boring tastes), almost all my tinkering and development starts with drawing all the prerequisites from Nix.
The only times you get weirdness from using non-NixOS linux are things like running opengl apps, which simply require a wrapper script like `nixGL` to function properly.
And darwin Nix is just about the only thing that makes macos a bearable platform for me.
Sounds to me like you're focusing on the wrong things or don't have the right resources for the task at hand. (that is, I don't think I've ever had to brush up on a proof for any algorithm or data structure when prepping for interviews)
Take it from someone who also had a shit background/education in math/CS and had to teach myself a lot of these things from scratch - it's doable, you just need time, and I guess the mindset that you can learn these things. See my comment history for some book recommendations. If you really feel like you need to learn things from scratch to give yourself the confidence of having a solid foundation, look at some of the intro computer science courses on Coursera. When I realized most of my uni's cs courses were... not great, I just ended up going through courses like Stanford's Introduction to Algorithms on Coursera and teaching myself.
I can also tell you as someone who was in Seattle for a while and had a bunch of friends at Amazon, I don't think anyone is going to think less of you, if anything, being able to survive at a place like Amazon seemed to be recognized as its own accomplishment...
edit: I'll also add (from personal experience), if it still feels like these things are difficult even once you feel like you have all the right resources, look into seeing medical professionals to make sure you're not being held back by anything, like sleep apnea, any cognitive disability (not sure if that's the right term...) etc
This has been my take on it as well. Sure it sucks having to jump through these hoops and sometimes even being evaluated by people you know don't understand the topic as well as you and are looking for specific answers/solutions, but this same process also helped me, a "C/D student" from a no-name school and a poor family with 0 connections in the tech industry end up with job offers in the $300k+ range.
The only thing I'm salty about is that I can only get these offers in the US and not in Canada and that my options for permanent residence in the US seem to be gated behind an 80+ year wait... https://medium.com/@happy_sushi_roll/the-endless-wait-for-a-...
These types of questions basically boil down very cookie cutter patterns that generally bucket into graph traversal/search, being comfortable with coding/manipulating linked data structures, enumerating+pruning search spaces, and knowing little tricks to reduce an order of magnitude from the runtime or space complexity after having applied one of these patterns. These all generally require knowing all your basic data structures as a prereq (stack/queue/deque/vector/hash table/self-balancing search tree, then using those primitives to compose graphs etc).
My advice for people who wanted to prep for these interviews originally used to be to read through some books that prep you for algorithm competitions (e.g. ICPC, IOI, GCJ), but I find that to be a bit overkill and not as efficient a use of time (although some of these books do a much better job of explaining things than any of these interview prep books I've gone through). Today the advice I'd give is get a book like 'Elements of Programming Interviews' in the language you plan on using and a leetcode subscription and just get down to learning about a data structure/algorithm/problem solving paradigm and then solving a bunch of questions that fit that technique.
If you really want to go with the overkill approach, I'd recommend a book like Competitive Programming 3: https://cpbook.net/
It's interesting, when I tell people they need to be prepared for this bar I get a surprising number of people who just don't seem to want to accept that it's true or flat out claim that it's not true when I know they haven't interviewed in years.
In the past 2 years I've done about 40ish+ interviews at all the big tech co's + smaller-midsize startups and aside from 3 or 4 take home assignments I was given, _every_ interview was leetcode style, to the point where I would just start noting which exact leetcode questions I was asked when friends asked about my experience. The only exception would be 1 or 2 "system design" rounds out of the 5/6 on-site interviews if the company was calibrating me for senior level.
Hell, I'm Canadian and I also bucket into the India group since I was born there. Doesn't seem to matter that I was less than a year old when my parents immigrated to Canada either.
yea, its more of a quirk of where you were born rather than nationality though both are almost often the same. I have an Indian friend who was born in Iran because his dad was working there when he was born. He lived in Iran less than a year but got his green card in less than 3 months.
When I started learning socket programming I didn't feel completely comfortable with using it until I had a complete understanding of the networking stack from the PHY upward. I even needed to write my own simple ARQ implementation to feel like I could fully appreciate and have a "feel" for what two machines are "actually doing" when they're sending packets back and forth.
I had the same obsessive desire to completely understand from the bottom-up how database engines were implemented because I just didn't feel comfortable with just the knowledge that "selects on tables with indexes on the field being queried are 'fast'". I just _had_ to know what was making them fast. Funny thing, when I found out that databases and database indexes were just disk-persisted/in-memory self balancing search trees or hash tables, I thought to myself, "that's it? why didn't anyone just tell me that instead of just repeating "queries are fast/optimized by the db"?"
I don't know if my brain just works differently because it seems most people I've worked with seem to get frustrated/annoyed with these details and not even have any desire to learn a system in-depth like this. They seem to only want to reason about a system at its highest level of abstraction, whereas to me, with the database example, once I learned that B+ trees were one of the primary data structures that backed database systems, I felt like the system as a whole just became that much more transparent and easy to reason about because now I could leverage all of my intuitions+past experiences relating to performance of operations on tree structures, having an intuition of perf implications of using linked data structures, reasoning about issues like paging chunks of data in and out of disk, etc.