Hacker News new | past | comments | ask | show | jobs | submit login
Ask HN: “Write your own” or “Build your own” software projects
576 points by n_t on March 15, 2018 | hide | past | favorite | 82 comments
I am looking for writings/tutorials/videos which describe a specific technology or feature by implementing them, ideally in no more than few thousands lines of code (and not just 10-20 line code snippets). Idea is to teach about underlying technology by a hands-on project, which is not overwhelming like trying to implement full-feature game engine and yet captures the essence of technology. Some examples are -

* Build a simple database (https://cstack.github.io/db_tutorial/)

* Containers in 500 lines (https://blog.lizzie.io/linux-containers-in-500-loc.html)

* Malloc tutorial (https://danluu.com/malloc-tutorial/)

* Nativecoin - build your own crypto-currency (https://lhartikk.github.io/)

I'm sure there are great such projects/tutorials in domains like networking, filesystem, databases, compiler, web design, messaging, game design, fintech, etc. If you have come across such writings/projects, kindly share.




"500 Lines or Less" is an entire book of articles just like this. Each chapter guides you through a small (500 loc or less) implementation of a common component (eg a web server). http://aosabook.org/en/index.html


Came here to say this. Also; the other books in the series are interesting.


On a related note, I just became aware of hitch, 2nd generation stud(?) - a real-world, modern, high performance proxy server:

https://hitch-tls.org/


This is the first generation stud project - https://github.com/bumptech/stud


Thank you so very much for this resource !


It's still very early days, but Bitwise is interesting (https://github.com/pervognsen/bitwise)

There's also Handmade Hero (https://handmadehero.org/)

The Raytracing books by Peter Shirley are also very interesting, starting with "Raytracing in one weekend" (https://www.amazon.com/Ray-Tracing-Weekend-Minibooks-Book-eb...)

And lastly there's Crafting Interpreters (http://www.craftinginterpreters.com/)


I'm not quite through part 1 yet, but I highly recommend Crafting Interpreters to anyone who wants to see what it is like writing a recursive decent parser and building a language out of it from scratch.


The second part (not finished yet) shows you how to build a VM and compile Lox (the toy language used in the book) too run on it.


Yup, actually planning to take a break after finishing part 1 to do something else, currently leaning towards getting more serious with Rust (to then use on part 2 or another similar style of tutorial)


That's great. I implement part 1 in Rust and I'll do the same for the second part


I started to do it in rust but was struggling too much trying to get the concepts straight AND get more comfortable with various things with Rust, decided to focus on one thing at a time so am doing phase 1 in c#. But as I want to build a compiler in Rust at some point I need to get comfortable with it in this context.


Related to the ray tracing materials, I taught a two-day crash course on writing a ray tracer. The materials are available online: https://avik-das.github.io/build-your-own-raytracer/


That looks really interesting. One question I have. The Peter Shirley material creates something that has a realistic-ish look. The picture on your page, which I assume is representative of the built tracer, has a very unrealistic look. Do you know what causes this discrepancy?


I use the Phong reflection model (https://en.wikipedia.org/wiki/Phong_reflection_model), which gives materials a distinct plasticky look. Peter's materials are either purely diffuse, purely reflective or glass-like (the third one is something I didn't cover in the course at all).

Peter also uses area lighting derived from the surroundings, whereas my implementation uses point lights that result in sharp specular highlights. There are no surroundings in my image, only the lights.

No doubt Peter's materials look great. I wish I had time to cover more in my course, but I wanted to make sure I covered the prerequisite vector math as well for students without strong mathematical backgrounds.


What images are you comparing? One issue might be that three spheres floating in space just isn't something most people see in real life.


You're absolutely right. Normally, the surroundings provide a form of area lighting that my image doesn't have.


I am following bitwise very closely, particularly to get better at C. Any companion text/resource that I should follow along?


NAND to Tetris[0] has similar goals, but is probably significantly different in terms of implementation that it's only helpful to explain the concepts.

[0]: http://nand2tetris.org/


The Oberon book by Wirth is something Per references quite often, and is also linked in the README. I think that is the only companion text you'll really find. Other than that it's reading up on the concepts in general from different sources.


Build Your Own React https://github.com/hexacta/didact

Create Your Own Programming Language http://createyourproglang.com/

Build Your Own Sinatra: https://getgood.at/build-your-own/sinatra (disclaimer: I'm the author of this)

Vanilla JS series https://javascript30.com/

Super Tiny Compiler https://github.com/jamiebuilds/the-super-tiny-compiler


I've programmed my own simple synthesizer [1] by following along "The Audio Programming Book" [0].

It gives a concise explanation for the techniques and theories for signal processing while showing practical code examples.

[0]: https://mitpress.mit.edu/books/audio-programming-book

[1]: https://github.com/badosu/sine_synth.lv2


This looks like much more involved and longer project. But still I'll get your book, looks very interesting :)


I worked through most of this tutorial on building a simple text editor in C and greatly enjoyed it. Highly recommend it!

https://viewsourcecode.org/snaptoken/kilo/


Not just a basic one either, it had syntax highlighting! I always thought that was a pretty involved and complex feature, but it's simplified and demystified here.


Good one!


I've got a bunch of "Build a mini React" [0] and "Build a mini Redux" [1] articles referenced in my React/Redux links list. I particularly recommend the "Didact: a DIY guide to build your own React" post series [2] and "Build Yourself a Redux" [3].

[0] https://github.com/markerikson/react-redux-links/blob/master...

[1] https://github.com/markerikson/react-redux-links/blob/master...

[2] https://engineering.hexacta.com/didact-learning-how-react-wo...

[3] https://zapier.com/engineering/how-to-build-redux/


This describes the purpose of the (paid) resource available at “The Great Code Club” [1].

Marc-André Cournoyer has put together several different projects: - 2D/3D Game - Database Engine - Virtual Machine - Backend + Frontend Framework - Neural Network - Language - Server - Real-Time Web Engine

Full Disclosure: I am a happy customer of Marc Andre’s “Owning Rails” [2] workshop. No other affiliation.

[1] http://www.greatcodeclub.com [2] http://owningrails.com


Very nice! These days think of something and someone somewhere has not just thought about it already but also making money out of it :)


I built a live streaming system with VoD constrained to RasPi+CDN with minimal code in a few weeks, fully FLOSS and offline capable. All design and hours of live hacking are online, comparison here: https://ispooge.com/2018-03-14-video-platforms-compared.html lmk if you want specifics — tutorials being produced on tinydatacenter.com. I am documenting as I learn, given I have no background in vid nor feature list in mind.

Tldr: all you need to make your own scalable Twitch without coding (but with patience). It’s a <2mo old so feedback appreciated, but my live vlog has been going strong for weeks and syndicating to 4+ other networks.


MAL (Make a Lisp) is a good one around building a lisp.

https://github.com/kanaka/mal

Lots of examples across 72 languages.


Came across this website some time ago:

The Architecture of Open Source Applications

500 Lines or Less

http://aosabook.org/en/index.html


I know some of the people behind that book. It's a very interesting take on the craft of programming. I highly recommend it.


It's great book. I liked it too.


I know it isn't quite what you're asking for, but I recommend writing a static site compiler from scratch for a static site you actually use. Buy a nice HTML / CSS template (I found this one https://html5up.net/story then modified it) and build yourself a nice personal site from scratch.

You'll be surprised at how nice it is to just fully understand everything without a mess of libraries getting in the way and the resulting project will be perfectly tailored to your needs.


I recommend the interpreter book [https://interpreterbook.com/], which isn't entirely free of charge however.

On the other hand, it was enough to get me off the ground for my lexer and write a parser that wasn't entirely dumb.


As the author of the mentioned book I just want to add that I specifically wrote it because I'm also a huge fan of the technical writings the author is looking for: from the ground up, all code shown, no toys and shortcuts, capturing the essence in a few thousand lines.

And just FYI, the interpreter we build in the book ends up with ~3900 lines, including the full test suite.


I think I'm atm at around 2500 lines, of which the parser and lexer is probably about 1200 lines. I mostly ~~stole~~ used the lexer from the book as inspiration for my own and then rolled a LISP-like language from there (which is rather easy given that a LISP AST is simple in nature)


Great stuff! Thanks for it.


Building a Modern Computer from First Principles http://nand2tetris.org/


It certainly is a good course and I recommend it too. However, this is a full-fledged class requiring many weeks instead of few hours or days.


I was on the fence about whether to suggest it, but I don't think you should dismiss it as being a weeks-long course.

Instead, think of it as 12 mostly-independent projects implementing a small part of a computer's functionality: the set of logical gates, the ALU, the CPU, the computer hardware, the assembler, assembly programming, stack op implementation, a compiler, and and OS. Any one of those will be only a few days.

(I'm in the middle of the Coursera course for it and have done 1-5 and 7-10.)

http://www.nand2tetris.org/course.php

It only has up to chapter 6 online, but there's an old version of the book available for free but I don't have the link handy.


"Let's Build a Compiler" by Crenshaw: https://compilers.iecc.com/crenshaw/

It's an old classic, using somewhat outdated technologies, but there are more moderns versions floating around, for example a C + x86 one: https://github.com/lotabout/Let-s-build-a-compiler


The "Getting Started with Redux" course on egghead.io, from Dan Abramov, developer of Redux, basically has you write Redux from scratch to learn how it works:

https://egghead.io/courses/getting-started-with-redux


I wrote Hello Web App to teach Django + Python web app development by talking readers through creating a unique project (using a project template described in the tutorial): https://hellowebbooks.com/learn-django

I'm actually fundraising for the new edition of the book, updated for Python3+Django2.0: https://www.kickstarter.com/projects/1868398473/hello-web-ap...


If this were available right now, I would likely buy it! I am tasked with learning Django in the next couple of weeks.

This got me realizing that I usually purchase and consume tutorials on an as-needed basis and I usually want to start right away. I used to collect them but found it wasn’t very optimal because when I found the time to do the learning, I wanted something a little different. For what it’s worth, I don’t think your Kickstarter numbers would account for people like myself.


I bought Tracy's "Hello Web Design" book and it's fantastic! I would assume the Hello Web App book is also quite good.


Here is a Javascript program that implements the basics of the git version control system. It is a literate program with a prose explanation in side-by-side format:

http://gitlet.maryrosecook.com/docs/gitlet.html

Here is another side-by-side example. It is a Python program that implements a very simple interpreter for the Lisp programming language.

http://khamidou.com/compilers/lisp.py/


Build your own really shitty Redis. Then make it less shitty.

A really shitty Redis/HTTP can be built in around 100 lines in many languages (assuming the language stdlib helps you out a bit -- Python, Go, Clojure would all fit).

The fun begins when you make a concentrated effort into making it less shitty. What tradeoffs will you make to what gain? It's a fun space to explore.


Take a look at Gary Bernhardt's From Scratch series:

https://www.destroyallsoftware.com/screencasts/catalog

Not free, but very good.


They are great. Currently the 'A compiler from scratch' episode is free to watch in case you want to have a preview


'How to Build Tinder with React Native & Expo' It's a bit longer and more complex than what your asking for but I just finished building out a course learning React Native & Expo by building a Tinder clone https://www.udemy.com/how-to-build-tinder/?couponCode=LEARNR...


Not my project, but Writing An Interpreter in Go is excellent: https://interpreterbook.com/


Lots of good stuff already in the comments! Here are some examples/tutorials I wrote on my blog:

- A TDD approach for building a simple microprocessor model in Python: http://joaoventura.net/blog/2017/simple-microprocessor/

- Building a basic HTTP Server from scratch in Python: http://joaoventura.net/blog/2017/python-webserver/


Redox OS has `orbutils`, a bunch of simplistic GUI utilites written in Rust and somewhat cross-platform (it's possible to build and run them on other platforms) [0].

The most interesting and educational one for me is `browser`, which crudely shows web pages and is written in less than a thousand lines of Rust from scratch.

[0] https://github.com/redox-os/orbutils/tree/6764004b9f6f386af1...


I enjoyed https://viewsourcecode.org/snaptoken/kilo/ and http://www.buildyourownlisp.com/


A series on building modern business apps https://www.voucherify.io/blog/building-an-online-marketplac...


Writing a garbage collector

http://journal.stuffwithstuff.com/2013/12/08/babys-first-gar...


There is the famous Linux From Scratch [1] which teaches how to build a functioning GNU Linux system from the grounds up.

[1] http://www.linuxfromscratch.org/


I started an awesome list with the content from this discussion: https://github.com/cweagans/awesome-diy-software


Writing a full-featured multi-tenant web app from scratch: https://medium.com/@ashokgelal/writing-a-full-featured-multi...


A pretty concise regex engine, along with the explanation: https://www.cs.princeton.edu/courses/archive/spr09/cos333/be...


Build your own React (https://github.com/hexacta/didact).


There's a Github repo[1] with a collection of project based tutorials.

1. https://github.com/tuvtran/project-based-learning


I think building a web server is a good way to learn (teach) HTTP protocol.


Write your own software renderer, and learn how the principles of computer rendering work: https://github.com/ssloy/tinyrenderer

The wiki has lessons in detail: https://github.com/ssloy/tinyrenderer/wiki/Lesson-1:-Bresenh...


Implementing a Key-Value Store

http://codecapsule.com/2012/11/07/implementing-a-key-value-s...

Let's Make a Voxel Engine

https://sites.google.com/site/letsmakeavoxelengine/


This Tutorial shows how to create a full game (Asteroids-Like) with Love2D and Lua. The game itself is available on Steam. The course is free, and you can pay to get the full source code and the solutions to the tasks. Doing it right now, about halfway through. Really engaging and interesting so far.

https://github.com/SSYGEN/blog/issues/30


Request: Build a JVM (java virtual machine)


JVM might be much more involved project but here is a simple virtual machine - https://github.com/skx/simple.vm


That's my project, and it seems to be surprisingly popular, yet something I've never really received any feedback on.

At the time I wrote it I was modeling the opcodes on the Z80, but I guess I simplified once I'd got it working enough to make myself happy. (Lots of toy-virtual machines, of which this definitely is one, don't implement labels or "decompilers".)


http://craftinginterpreters.com/ part 2 (still a work in progress) will show you how to write a VM and compile a language to its bytecode.

It won't be the JVM but it could be a good start


This is the closest that I could find - https://github.com/lihaoyi/Metascala

Need to find some time to look more and actually try and work thru it.


Unity's roll-a-ball tutorial is pretty good for starting out with their IDE and also gives some decent exposure to C# basics.

https://unity3d.com/learn/tutorials/s/roll-ball-tutorial


This is properly hands-on as you have to make something :) http://www.technoblogy.com/show?NMX

Click on the ATTiny85 tag for more projects. I've made / adapted quite a few of those. Great fun and challenging!


So with these sort of tutorials/guides is it generally a good practice to build on extra functionality to learn more?

I always feel like it's too much hand holding, or I'm just copying/re-building the wheel and not learning enough, but I do very much enjoy them.


Always a good idea! Another thing is to CONSTANTLY question what the article is having you do. Try to disprove rationale given and think of alternative paths. Almost in a playful devil's advocate way.

As the project progresses you'll get more context on why things are what they are than is explicitly written. Eventually you'll hit on something that actually is better and now you're a what-the-article-teaches developer!


Write You A Scheme, Version 2: https://github.com/write-you-a-scheme-v2/scheme

Use Haskell to write a Scheme!


"Write Yourself a Roguelike" is an Open Source project that started as a ebook by a former Thoughtbot employee. https://github.com/thoughtbot/write-yourself-a-roguelike

It's very much still in progress, but certainly fits the brief.


This guide to Build Your Own React is great and even includes React Fiber. well worth it. https://github.com/hexacta/didact


One of my favorites: http://peter.michaux.ca/articles/scheme-from-scratch-introdu...


Before switching to React, I learned a lot about Angular from https://teropa.info/build-your-own-angular/


'Write your own Operating System' by tuhdo

https://github.com/tuhdo/os01




Consider applying for YC's W25 batch! Applications are open till Nov 12.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: