Absolutely - there are plenty of more cost-effective, cloud-agnostic ways to build something like this. This is just an experiment to explore Lambda networking and push it beyond its intended use cases.
Like most things with AWS, it’s tricky to pin down exactly - and it’ll probably cost more than you’d expect once you factor in Lambda invocations, GB-seconds of execution time, S3 request fees, and especially data‐transfer charges.
I did toy around with Tailscale initially trying to get it to spin up as an exit node but wasn't able to get that functional. I did manage to get Tailscale Funnel to work as the tunnel mechanism to Lambda, but unfortunately the performance was really poor.
CalyxOS is an awesome project. I have worked with the lead developer a bit over the past few years and it's been such a pleasure. We share some bits of code between our projects here: https://github.com/AOSPAlliance.
If anyone is interested in building their own custom android OS in the cloud (AWS) with same ability to lock your bootloader like CalyxOS, you can checkout my project I've been maintaining for a few years now called RattlesnakeOS: https://github.com/dan-v/rattlesnakeos-stack.
And if you prefer to not build in the cloud, there is also a really great project called robotnix (https://github.com/danielfullmer/robotnix) which provides a way to build many flavors of OS (AOSP, GrapheneOS, LineageOS, etc).
Could you explain why you would build in the cloud? Based on a sibling comment, it sounds like it might be because it’s crazy resource-intensive? I’ve honestly not heard of cloud building before. Is it common for large projects like operating systems?
Yes, building AOSP requires a fairly powerful machine (at least to do it quickly): https://source.android.com/setup/build/requirements. It's definitely possible to do on a local machine with decent specs though.
By my reading, my not-really-a-gaming-desktop could do it in 3 hours, that doesn't seem bad at all.
Now granted, those were heavyweight specs when Android came out in 2007, but I'd figure about half of us probably have a similar box sitting around today, and the other half would just need to beef one up with some additional RAM.
How expensive is it to build android in the cloud? And how are the build times? Is it possible to do it on a local Mac mini instead or will that take too long to build?
From memory, I think LineageOS 17 took roughly 8-12 hours for an initial build and 3.5 hours for subsequent (ccache) builds on an Intel i5-3570K and spinning hard drive. That's not including the initial git clone.
The idea might seem daunting, but assuming midrange hardware and a decent net connection, it's very much doable in under a day without resorting to cloud services.
I would expect it to scale pretty well, at least until you reach the limits of your disk and buffer RAM.
The build process supports the -j option just like make. You can use -j N+1 if you want to keep all your cores busy, or -j N-1 to keep your machine more responsive during the build, or nice and -j 1 if you're in no hurry and your machine has more important tasks. (Actually, I think reasonable defaults for these might already be part of the build scripts, but it has been a while since I looked.)
With rattlesnakeos-stack, it uses spot instances and defaults to a c5.4xlarge which takes about 7-8 hours to build AOSP and Chromium (for an up to date webview) and equates to about ~$1 a build. I typically build on a c5.24xlarge instance which takes about 2.5 hours and costs about ~$2 per build. Unfortunately both AOSP and Chromium are massive projects that require a ton of computing power to build quickly. It's definitely still possible to do on less powerful machines, but it's just going to take a lot longer to do builds.
In order to limit costs, everything is pulled from source on each build and there is nothing cached. This strategy takes advantage of the fact that AWS doesn't charge for ingress traffic and unfortunately puts additional load on Google's servers. I've attempted a few different strategies on caching AOSP and Chromium source trees, but since you have to incur the storage costs on an ongoing basis, it's just not very economical.
You could certainly do it. No point having multiple EBS volumes lying around - just create a snapshot of the volume with the git checkout / build cache after each new build is done.
When you want to build again, create the instance and then recreate the EBS volume from the snapshot and attach it to the new instance. Pull the latest set of changes from the git repo and build with the old cache!
Obviously there are cache purging considerations (e.g. starting from scratch once per week/month) you could optimise as well.
I investigated EBS snapshot as an option, but there were two problems. 1) cost as i mentioned initially - for just AOSP source tree alone you are looking at > 250GB and at a cost of $0.05 per GB you are already at > $10/month and 2) EBS snapshots lazy load from S3 which gives TERRIBLE performance which means you end up with far far slower builds. AWS released a feature "EBS Fast Snapshot Restore" to workaround this issue, but it's extremely expensive.
That's likely the size of the entire checked out tree, which would include all of the files.
I just ran the command here and my AOSP 11.0 checkout is 54GB, minus any git history, since I clone from a local mirror and use '--reference' to avoid having to copy objects.
A lot of the size here is from the various prebuilts, AOSP build is quite self-contained (jdk, clang, etc) and barely uses anything from the host.
Haha, I was gonna try exactly that, but I got too lazy trying to install pypy3 on my shitty centos box. My other option was to translate it into real assembly, but that also seemed like more work that I wanted to put into it.
> My other option was to translate it into real assembly
I wrote a compiler from emoji-code to amd64 (mostly because I'm more interested in compilers than reversing). It runs quite fast - prints the whole domain in ~1 min. I'd highly recommend it to people who are into assembly, it was a fun exercise.
How did you implement the JUMP_TOP instruction? You need to jump to the x86_64 instructions that correspond to the given emoji index; did you implement a jump table?
That's pretty cool! I just transliterated the instructions into C macros; but i didn't bother with the jump tables. The nice thing with this approach is that you can mix vm instructions with c code freely; and get gdb support. I needed that because speeding up via C wasnt enough to decode the full URL and I still needed to do additional reversing.
Was your method fast enough to get all three parts of the URL?
Yeah, I found 55 profiles/pngs. Tried ordering the profiles by the image # (from 2 to 56). Noticed the profile names are also a variation of two emojis, so tried looking into binary/morse, but nothing.
As the comment above implies, it seems like my approach was wrong and the answer is in one of the images I guess
I might be going the wrong way with that then. Using stegoveritas I noticed that PNG has a bunch of "ISO-8859 text, with very long lines, with no line terminators" files which isn't normal for PNGs.
For anyone interested in doing their own customizable builds of AOSP for Pixel devices check out: https://github.com/dan-v/rattlesnakeos-stack. This doesn't have any of the security hardening features of GrapheneOS, but does have some of the same security properties like verified boot, OTA software updates that included updated firmware/drivers, support for remote attestation, etc.
Agreed. For anyone interested in doing their own monthly signed AOSP builds for Pixel phones with OTA updates, take a look at a project that I built that fully automates the process in AWS: https://github.com/dan-v/rattlesnakeos-stack.
For those that prefer algo, there is an open issue to add Pi-Hole (https://github.com/trailofbits/algo/issues/1258) and in my fork I've added Pi-Hole support: https://github.com/dan-v/algo. It's a really nice setup especially on mobile devices where you typically have to choose between using a VPN or using adblock as they typically require a local VPN hack to function.
Algo user here -- it's really great for ISP-oriented privacy protection. I have a $5/month droplet running Algo in DigitalOcean, and an auto-on mobile config on my iPhone that uses it when I'm not on WiFi.
I also set up my home pfSense router to route all traffic through it (via an IPSec/IKEv2 tunnel). This works great, except when we're using Netflix or Amazon video -- I have to turn it off since those services block DigitalOcean IPs. Still haven't figured out how to configure the pfSense tunnel to route requests to those services through my ISP instead.
If you are looking to maintain a secure device similar to stock Android without google services, a Pixel phone with verified boot using your own signing keys, latest AOSP, and up to date drivers/firmware is a far better option. I've been working on a project that automates the entire process in AWS and it supports all Pixel phones now: https://github.com/dan-v/rattlesnakeos-stack.
I think the Firefox OS and phone was just a little too early. We're approaching the point where the web experience of an application is superior to almost every mobile app. There are exceptions, of course, but I foolishly bought an Amazon tablet recently because my brain doesn't work right when it sees a bargain, but the lack of apps isn't such a terrible thing.
I eventually did put the Play store on it so I could get a few must have apps, but, Reddit is better on the web (though it is an absolute pain in the ass about it with constant overlays trying to get you to use the app). Wunderlist is fine, gmail is fine, Twitter is ok, etc. The big win, however, is that I trust Firefox/Mozilla not to sell me out (I have mostly forgiven the Pocket thing), unlike just about anybody else. That has some value...also, all of my accounts and stuff is synced via Firefox, where with apps I have to login on every new device...some apps demand a fresh login every time, like my bank app, so Firefox is a better experience there.
This. I recently got a new Android phone, and decided to go Google-less with microG and LineageOS. Since I had to install everything from scratch anyway, I only installed apps as I needed them, and I've found that I have not reinstalled a large portion of the apps on my old phone—Amazon, Yelp, YouTube, etc all work perfectly fine in the browser, and I now have the added advantage that uBlock in Firefox blocks all of the trackers that are impossible to avoid in native apps. Hell, many of these "native" apps are just the web versions wrapped in a web view, so there is absolutely no discernible difference in functionality or UX between the two.
I don't know what device you are running but the reddit and twitter websites seem to be built to be absolutely horrible to use. Both get stuck on long loading screens and have constant popups getting in my way.
Try using i.reddit instead of www.reddit not sure how long it will stay up though. When they take those down, I'll leave Reddit (if their main site is still so shit on mobile).
Use the ublock origin extension to remove the parts of Reddit that are annoying.
Use the picker to select the problem overlays and create a rule and they will be gone. You have to do this a few times since some seem to be page sensitive.
Even if you insist on having closed-source apps from the Play Store on your device you don't need to install the actual Play Store or any of its dependencies. Use either Yalp [1] (or its derivative Aurora [2] for those who want a more polished interface) and you can install free Play Store apps without ever needing to sign in to Google.
Banking apps may be more convenient than their web counterparts, because they eliminate the need for a token device for small payments or checking the balance. Depends on the bank, of course.
Holy freaking paranoid waste of lifespan, i have to admit i thought, partly because i'm 51 now and while I dig what you wrote, I think priorities shift as the rear view mirror shows more. Cool project, though.
Thanks for the pointer! How slick would it be if someone extended this project to distribute verifiable binaries which then underwent a final, local keysigning step.
