You can only do this on a tiny number of devices supporting free drivers (and mainline kernel), otherwise you are tied to an ancient Linux kernel. I'm using Librem 5 btw and don't believe that Android, whose development completely depends on Google, is a viable long-term solution.