Qualcomm has had DSPs in its chips for a long time, providing a lot of NPU-like functionality before the term NPU had been coined. What Qualcomm currently calls its NPUs are just Hexagon DSP cores with specific instructions and abilities for matrix math and common inferencing datatypes.
I am on the Vesuvius Challenge team. We came across this press release back in July and were quite impressed! It's great to see other groups using non destructive means to read ancient documents.
They say they're hiring but I didn't even get an email back about my application and I've been awarded $20k through the vesuvius challenge and have 10 years experience in the exact job they're hiring for so I really don't know what they're looking for or if they're looking that hard.
I am an ex Qualcomm employee. We often called ourselves a law firm with a tech problem. QC doesn't actually have more lawyers than engineers, but I'd not be surprised if the legal department got paid more than all the engineers combined.
Same, these are all great points that I find as well. LLMs have made me a way more productive programmer, but a lot of that is because I already was an alright programmer and know how to take advantage of the strengths and weaknesses of the LLM. I think your last bullet point is most poignant, using Claude 3.5 I've been able to do tons of GUI and web programming, things I absolutely despise and refuse to do if I'm writing code by hand.
I sort of understand some of the vitriol that I see on HN but it is incredibly overblown. I don't really get a lot of the criticisms. LLMs aren't deterministic? Neither are humans. LLMs write bugs they can't fix? So do humans. LLMs are only good at being junior programmer copy paste machines? So are lots of humans.
My current project is training an LLM to do superoptimization and it's working exceedingly well so far. If you asked anyone on hacker news if that's a good idea, they'd probably say no.
The NPU is a Hexagon DSP with HVX, Hexagon Vector Extensions, and HMX, Hexagon Matrix Extensions. The core ISA and vector ISAs are pretty well documented and supported by upstream llvm, but AFAIK HMX is not publicly documented. Core ISA + HVX by itself can probably get you to 1/3 to 1/2 of so of the theoretical peak TOPS for Hexagon. It's been a bit since I've run code on device, but all the support code is in their SDK, and it's easy as pie to get it running on the simulator.
QCOM have said that up to ~13B LLMs will run at reasonable, and I think that's a pretty good peak for ~40TOPs and ~150GB/s bandwidth.
I've done some similar LLM compiler work, obviously not on Meta's scale, teaching an LLM to do optimization by feeding an encoder/decoder pairs of -O0 and -O3 code and even on my small scale I managed to get the LLM to spit out the correct optimization every once and a while.
I think there's a lot of value in LLM compilers to specifically be used for superoptimization where you can generate many possible optimizations, verify the correctness, and pick the most optimal one. I'm excited to see where y'all go with this.
Thank you for freeing me from one of my to-do projects. I wanted to do a similar autoencoder with optimisations. Did you write about it anywhere? I'd love to read the details.
There's code there to generate unoptimized / optimized pairs via C generators like yarpgen and csmith, then compile, train, inference, and disassemble the results
I think I recognize the author of this from /r/localllama, where plenty of other people are building similar frankenstein rigs. This post only mentions Intel setups, but AMD Epyc Milan and Rome based rigs are also very viable alternatives. They're a bit more expensive, but much better perf/watt, and the incremental price increase after factoring in a lot of GPUs is fairly slim. With 7 PCIE bifurcators on a motherboard such as the AsRock ROMED8-2T and a 14 risers, you can get up to 14 GPUs at PCIE 4.0x8.
A bit sad hobbyist have to resort to such measures to get tinkering, not to mention the initial capital needed. We're all slaves to Nvidia's VRAM monopoly until AMD or Intel steps in and release a competitive alternative with beefy vrams.
Beefy VRAM is a start, but Nvidia's real moat is CUDA. If PyTorch runs on AMD's ROCm, or the Intel equivalent, as well as it runs on CUDA, then we'll see some real competition here.
Chris Lattner's Mojo programming language may present an alternative here, but it's still closed source.
I assume that's the same Itek that made lenses for the Corona Spy Satellite program?
That's cool as heck. I am just a photographer, but I collect aerial lenses and own 2 Pacific Optical 18" f/3 lenses in 70mm format, serial numbers 6 and 13. I'll keep my eyes peeled for a surplus Itek 24" f/3.5 though :)
Yes, same one. I find de-classified articles and books now, and share them with former colleagues. All the designs I worked on were meant for space. And I have no idea if any were actually deployed.
As I noted in the “handle” post, k310 is the office number I had at Itek, as well as a nifty sonata by Mozart.
I later worked for Lockheed. More space optics and other cool stuff. I was in the R&D division.
I hope you can make good use of those lenses. I stuck with commercial optics (Nikon and Hasselblad). My optical hacking never exceeded making some adapters with a Unimat SL that I got. Surprise, computers got my attention, and my first “real” computer was an IMSAI that I built.
I read a lot. Kingslake, Smith and so on. A nice book, if you can find it is “Photographic Optics” by Neblette. It goes over all the classic designs, including all the familiar camera lenses from the 70’s and 80’s. Nowadays, computers do all the designing, and you can’t recognize “classic” designs like the tessar, Sonnar and so on, in them. But TBH, I finally got a new 105 macro lens with unrecognizable (to me) design from Nikon to augment the old 55mm macro that I started out with in 1970 or so. A simple double-gauss design.
Did you know what those “P” and “H” and other suffixes meant on Nikon lenses? They are the number of elements, in Optics Latin. H for hex, or 6 elements, P for penta or 5, and so on. The old lenses, without the “AI” aperture index gizmo, fit the new mirrorless body with the FTZ adapter. I gather that they were no-go on the DSLR’s. I skipped the DSLR generation entirely, since I used a Coolpix with 24-1000mm effective focal length for many, many years. Heck, it worked and got me great photos.
That's awesome! Thanks for taking the time to reply.
I am a young guy - just turned 30 last month - but I exclusively shoot and develop film and have my own mini darkroom. I enjoy the challenge and physicality of it all, plus I already spend enough time around computers and digital tech as a software engineer so it's nice to have some analog pursuits. An IMSAI is a bit before my time, but I have a soft spot for older CPUs like the similar z80 or 6502 from writing emulators.
For 35mm I mostly shoot on Canon FD, so while I recognize some of those classic design names I haven't shot on them. I haven't been able to shoot on the behemoth PO lenses as much as I'd like since they weigh nearly 60 pounds a piece and are a hassle to use, but I've jerry rigged a Graflex Crown Graphic to the back and can do a manual shutter with some ND filter and a quick on/off of the lens cap. They cover 4x5 or more at the distances I use them and are incredible lenses for portraiture.
My next projects are to mount both PO lenses together and put Graflex Speed Graphic 4x5 cameras behind each in order to make a massive binocular camera. I also have a Goerz 47" process lens that's got an image circle of nearly a meter, so my next shoot with that will be a 1:1 self portrait on xray film, since it's the only semi affordable option for such a massive exposure.
Great! You’re having fun. I got a few odd surplus lenses but since I started photography by taking flower and landscape photos, the “optics lab” part never took flight.
Itek made semiconductor lithography stuff way back when right? I did a stint at one of its competitors. I didn't know about the spy satellite stuff. Neat!
?
It's not controversial that the Norse made it to modern day Newfoundland.