Would like to give them some guidance on how to get AI to help prepare them for their interviews next year, will definitely take a look at your AGENTS.md approach.

What's your student feedback on it been like?

This is amazing and wish professors had done this back when I did CS in the late 1990s.

my feedback has been pretty good on the in-person quizzes, we just had our first set

Great example of AI decreasing labour efficiency (because you need to invest more in fraud prevention).

Was there something fundamentally different from those who used "AI" a "lot" vs those who didn't?

Did they mention the issue of hallucination and how they addressed it?

If a lecturer prepared slides with basically an x86 assembly to show how to loop, what is so bad about an AI regurgitating that and possibly even annotating it with the inner workings.

I feel like this is still underappreciated. Awesome meaningful diagrams with animations that I would take me days to make in a basic form can now be generated in under an hour with all the styling bells and whistles. It's amazing in practice because those things can deliver lots of value, but still weren't worth the effort before. Now you just tell the LLM to use anime.js and it will do a decent job.

  Of course I can't ensure they aren't just having AI do the projects, but I tell them that if they do that they are cheating themselves

IMO the natural fix is to expand your trusting, “this is for you” approach to the broader undergrad experience, but I can’t imagine how frustrating it is to be trying to adapt while admin & senior professors refuse to reconsider the race for a “””prestigious””” place in a meta-rat race…

For now, I guess I’d just recommend you try to think of ways to relax things and separate project completion from diligence/time management — in terms of vibes if not a 100% mark. Some unsolicited advice from a rando who thinks you’re doing great already :)

Millions of students prior to the last few years figured out how to manage conflicting class requirements.

Sure, and they also didn't have an omniscient entity capable of doing all of their work for them in a minute. The point of the GP comment, in my reading, is that the temptation is too great.

This is why I'm going to in-person written quizzes to differentiate between the students who know the material and those who are just using AI to get through it.

I do seven quizzes during the semester so each one is on relatively recent material and they aren't weighted too heavily. I do some spaced-repetition questions of important topics and give students a study sheet of what to know for the quiz. I hated the high-pressure midterms/finals of my undergrad, so I'm trying to remove that for them.

The pressure was what got me to do the necessary work. Auditing classes never worked for me.

> I do some spaced-repetition questions of important topics and give students a study sheet of what to know for the quiz.

Isn't that what the lectures and homework are for?

I do spaced repetition in lectures, my homeworks are typically programming problems and, as I said in OP, rely on the student committing to doing them w/o AI. So spaced repetition of the most important topics on quizzes seems reasonable. (It's an experiment this semester)

I find, as a parent, when I talk about it at the high school level I get very negative reactions from other parents. Specifically I want high schoolers to be skilled in the use of AI, and particular critical thinking skills around the tools, while simultaneously having skills assuming no AI. I don’t want the school to be blindly “anti AI” as I’m aware it will be a part of the economy our kids are brought into.

There are some head in the sands, very emotional attitudes about this stuff. (And obviously idiotically uncritical pro AI stances, but I doubt educators risk having those stances)

Our university is slowly stumbling towards "AI Literacy" being a skill we teach, but, frankly, most faculty here don't have the expertise and students often understand the tools better than teachers.

I think there will be a painful adjustment period, I am trying to make it as painless as possible for my students (and sharing my approach and experience with my department) but I am just a lowly instructor.

People need to learn to do research with LLMs, code with LLMs, how to evaluate artifacts created by AI. They need to learn how agents work at a high level, the limitations on context, that they hallucinate and become sycophantic. How they need guardrails and strict feedback mechanisms if let loose. AI Safety connecting to external systems etc etc.

You're right that few high school educators would have any sense of all that.

I do know people who would get egregiously wrong answers from misusing a calculator and insisted it couldn't be wrong.

Not to mention programming is a meta skill on top of “calculators”

Can you expound on both points in more details please, ideally with some examples?

In OP I showed an AGENTS.md file I give my students. I think this is using AI in a manner productive for intellectual development.

That said I agree with all your points too: some version of this argument will apply to most white collar jobs now. I just think this is less clear to the general population and it’s much more of a touchy emotional subject, in certain circles. Although I suppose there may be a point to be made about being more slightly cautious about introducing AI at the high school level, versus college.

That's true, but you can't use AI in coding effectively if you don't know how to code. The risk is that students will complete an undergraduate CS degree, become very proficient in using AI, but won't know how to write for loop on their own. Which means they'll be helpless to interpret AI's output or to jump in when the AI produces suboptimal results.

My take: learning to use AI is not hard. They can do that on their own. Learning programming is hard, and relying on AI will only make it harder.

Depends on what your definition of "hard" is - I routinely come across engineers who are frustrated that "AI" hallucinates. Humans can detect hallucinations and I have specific process to detect and address them. I wouldn't call those processes easy - I would say it's as hard as learning how to do integration by summing.

> but you can't use AI in coding effectively if you don't know how to code

Depends on the LLM. I have a fine-tuned version of Qwen3-Coder where if you ask it to show you to compare to strings in C/C++, it will but then it will also suggest you look at a version that takes unicode into account.

I have stumbled across very few software engineers who even know what unicode codepoints are and why legacy ASCII string comparison fails.

> but won't know how to write for loop on their own. Which means they'll be helpless to interpret AI's output or to jump in when the AI produces suboptimal results

That's a very large logical jump. If we went back 20 years, you might come across professors and practising engineers who were losing sleep that languages like C/C++ were abstracting the hardware so much that you could just write for loops and be helpless to understand how those for loops were causing needless CPU wait cycles by blocking the cache line.

My students don't seem to have a problem using AI: it's quite adequate to the task of completing their homework for them. I therefore don't feel a need to complete my buzzword bingo by promoting an "AI-first classroom." The concern is what they'll do when they find problems more challenging than their homework.

> I have stumbled across very few software engineers who even know what unicode codepoints are and why legacy ASCII string comparison fails.

You are proving my point. If the programmer doesn't know what Unicode is, then the AI's helpful suggestion is likely to be ignored. You need to know enough to be able to make sense of the AI beyond a superficial measure.

> That's a very large logical jump. If we went back 20 years, you might come across professors and practising engineers who were losing sleep that languages like C/C++ were abstracting the hardware so much that you could just write for loops and be helpless to understand how those for loops were causing needless CPU wait cycles by blocking the cache line.

We still teach that stuff. Being an engineer requires understand the whole machine. I'm not talking about mid-level marketroids who are excited that Claude can turn their Excel sheets into PowerPoints. I'm talking about actual engineers who take responsibility for their code. For every helpful suggestion that AI makes, it botches something else. When the AI gives up, where do you turn?

No, it's not.

Nothing around AI past the next few months to a year is clear right now.

It's very, very possible that within the next year or two, the bottom falls out of the market for mainstream/commercial LLM services, and then all the Copilot and Claude Code and similar services are going to dry up and blow away. Naturally, that doesn't mean that no one will be using LLMs for coding, given the number of people who have reported their productivity increasing—but it means there won't be a guarantee that, for instance, VS Code will have a first-party integrated solution for it, and that's a must-have for many larger coding shops.

None of that is certain, of course! That's the whole point: we don't know what's coming.

My point is, I can get somewhat-useful ai model running at slow-but-usable speed on a random desktop I had lying around since 2024. Barring nuclear war there’s just no way that AI won’t be at least _somewhat_ beneficial to the average dev. All the AI companies could vanish tomorrow and you’d still have a bunch of inference-as-a-service shops appearing in places where electricity is borderline free, like Straya when the sun is out.

Yes, you, a hobbyist, can make that work, and keep being useful for the foreseeable future. I don't doubt that.

But either a majority or large plurality of programmers work in some kind of large institution where they don't have full control over the tools they use. Some percentage of those will never even be allowed to use LLM coding tools, because they're not working in tech and their bosses are in the portion of the non-tech public that thinks "AI" is scary, rather than the portion that thinks it's magic. (Or, their bosses have actually done some research, and don't want to risk handing their internal code over to LLMs to train on—whether they're actually doing that now or not, the chances that they won't in future approach nil.)

And even those who might not be outright forbidden to use such tools for specific reasons like the above will never be able to get authorization to use them on their company workstations, because they're not approved tools, because they require a subscription the company won't pay for, because etc etc.

So saying that clearly coding with LLM assistance is the future and it would be irresponsible not to teach current CS students how to code like that is patently false. It is a possible future, but the volatility in the AI space right now is much, much too high to be able to predict just what the future will bring.

The promise of AI is a capitalist's dream, which is why it's being pushed so much. Do more with less investment. But the reality of AI coding is significantly more nuanced, and particularly more nuanced in spaces outside of the SRE/devops space. I highly doubt you could realistically use AI to code the majority of significant software products (like, say, an entire operating system). You might be able to use AI to add additional functionality you otherwise couldn't have, but that's not really what the capitalists desire.

Not to mention, the models have to be continually trained, otherwise the knowledge is going to be dead. Is AI as useful for Rust as it is for Python? Doubtful. What about the programming languages created 10-15 years from now? What about when everyone starts hoarding their information away from the prying eyes of AI scraper bots to keep competitive knowledge in-house? Both from a user perspective and a business perspective?

Lots of variability here that literally nobody has any idea how any of it's going to go.

The genie is out of the bottle, never going back

It's a fantasy to think it will "dry up" and go away

Some other guarantees over the next few years we can make based on history: AI will get batter, faster, and more efficient like everything else in CS

Local models are already good enough to handle some meaningful programming work, and they run very well on an expensive-but-not-unattainable PC. You could cheat your way through an undergrad CS curriculum with Qwen 80b, certainly, including most liberal-arts requirements.

The genie is not going back in the bottle no matter what happens, short of a nuclear war. There is no point even treating the possibility hypothetically.

Plenty of tech becomes exploitative (or more exploitative).

I don't know if you noticed but 80% of LLM improvements are actually procedural now: it's the software around them improving, not the core LLMs.

Plus LLMs have huge potential for being exploitative. 10x what Google Search could do for ads.

I personally think GSuite is much better today than it was a decade ago, but that is separate

The underlying hardware has improved, the network, the security, the provenance

Specific to LLMs

1. we have seen rapid improvements and there are a ton more you can see in the research that will be impacting the next round of model train/release cycle. Both algorithms and hardware are improving

2. Open weight models are within spitting distance of the frontier. Within 2 years, smaller and open models will be capable of what frontier is doing today. This has a huge democratization potential

I'd rather see the Ai as an opportunity to break the Oligarchy and the corporate hold over the people. I'm working hard to make it a reality (also working on atproto)

We can't fix social problems with technological solutions.

Every scalable solution takes us closer to Extremistan, which is inherently anti democratic.

Read the Black Swan by Taleb.

Democratization, the way I'm using it without all the bias, is simply most people having access to build with a tool or a technology. Would you also argue everyone having access to the printing press is a bad thing? The internet? Right to repair? Right to compute?

Why should we consider Ai access differently?

Show me actual studies that clearly demonstrate that not only does using an LLM code assistant help make code faster in the short term, it doesn't waste all that extra benefit by being that much harder to maintain in the long term.

Clearly AI is much faster and good enough to create new one-off bits of code.

Like I tend to create small helper scripts for all kinds of things both at work and home all the time. Typically these would take me 2-4 hours and aside from a few tweaks early on, they receive no maintenance as they just do some one simple thing.

Now with AI coding these take me just a few minutes, done.

But I believe this is the optimal productivity sweet spot for AI coding, as no maintenance is needed.

I've also been running a couple experiments vibe-coding larger apps over the span of months and while initial ramp-up is very fast, productivity starts to drop off after a few weeks as the code becomes more complex and ever more full of special case exceptions that a human wouldn't have done that way. So I spend more and more time correcting behavior and writing test cases to root out insanity in the code.

How will this go for code bases which need to continuously evolve and mature over many years and decades? I guess we'll see.

If AI just generates piles of unmaintainable code, this isn't going to be any worse than most of the professionally-written (by humans) code I've had to work with over my career. In my experience, readable and maintainable code is unfortunately rather uncommon.

There are a whole range of interesting questions here that it’s possible to have a nuanced discussion about, without falling into AI hype and while maintaining a skeptical attitude. But you have to do it from a place of curiosity rather than starting with hatred of the technology and wishing for it to be somehow proved useless and fade away. Because that’s not going to happen now, even if the current investment bubble pops.

If anything, I see this moment as one where we can unshackle ourselves from the oligarchs and corporate overlords. The two technologies are AI and ATProto, I work on both now to give sovereignty back to we the people

For me, modern AI appears to be controlled entirely by oligarchs and corporate overlords already. Some of them are the same who already shackled us. This time will not be different, in my opinion.

I like your optimism.

I think that Microsoft will not be willing to operate Copilot for free in perpetuity.

I think that there has not yet been any meaningful large-scale study showing that it improves performance overall, and there have been some studies showing that it does the opposite, despite individuals' feeling that it helps them.

I think that a lot of the hype around AI is that it is going to get better, and if it becomes prohibitively expensive for it to do that (ie, training), and there's no proof that it's helping, and keeping the subscriptions going is a constant money drain, and there's no more drumbeat of "everything must become AI immediately and forever", more and more institutions are going to start dropping it.

I think that if the only programmers who are using LLMs to aid their coding are hobbyists, independent contractors, or in small shops where they get to fully dictate their own setups, that's a small enough segment of the programming market that we can say it won't help students to learn that way, because they won't be allowed to code that way in a "real job".

In spite of obvious contradictory signals about quality, we embrace the magical thinking that these tools operate in a realm of ontology and logic. We disregard the null hypothesis, in which they are more mad-libbing plagiarism machines which we've deployed against our own minds. Put more tritely: We have met the Genie, and the Genie is Us. The LLM is just another wish fulfilled with calamitous second-order effects.

Though enjoyable as fiction, I can't really picture a Butlerian Jihad where humanity attempts some religious purge of AI methods. It's easier for me to imagine the opposite, where the majority purges the heretics who would question their saints of reduced effort.

So, I don't see LLMs going away unless you believe we're in some kind of Peak Compute transition, which is pretty catastrophic thinking. I.e. some kind of techno/industrial/societal collapse where the state of the art stops moving forward and instead retreats. I suppose someone could believe in that outcome, if they lean hard into the idea that the continued use of LLMs will incapacitate us?

Even if LLM/AI concepts plateau, I tend to think we'll somehow continue with hardware scaling. That means they will become commoditized and able to run locally on consumer-level equipment. In the long run, it won't require a financial bubble or dedicated powerplants to run, nor be limited to priests in high towers. It will be pervasive like wireless ear buds or microwave ovens, rather than an embodiment of capital investment.

The pragmatic way I see LLMs _not_ sticking around is where AI researchers figure out some better approach. Then, LLMs would simply be left behind as historical curiosities.

The last part...I'm not sure. The idea that we will be able to compute-scale our way out of practically anything is so much taken for granted these days that many people seem to have lost sight of the fact that we have genuinely hit diminishing returns—first in the general-purpose computing scaling (end of Moore's Law, etc), and more recently in the ability to scale LLMs. There is no longer a guarantee that we can improve the performance of training, at the very least, for the larger models by more than a few percent, no matter how much new tech we throw at it. At least until we hit another major breakthrough (either hardware or software), and by their very nature those cannot be counted on.

Even if we can squeeze out a few more percent—or a few more tens of percent—of optimizations on training and inference, to the best of my understanding, that's going to be orders of magnitude too little yet to allow for running the full-size major models on consumer-level equipment.

If you want to prove that there are not diminishing returns, you need to add at least one more data point in there.

You're really not showing any evidence that you even understand how this kind of math works, let alone that my statement is false.

1) AI companies make money on the tokens they sell through their APIs. At my company we run Claude Code by buying Claude Sonnet and Opus tokens from AWS Bedrock. AWS and Anthropic make money on those tokens. The unit economics are very good here; estimates are that Anthropic and OpenAI have a gross margin of 40% on selling tokens.

2) Claude Code subscriptions are probably subsidized somewhat on a per token basis, for strategic reasons (Anthropic wants to capture the market). Although even this is complicated, as the usage distribution is such that Anthropic is making money on some subscribers and then subsidizing the ultra-heavy-usage vibe coders who max out their subscriptions. If they lowered the cap, most people with subscriptions would still not max out and they could start making money, but they'd probably upset a lot of the loudest ultra-heavy-usage influencer-types.

3) The biggest cost AI companies have is training new models. That is the reason AI companies are not net profitable. But that's a completely separate set of questions from what inference costs, which is what matters here.

If you're using them for particular frameworks or libraries in that language, they'll need to know about those, too.

If training becomes uneconomical, new advances in any of these will no longer make it into the models, and their "help" will get worse and worse over time, especially in cutting-edge languages and technologies.

That's not going to happen. It's already too late to consider that a realistic possibility.

This is my exact experience as well and I find it frustrating.

If current technology is creating an issue for teachers - it's the teachers that need to pivot, not block current technology so they can continue what they are comfortable with.

Society typically cares about work getting done and not much about how it got done - for some reason, teachers are so deep into the weeds of the "how", that they seem to forget that if the way to mend roads since 1926 have been to learn how to measure out, mix and lay asphalt patches by hand, in 2026 when there are robots that do that perfectly every-time, they should be teaching humans to complement those robots or do something else entirely.

It's possible in the past, that learning how to use an abacus was a critical lesson but once calculators were invented, do we continue with two semesters of abacus? Do we allow calculators into the abacus course? Should the abacus course be scrapped? Will it be a net positive on society to replace the abacus course with something else?

"AI" is changing society fundamentally forever and education needs to change fundamentally with it. I am personally betting that humans in the future, outside extreme niches, are generalists and are augmented by specialist agents.

I had a discussion with a recruiter on Friday, and I said I guess the issue with AI vs human is, if you give a human developer who is new to your company tasks, the first few times you'll check their work carefully to make sure the quality is good. After a while you can trust they'll do a good job and be more relaxed. With AI, you can never be sure at any time. Of course a human can also misunderstand the task and hallucinate, but perhaps discussing the issue and the fix before they start coding can alleviate that. You can discuss with an AI as much as you want, but to me, not checking the output would be an insane move...

To return to the point, yeah, people will use AI anyway, so why not teach them about the risks. Also LLMs feel like Concorde: it'll get you to where you want to go very quickly, but at tremendous environmental cost (also it's very costly to the wallet, although the companies are now partially subsidizing your use with the hopes of getting you addicted)..

everything you learn about working using chatbots is completely obsoleted by ai five years from now

both are possible, but 2 is pretty much guaranteed if we get 1, so learning about chatting with opus is pretty much always less useful than learning derivatives by hand unless you're starting job applications in less than a few months

Competing with LLM software users, 'honest' students would seem strongly incentivized to use LLMs themeselves. Even if you don't grade on a curve, honest students will get worse grades which will look worse to graduate schools, grant and scholarship committees, etc., in addition to the strong emotional component that everyone feels seeing an A or C. You could give deserving 'honest' work an A but then all LLM users will get A's with ease. It seems like you need two scales, and how do you know who to put on which scale?

And how do students collaborate on group projects? Again, it seems you have two different tracks of education, and they can't really work together. Edit: How do class discussions play out with these two tracks?

Also, manually doing things that machines do much better has value but also takes valuable time from learning more advanced skills that machines can't handle, and from learning how to use the machines as tools. I can see learning manual statistics calculations, to understand them fundamentally, but at a certain point it's much better to learn R and use a stats package. Are the 'honest' students being shortchanged?

Thanks for taking the time for your students. Your students will thank you, too, but that will be years from now.

I once got "implement a BCD decoder" with about a 1"x4" space to do it.

I'm concerned about handwriting, which is a lost skill, and how hard that will be on the TAs who are grading the exams. I have stressed to students that they should write larger, slower and more carefully than normal. I have also given them examples of good answers: terse and to the point, using bulleted lists effectively, what good pseudo-code looks like, etc.

It is an experiment in progress: I have rediscovered the joys of printing & the logistics moving large amounts of paper again. The printer decided half way through one run to start folding papers slightly at the corner, which screwed up stapling.

I suppose this is why we are paid the big bucks.

Oh man, this reminds me of one test I had in uni, back in the days when all our tests were in class, pen & paper (what's old is new again?). We had this weird class that taught something like security programming in unix. Or something. Anyway, all I remember is the first two questions being about security/firewall stuff, and the third question was "what is a socket". So I really liked the first two questions, and over-answered for about a page each. Enough text to both run out of paper and out of time. So my answer to the 3rd question was "a file descriptor". I don't know if they laughed at my terseness or just figured since I overanswered on the previous questions I knew what that was, but whoever graded my paper gave me full points.

So how do you handle kids who can‘t write well? The same way we‘ve been handling them all along — have them get an assessment and determine exactly where they need support and what kind of support will be most helpful to that particular kid. AI might or might not be a part of that, but it‘s a huge mistake to assume that it has to be a part of that. People who assume that AI can just be thrown at disability support betray how little they actually know about disability support.

There's not such thing as "disabled people who can't write well", there's individuals with specific problems and needs.

Maybe there's jessica who lost her right hand and is learning to write with the left who gets extra time. Maybe there's joe who has some form of nerve issue and uses a specialized pen that helps cancel out tremors. Maybe sarah is blind and has an aide who writes it or is allowed to use a keyboard or or or...

Reasonable accommodations absolutely should be made for children that need them.

But also just because you’re a bad parent and think the rules don’t apply to you doesn’t mean your crappy kid gets to cheat.

Parents are the absolute worst snowflakes.

(◔_◔)