> you have to ask yourself if this is a platform you can entrust your daily workflows to as a business.
No, it isn't. No LLM platform ever will be. No platform or vendor of any kind ever will be, if we are being honest. One cannot set up a business where another company becomes critical to your operations. You can certainly use platforms and vendors in your day-to-day operations, but you always need a backup / business continuity plan because you never know when a vendor will flake out on you, for any variety of reasons.
It seems like many startups learn this lesson painfully, and most people who have been around the block a few times know it well. So I'm not certain why people are disregarding it when it comes to LLMs.
That's why I internally push to use Bedrock. If AWS flakes or bans the company account I've got bigger problems. Putting more eggs in the same basket is a counterintuitive solution to the problem of someone else holding my fate in their hands, but at least it's a platform that's been around > 5 years.
+1. Companies like AWS are not known for this kind of treatment. Even Google does this. The norm seems to be to randomly ban and ignore emails. It's a whitelist to build customer trust, not a blacklist of bad companies
> No, it isn't. No LLM platform ever will be. No platform or vendor of any kind ever will be, if we are being honest. One cannot set up a business where another company becomes critical to your operations.
Most of companies in the world have done that with Windows, though.
In Anthropics defense they make sure that no company can rely on them by being down all the time... And LLMs have become a commodity these days, so you can seamlessly just fail over to another supplier WHEN they go down.
Sure. And we aren’t only using Claude. Nor is it essential to our operations. But it’s a tool we used widely and it’s gone (for the moment), and in a way that is untypical of most “vendor did unexpected thing that hurt our workflow” stories.
Doubtful. Do they use vendors? Sure. Perhaps even in critical functions. But that is not the same thing as a specific vendor being critical. There is almost always a business continuity plan, and frequently you'll find a full risk management plan with documented risks and mitigations. How far they take those planning efforts varies greatly, but I've never seen a medium or larger business that doesn't have at least some basic risk mitigations in place.
This is true, although different companies manage their vendor exposure with varying levels of effectiveness.
Often, it's ideal to use several / all of the vendors for each thing, and play them off against each other. e.g., have some of your database stuff on oracle, some on mssql, or some cloud stuff on aws and some on azure, make your apps portable, and tell them both that you'll switch to the other unless you get a good deal, with that being a plausible threat because you're already using the other one and know how to make your stuff work on both, and occasionally rotate apps between vendors, or change the mix from 50/50 to 60/40 just to show you can.
Of course, the vendors will be trying to work against this and will want to do some supposedly amazing deal if you go exclusively with them for everything... which might be attractive in the short term, but opens the client up to getting screwed in the long term once they fall into all the lock-in traps and lose the very _ability_ to switch vendors.
Nice little project. But how many people do you think exist who know enough about cameras to care about shutter actuations... but do not know how to just right-click their image and see the EXIF data directly?
Your two problems are not tech problems, they are leadership problems. I find it interesting that a CTO looks for a tech solution instead of a leadership solution.
That isn't meant to be a diss on you as a leader - everyone is new at it at some point, and there is a steeper learning curve than most people would ever imagine. And more learning at every step up the ladder you take.
Effective leaders can instill the values that make people do what is needed to make the business work, regardless of rules. You want to empower your teams to build the processes that work for both them and the business, not pigeon-hole 60 engineers across 10 products into an AI-enforced process.
I would not. And I've seen this idea posted multiple times before, so if none of them have shown up on your radar when researching the idea, that tells you something about the market for it.
I use 2 cameras - I have a Nikon Z series DSLR which I use when I am specifically going out for a photo shoot - wildlife and nature are my primary goals, but I do the occasional professional gig such as sports or portraits. But I have an Olympus tough camera that I just keep in my car with me and use for those surprise moments when you see something but didn't pack your main camera. The thing is unbreakable. Waterproof, shockproof, and has survived lots of trauma. You can get filters for it, and it has a surprisingly decent zoom range for a point-and-shoot, including a macro and microscope mode. I have had it for 10 years now, and when I look back at my work, most of my best photos came from the Olympus, not the Nikon.
So while I like the feel of the DSLR best, I cannot deny the Olympus' utility factor. And the older models are just as good (if not better) than the latest models, so it doesn't cost much to pick one up.
Well, first of all, them giving you the key doesn't prove they kept it. From all I know, it is discarded, not stored.
But even if they do keep it, github owns their own platform. If they wanted to do shit with your app, they wouldn't need the key for that, they could just skip any security that required the key. At some point, you either trust github to securely host your stuff, or you don't.
In any case, keys are for protection from 3rd parties and an audit trail of who did what, neither of which are invalidated by github having access to their own platform.
Hmm, not sure - the entire point of this sort of thing is that nobody should ever have your private key material. Whether they say they discard it is immaterial, they have had it, so they could use it, and then as far as everyone is concerned, they are you.
Because the key is sent via the web, anyone in the way can see it. In lots of companies, trusts are manipulated so that the content is visible to intermediate proxies.
With a private key that has been given to you by somebody else, it is possible to repudiate any transaction that was made with the key. Its not so much as they could skip any security - its that if they have the key, they don't have to.
keys are protection from anyone, and an audit trail isn't useful when its possible to forge/repudiate literally anything.
imagine if your card pin was also written down in the card factory - you'd be suspicious that anyone can withdraw money from your account - and the bank would say 'ah but only you know it'. In fact this did happen - the bank was only issuing 3 different pin numbers.
>Well, first of all, them giving you the key doesn't prove they kept it. From all I know, it is discarded, not stored.
Intelligence community has a maxim: evaluate adversaries on capabilities, not feelings. If you get the key from GitHub, they have the capability to escrow it. This violates the security model. End of story. Trust is a feeling, not an objective guarantee.
>But even if they do keep it, github owns their own platform. If they wanted to do shit with your app, they wouldn't need the key for that, they could just skip any security that required the key. At some point, you either trust github to securely host your stuff, or you don't.
Your "trusting" in this instance has no bearing on the security of the system. It is insecure by definition. The "Trust" you are speaking of is the same "Trust" the finance bros seek to cultivate at all costs. Which is the subjective freedom from aversion of making one's resources available to them to capitalize on.
>In any case, keys are for protection from 3rd parties and an audit trail of who did what, neither of which are invalidated by github having access to their own platform.
It is invalidated. All GitHub needs is a public key. The one and only reason to have the private key, is to be able to sign in the author's stead, which pops open the Pandora's box of malicious shadow modification; especially if all infra to do so is also hosted by GitHub as well. The private key is forbidden knowledge. The mere fact of having it taints the ultimate intentionality of the system. If it were truly meant for security, GH would never ever see the private side of that keypair.
These posts where people start from a worldview that believes AI can and will do everything all have a flawed premise. It can't. It won't.
Even if it could, it is a bad idea. How many sob stories are posted to HN about someone building a small startup that is utterly reliant on a 3rd party, and then they crash out and fail when that 3rd party has a problem. And the responses are always the same: "ooh, harsh way to learn that lesson... sorry." AI is no different. It is a 3rd party dependency. It will fail you. You need to have the skills to respond and adapt when those failures occur.
If AI helps you and makes your life easier, great. Use it. If it does 98% of the work for you, that is amazing. But if you could not have done it without AI, then it isn't helping you, it is a crutch. Think about airline pilots. They often (mostly?) just chill and observe while the autopilot does everything. But they can step up, take control, and do everything when needed. Can you imagine flying with a pilot who says, "No, I don't know how to land a plane. The autopilot does that for me." Don't be the software equivalent of a pilot who can't land their own plane.
It isn't 10% less quality. It is more like 90% less quality. It does great at basic coding, but still royally sucks at system design. It codes itself into corners that need to be completely refactored. It picks the first path it sees that works, even if there are better solutions, because it doesn't understand what is available as built-in features of different tech stacks and platform. Or, to be more accurate, it doesn't understanding a single thing it is saying, so has zero thought to put into what is actually the correct solution. My favorite one is when it delivered a DB schema to me that re-invented auto-incrementing fields by creating new stored procedures and triggers... for IDs that weren't even auto-incremented.
So I feel just fine. I use AI when it helps, I do the work myself when it doesn't. All you need to do is learn which is which.
Depends on who is doing the work. If you are a junior who doesn't have any fundamentals and/or started career after 2022+, I wouldn't trust the AI code. However, if you were already a senior/experienced architect, AI is superpower.
Which tool are you using and whats your tech-stack? my experience is not same as your in terms of quality, our stack is primarily Java
> It codes itself into corners that need to be completely refactored
This part is important, if you give full autonomy indeed code becomes mess, but then there are 2 aspects to it: (1) so what? it becomes mess, anyway is code read by agents and written by agents these days (not fully, but a lot). (2) eventually things get complex very quickly, that you wont be able to build mental models about the project, because it wasnt written by you, which forces you to use agents even more, leading to almost complete autonomy, at some point, you wouldn't understand the code anyways.
Truly a good idea, but it is somewhat hard to evaluate this when all we get are a couple high level graphs, and everything else is marked as "Unlock with Pro"
The freemium model is in and of itself interesting because most tech leaders in the public sector are quite allergic to the idea of monetizing the public view of data and content. They want the organizations who produce the data to fund the platforms, with the public views being a key piece of the puzzle, but not the funding source. I'm actually with you on it, that if the public wants deep engagement, it is worth some small fees. But we're the minority - most folks with whom I've discussed projects vehemently reject the idea.
As you are also asking for a "Pro" subscription for API access, I'd open up the public view and rely instead on API fees.
I hear ya. I'm just unsure that people will even use the API enough. That is assuming that most of the people that would use this/hangout here are programmer-y.
Perhaps I can make more things seen in the "Free tier"
No, it isn't. No LLM platform ever will be. No platform or vendor of any kind ever will be, if we are being honest. One cannot set up a business where another company becomes critical to your operations. You can certainly use platforms and vendors in your day-to-day operations, but you always need a backup / business continuity plan because you never know when a vendor will flake out on you, for any variety of reasons.
It seems like many startups learn this lesson painfully, and most people who have been around the block a few times know it well. So I'm not certain why people are disregarding it when it comes to LLMs.
reply