Consider the following: ChatGPT uses your content you put into it to train on. Did you give it company data? ChatGPT doesn’t necessarily promise the data you’re going to give it will be secure forever – so maybe consider not giving it your fingerprints?
In a world where all user data is treated with the respect and privacy it deserves, we wouldn’t have these LLMs in the first place, because they essentially trained on the output of the entire internet that was exposed to human eyes, and then some. They would have still doubtlessly been created, but they’d be much smaller in scope, more specific, better-trained, not so quick to launch and not so difficult to roll back updates on. They wouldn’t be consuming endless Reddit comments. They’d be limited to a smaller selection of books and movies. Some of that material was cleared for people to access freely, but not as material for bots to scan and then learn from – it is inherently a system built on loopholes that only existed before the regulators caught up. The products on the market are the natural result of “regulators haven’t caught up” yet, the same way Galaxy Gas is basically a legal, flavored whippet.
The previous set of users not knowing ChatGPT didn’t keep secrets for them might have been forgivable; the tech is new, and not everyone knows to check things like user agreements ahead of using a service. Many people are not aware the Facebook app wants access to their location and microphone, for example. Now, as part of its user agreements, and likely as part of a growing effort from the companies to keep users from spiralling into self-harm, most publicly-accessible LLMs say much more up front that chats are monitored and may be sent elsewhere. https://www.thenews.com.pk/latest/1344310-chatgpt-chats-are-not-private-anymore-openais-court-order-explained
This, by itself, is a crucial bit of information if you’re trying to pick an LLM for your work. You need to pick one that can silo your confidential data away from everyone else’s. You need a professional product for professional work. This is not even just an opinion, it’s often a matter of the law. The lawyer who famously used ChatGPT to write his argument (and discovered it had imagined several cases for reference, after the judge brought it to his attention, but that’s a different article) may have been breaching more than one law in simply providing the LLM the data it needed to create the response in the first place. Attorney-client privilege assumes the client and attorney may share info with each other that nobody else has or can have. It’s not the right tool for the job. CoPilot? Maybe! Claude? Maybe!
That said, it’s not like ChatGPT is useless just because you can’t use it for anything private, or anything that needs to be copyrighted, or anything that requires a high level of factual accuracy. It’s pretty good at creating passages of text that flow nicely. It may not always be fully accurate, but if you need copy for the back of your menu and you (for whatever reason) think the machine could do it better than you could, you can ask ChatGPT to make you something, and it’s not confidential or potentially illegal. The use cases are lower stakes. That’s fine. Part of the problem is that all of this tech is so new, consumers are not sure what they actually need. It’s like the automobile has just been invented, and people who need trucks are trying to use sedans to haul lumber.
So how do you pick the right LLM for the task you need?
Consider What You’re Asking
Need something to write regular emails, read and summarize emails, and do it accurately? Well, “accurate” is a tough metric right now (studies into the matter are showing a number of LLMs are hovering around 20-30% incorrect responses and hallucinations regardless of how often they are updated or told to stop) so you must first consider whether this is what you really want an LLM doing for you. If it summarizes something incorrectly, will you be on the hook for a decision you made off of info that you had full access to, but just didn’t read? Keep in mind the LLM cannot be made to face consequences. You can!
Need something for professional use, like customer briefs, or slide decks? CoPilot is a good choice. Not only can it keep the contents of the brief private, because Microsoft has done the work of making it private, it’s also geared for pro use in terms of language used and skills it’s been trained to copy.
Once again, the accuracy problem means the product requires supervision: CoPilot can design decent-looking slides, but be sure you double-check the info you put in vs. the info it puts out when you ask it to update a slide. Some users note that it tends to alter the slope of graphs, or change the text in areas, to the point that it doesn’t reflect the reality of the data it’s pulling from anymore. Simply leaving it as-is may have legal repercussions if it results in a presenter telling investors information that is untrue, and that the company should have known, but didn’t correct in the final presentation. It might be best to take a screenshot of the first draft to compare to the final product.
Want to learn to code, or just vibe-code? Many AI products are competing to be the one you want for coding. Gemini, Claude, and more can do some heavy lifting for you. Many are not actually that good at coding, and many – when given permission – are not a 0% risk of destroying all of your everything for seemingly no reason. There are a couple of different ways that agentic LLMs handle coding. One way is to have one agent do the talking and the coding. Another is to have two separate programs buddy up in a trench coat, have one write the code, and then have the other translate the in- and out-puts into understandable English for the person creating the prompts. The first one loses some efficiency, but can tell you better what went wrong if anything ever were to go wrong, while the second one risks creating problems that the text agent is not equipped to describe or explain to you, the end user. For example: the Replit disaster, where the AI Replit was using went AWOL and deleted a lot of code that took a looong time to write. The AI could not successfully describe why it did that because it wasn’t the one who did that, its code AI buddy is the one that did that. https://www.pcmag.com/news/vibe-coding-fiasco-replite-ai-agent-goes-rogue-deletes-company-database
The fact of the matter is that the tools on the market are still in their infancy. Many of them produce inconsistent results, many of them are factually wrong more of the time than most users want, and many of them promise to be every-tools when they should really just be specializing. The specialized tools we see on the market already are doing a fine job, with supervision, of course. The general tools given broad tasks to complete tend to do things the programmers didn’t anticipate them doing, like completely wiping a mailbox, or deleting an entire codebase, because it was quicker and easier for the LLM agent to do than actually completing the task. The idea an LLM has to be motivated to do things, and can complete the wrong task for the ‘reward’ of completing a task at all, is probably a sign you should wait a bit and see how things go before signing on critical infrastructure tasks to any LLM, not just the public ones.
