As an attorney, I feel like vetting AI output takes longer than just doing it from scratch, let alone versus just using a traditional form.
With AI, I have to read through everything, often explain why it's wrong, and then rewrite everything anyways. I mean, I get way more billables, but I think it's symptomatic of how AI loses its advantage of being quick and accessible to those who don't understand the subject matter.
Another attorney here. I understand your plight. But I can't believe law firms are sending out briefs and opinions without carefully checking all of the citations. I mean, even when Lexis or Westlaw identifies an (actual) case on point, you still have to check if the case has been overturned, whether it is truly on point, or if it can be distinuished from your case. So even if the cited case is not a halucination, someone would still have to read and analyze the cited case in the context of the present case.
Fact-checking and editing a mediocre piece of writing be way harder than writing from scratch. Proving that something isn’t true or can’t be substantiated is hard work, and so is arguing that a word choice is subtly inappropriate.
And making a ton of corrections to a document everyone was hoping was ready to go is never fun politically.
This is the realization I had too. We had a manager update a policy at our org. He just shit it out through AI. It had tons of mistakes, people who read it had questions. Not only did it have mistakes it was causing people to do things in a way that added a manual step when an automatic process existed. Then the engineer VP commented on it asking the original author what its about who then had to bring it back up to the attention of the manager who made the first change.
It wasted many people's time, probably an order of magnitude of time wasted (and money) than if the initial person put a modicum of effort into making it right in the first place. Instead they hand it off to their life partner claude and just assume its good enough.
It's to the point where I am feeling insulted when I get ai slop like this from people. If I am expected to perform at a high level then I expect that at the very minimum the slop throwers will proof read their slop.
I have experienced this several times lately when writing software with claude/codex. Sometimes vetting and steering the agent takes longer than it would have taken me if done manually. Sure you can just decide not to vet the output and go into full vibecode, but agents tend to do a lot of dumb things (such as not deleting unused private methods or having temporary variables that are not needed).
In my experience the most effective work pattern for me is using agents to perform research and feedback on high level design, then I write the code manually, then I ask the agent to review the code for potential bugs/issues and fix those. The agents have a much easier time making small changes once the design is 90% there without going fully off the rails and generating slop.
I am working on writing skills to make the agent better but it is a bit painstaking. For example I had to write this inside of a skill because sometimes the agent would just stub out methods and leave TODOs: “always fully complete the requested task before finishing edits unless input is needed”.
You can also feed the document or source file to another frontier-level model, ideally two others, and tell it to vet it aggressively. The goal is to goad the models into erring on the side of false positive findings rather than potentially missing true positives.
I find that if Gemini Pro agrees with Claude Opus 4.8 and GPT 5.5 on something, it's almost certainly correct at a level where I wouldn't be likely to catch any errors myself.
It's not really any different in programming. Like if you have a well structured code and want to do a clear refactoring across it and you know what to expect, it can speed things up. But if it's generating any significant (and relatively complex) new code, you have to go through the whole thing manually again and then you find out you have to fix way to many things and get bogged down in different paths the AI didn't do correctly.
Of course, it's pretty much impossible to hear a dissenting point of view today and everyone is going crazy on these drugs. I might be hilariously wrong but I think this is the best time to start a software company.
I think its the perfect time to be contrarian - think about it. If youre wrong - So what? The world will have changed for everyone in the field. If you are right? You stand to be positioned to win big financially whilst everyone elses brain is rotting away.
This is true in some states, like Texas; but not in Delaware where Disney is incorporated and where directors and officers owe fiduciary duties of care and loyalty to both the corporation and the shareholders.
(Not legal advice. I'm not licensed in either state.)
You are correct to note that there are two fiduciary duties: the duty of care, and the duty of loyalty. However, you are incorrect to imply I stated the law incorrectly.
The duty of care is otherwise known as the duty to be informed. And the duty of loyalty is otherwise known as the duty not to usurp
corporate opportunities. I stated the law in Delaware, which is consistent with the law on the rest of the United States on these points.
You and I simply use two different sets of words to describe the only two fiduciary duties of an officer.
This is an underrated distinction. Sadly, the line is so much more blurred now than even when I was a kid in the 90s.
There are so many businesses now which exist mainly to cheat you, operating at the very edge of what’s technically legal, and relying on their customers not really understanding the full terms of the deals they’re agreeing to. It’s sickening.
- Seniors are sold various quackish financial products like annuities which are a terrible deal for them.
- Timeshares, which nearly never work out in the favor of the consumer (and whose value collapses 50-80% instantly if you look at what they go for on the resale market)
- Prepaid card products that cost a bunch of money to load and then incur monthly fees too (exploiting those who have for whatever reason got blacklisted from banking)
- Every financial product that has a 25%+ interest rate, actually, which isn't limited to those with bad credit. Even if you have an 899 credit score, if you walk into Nordstrom and get their credit card, you will have a close to 30% rate on that. This whole business model is obviously built on tricking people into spending money they don't have and carrying a balance.
- Salesmen hawking solar panels that come to my front door and promise me all kinds of savings. Note: Probably only half these are scams! Just have to figure out which half.
- Health insurers, pretty much across the board. They do things like declare the most dominant ambulance service in San Francisco, the SFFD, "out of network", so the SFFD then sends you a bill for $1000 if you had to use an ambulance. The neat lifehack by the insurer is that most people will just curse, cry, maybe go into debt, and pay it. Only like 10-20% of patients will file a complaint with the insurer's state regulator, and those can just be quickly paid. Result: Savings of 80-90% for health insurance company! (If this one sounds oddly specific, you can guess why.)
Every payday loan company, the "we buy houses for cash" companies, rent-to-own companies, title loan companies, the entire buy-now-pay-later ecosystem, the timeshare industry.
Seriously dotancohen, get your people under control.
With AI, I have to read through everything, often explain why it's wrong, and then rewrite everything anyways. I mean, I get way more billables, but I think it's symptomatic of how AI loses its advantage of being quick and accessible to those who don't understand the subject matter.
reply