He may or may not be right here, but captain steeeve has a bad track record of baseless claims and/or bad speculation and is not a good reference due to his past failings.

I would not be referencing him as a good source.

I don't know the specific items that were called out here but I can tell you, as a flight instructor, that the failure to capture the glideslope was already enough to require a go-around. The late flap settings were too. So was the high airspeed. At no point did they have a stabilized approach. There were multiple unequivocal mandatory go-around conditions before they got to the runway.

Yes, skills as a "portable power" isn't really the use case for me unless it's entirely generic and even then sparingly.

I've mostly followed what anthropic suggests, which is putting less into context and more into skills, to keep the "how" out of context until it is needed to reduce context bloat.

Skills have some instructions but are primarily informed repo specific instructions and keep their context away from the rest of the repo to keep things sanitised for me.

I've found it to be useful in that context.

> Skills have some instructions but are primarily informed repo specific instructions and keep their context away from the rest of the repo to keep things sanitised for me.

Skills and agents in the Claude world can also be extended and evolved over time, as they are committed "code".

For example, we have an agent which can take a statement or a support ticket and identifies the services, tenants and infrastructure components likely meant in the ticket or request. Similar to a skill, Claude can invoke this on demand in a conversation.

This started very simple, but various people spent time tuning it over the last 4-6 months. They have "taught" it to pick up on jargon from different departments, writing style of different departments, how they think about their systems.

With all of that tuning over time it has become quite "clever" in identifying the mentioned systems and - if requested - the train of thought leading to this conclusion.

Similar things are happening with skills for various task, be it Ansible integration tests, upgrade chores and so on. The first version can be fairly underwhelming, but continuously improving it after each usage can make them very powerful.

When I hear this I do wonder to myself how they're using AI.

For me, as long as I'm properly RPI looping it and not blindly pressing "yes" then it will nearly always reach the solution, usually a fair bit quicker, because it effectively becomes an ideation machine that can keep more thoughts and knowledge in it than my brain can.

However, I'm using it through giving it the context, it has access to the repos, access to the sytems, I'm telling it where the logs are, I have docs to show it what each part of the system is for.

Along with that, it does depend on my own instincts/knowledge for me to read its response and for me to say "no, you've over-thought/engineered this and this is actually the better solution", but its very are nowadays for me that it can't find the 1 liner, as long as I've fed in the right context, mostly pre-done because I've spent a bit of time building out the context tree for the repos/systems in a way that allows it to know what it needs to know.

I'm not doubting that people spin their wheels and couldn't find the one liner, but if its as simple as you say, that does seem like people who aren't great at LLMs along with a lack of instinct/experience.

I guess in essence, I think you can use LLMs in an old school troubleshooting way, and I find it still speeds that up the majority of the time. Its basically how I use it most of the time. And like old-school troubleshooting, if you build out the LLMs context over time, it also grows in capability, as long as its being used as a tool and not blindly trusted.

Should I not assume that most senior developers who are serious about LLMs do this?

In my experience at a large bank with unlimited AI, my spend is in the top 5% and I'm leveraging AI just like you.

I was in a meeting with someone who had a bug in an application that I don't own myself on Friday so I told claude,

"I need you to find this bug the user is experiencing, find out if it's user error or a bug, let the user know and tell the developer what needs to be fixed if needed"

15 minutes later the developer ask me if I want it fixed today or Tuesday.

That user could have done the same thing as me, had access to all the same systems and tools as I have, and also received the same AI training I had. The difference is that some users are just not, for lack of better words, AI native.

> "I need you to find this bug the user is experiencing, find out if it's user error or a bug, let the user know and tell the developer what needs to be fixed if needed" 15 minutes later the developer ask me if I want it fixed today or Tuesday.

Did you find out if it actually fixed the issue? I was on the other end of this last week. Reported an issue to another team, they asked Claude to diagnose, and when they gave me the response back I pushed back cause it didn’t make sense given the behavior we were seeing. Turns out Claude had hallucinated reading a log (it said it did but it didn’t). Sent it down a hole rabbit hole from there.

I’ve had the one shots you describe and they’re great, but they’re the happy path and require almost 0 skill to find. I’ve yet to see a case where developing an expert level knowledge of your domain isn’t the best way to get good at LLMs. Knowledge of how to interface with these tools is helpful but changes (and depreciates) rapidly as the models get smarter and/or other people commoditize it. Deep subject knowledge is still the best way to get a lot out of these tools. This applies to development and other areas I have a deep knowledge of.

“LLM training” is kinda snake oil. Learn your craft deeply and you’ll be able to catch up on LLM training in a few days, but you’ll be light years ahead of a person who’s only expertise is different ways to prompt.

To be clear, that one shot prompt was supported by nearly a years worth of work around developing an enviroment that allows me that level of effortless automation.

The issues was really simple, I knew the application, I knew what the bug was and already had a workaround in my head. I simply stopped at going down that rabbit hole and told claude to deal.

It identified the issue quickly, found the workaround and made the developer aware. All those things I would have done I just watched it do for me, plus, the app is vibe coded anyways, I just need it fixed and regardless of the AI path it was faster shooting out the bug report. Which, was simply poor processing of a spreadsheet.

> The issues was really simple, I knew the application, I knew what the bug was and already had a workaround in my head.

If that's the case, Claude isn't gaining you anything. The hard (and time consuming) part of programming is finding the correct solution, not typing the characters into an editor. When I know the code and have a good idea of the solution, I can write the code to fix it in mere minutes.

If only it was that easy, we are gated behind controls and processes that consume a significant amount of time between finding the bug, and a production push. AI quite literally automates the stuff I don't want to do, like create a fill out a jira ticket in the exact way that team wants it.

I don't have time for that, and neither do the teams that consume these request. As time has gone on me and the other teams I work with are no longer touching code, the AI will follow processes and procedures that pre date AI just fine with some human reviewing and approving.

AI has turned what was weeks at minimum to get a bug fix out, to just doing it all with with some human gating. It's not been a smooth process but watching it play out in real time it's getting faster, more efficient, and less prone to problems.

And that the crux off the situation, patching and securing configurations need to be done much faster than today, project glasswing has shown us that AI can scale to identify and solve those problems faster than a human.

Often though working through a bug teaches the developer something about the relevant abstraction. That knowledge can get lost in this new process.

While you're right, and I'm constantly thinking/worrying about my own processes giving me AI apathy, you can use AI in a way that develops your understanding of a bug. It really depends on your workflow.

When I use AI to debug a problem, I'm constantly questioning the specifics because I want to learn it, and often I'm better than the AI at quickly recognising patterns that point to a specific issue, where the AI will spin for some time trying to work out where to start - even if it has full context.

So I guess I'm half with you, but an LLM used right can still teach you effectively.

Yes. I agree with that as well. If you can intuit that it's (for example) an off-by-one error somewhere in your logic, asking an LLM to pinpoint that issue can save you tons of time that is better spent thinking about your abstractions. Also, I tend to ask high-level questions to the frontier models about best practices with common abstractions. But "the screen flickers here, please fix" is a bad pattern, IMO. Understanding why the screen is flickering is likely to educate you about where your abstractions are leaking.

Claude will happily add a null guard without checking why the null is there in the first place...

I find this story awkward. When Claude "..tell the developer what needs to be fixed" was it impersonating you?

Regardless, why not help the developer use the tools effectively instead?

I explained in another thread, but yeah, it acts on my behalf using my access to take actions with my approval. It has knowledge of all my work and leverages various data sources, (Jira, Confluence various mcp's) to gather information and take actions. If I ever have to correct it, or if the actions it takes are going down the wrong path I make sure and understand why and correct that.

It just got really good over time, especially as new processes get brought it and things change. I can't keep up with that like that AI can, just have it review and go down the path of ensuring that it fit's in my workflow as efficiently as possible and move on.

As far as the developer? I gave him ideas but he's on another team and I got my own guys to feed.

> RPI loop

What's that?

Research Plan Implement.

At the risk of you already knowing this or similar - in essence for a slice of a piece of work, you get the LLM to research the information required into (usually) a research file, which you go back and forth with in-line or via conversation until you're happy.

You then build the plan for the piece of work, again going back and forth with the LLM until you agree with the plan, with the key here being you have a specific set of concrete steps with very limited vagueness. Again this is quite often a plan document.

You then implement the plan with your LLM, following the concrete steps and monitor/evaluate the result.

For me its the 101 mandatory first way to use LLM that turns it from guesswork to genuinely useful. (that or some other similar method, SDD or whatever)

Because without it or something similar you're fully dependent on AI's weaknesses not coming to the fore (context/scoping failures that are silent unless you demand visibility up front and ruthlessly chop/clarify), and you often ascribe bad AI slop as model limitations instead of context/scoping failures.

Ah, that makes sense. I'm doing the same thing but I knew it as Spec Driven Development (SDD, I guess).

I have no idea why you're being downvoted. Your comment makes a lot of sense and I've upvoted it...

You're hearing the echo and ascribing it to the noise.

A constant push and pull between leaders and the led is the fundamental. Criticism in both directions is necessary and healthy, and at different times both correct and misplaced.

There have been great leaders and awful ones. Acting as if all leaders are proven and deserving is as asinine as naming all CEOs as useless.

Right now we are at a point where CEOs are biased towards being quite anti worker and too easily rewarded. It's not surprising the cliches against them come out.

Maybe you simply don’t hear about great CEOs, so there is a bias

I would argue the bias is for ceos (of public companies) to act a certain way due to inherent incentives the way the world is set up, which biases their personality. But I won't go into that old argument again.

But it has the right creative vibe which is the point really. Apple has a general creative aesthetic which plays well with artsy games like mixtape.

It's a very deliberate choice when if it doesn't make sense to gamers.

Whether or not you don't like the game, it's a weird misrepresentation to say it's made by Megan ellison.

She's been the co-founder and major backer of Annapurna for a decade.

She's not a developer, she's a publisher, and this game is one of the tons of games Annapurna has put out over a decade.

I get that Annapurna is a lightning rod of politics due to ellison and is increasingly under scrutiny for many debates but they are a mainstay of the industry, hardly a blow in, and like it or not, a lot of the games industry and media really like these kinda of artistic pieces, with this being yet another instance of the age old walking simulator debate.

It should be judged on its own merits. Not it's Annapurna link and it's not a shock to see it come up in an artsy Apple Mac hipster circle.

It's not objectively terrible. It's subjective - entirely.

It is a 3 hour long game with 4k or so reviews.

Life is strange original has 70K reviews and is 14 hours long.

The remaster has 4000+ reviews and again, is 14 hours long. These games are also all time favourites that fans do replay from time to time. If has more replayability than mixtape, despite it being fairly linear.

A 3 hour one time only game does not retain players on the charts like a game that takes multiple play sessions to finish. Or a game like "tennis elbow" which has far lower max concurrent (like, nowhere near the concurrent max mixtape has) but is a sim game that if it's your jam you will repeatedly come back to.

Its very simple - it's not a game for you, and you have to understand that these types of games absolutely have their niche fandoms who love this kind of stuff.

There is no conspiracy here, artsy titles like this have always, across all mediums, got a disproportional critical appreciation vs more traditional titles.

It's just the same old story. Yes people do like these games. No they are not the majority. Yes you can just ignore it, no it's not a conspiracy.

I always include sprint points or at least some metrics to track, because even in areas that don't care about it, there is some chance a new CTO/manager/buyout or whatever jumps in. Day 1 asks for metrics and then retroactively judges people's output on it.

I've annoyed my team previously in demanding jiras and sprint points (and helping build it out as much as I could to not take too much of their time) because I could feel the turn happening to metrics.

Sure enough the only person in my team let go was because he outright refused to fill in his JIRAs and I could not get through to him the difference in what is logical and what plays well to management when they lose their minds.

Bit of a tangent, but even in the work you're describing, I treat sprint points and JIRAs as future CYA, not necessarily a work sheet.

Doesn't that just effectively make you that person? The alternative is surely simply saying we haven't historically tracked that, should someone join and want it.

Not quite sure what you mean by that, I'm not the one making decisions of who to cut. That's one to many levels above me.

Your stated argument would make sense if you're dealing with reasonable people. Which is sometimes.

But I'm talking specifically when shit hits the fan and people levels above me put down a mandate across many divisions.

As it is if they're making cuts they'll grasp whatever metrics they can and cut. If they don't have metrics that gives less protection, not more.

Take "as you can see I have X number of staff fully allocated and up to their eyeballs in work" vs "I don't have that metrics but trust me". That's part of my job or responsibility depending on my seniority to justify my team and keep them employed and not stressed by cuts and workload.

Sorry, I was referring to your first paragraph, about someone new coming along and caring about metrics retroactively not previously cared about.

My point being that by proposing the solution 'so we should care about them, in case that happens' you effectively are (or are equivalent to) that person, you're making it happen sooner (and for sure).

In my experience, when they are in a cutting mood, they often decide first how many people to cut, then go through teams based on their workload.

So your team not having metrics, they are more likely to go "wow that team is pretty empty of work, cut half of them (exaggeration of course) rather than "well they don't have any metrics so Ill take them at their word or wait and see".

Cynical maybe, but my best managers who've kept a team together and protected tended to be really effective at selling the work they're doing and how long it'll take and how much workload each team member has, and showing how busy the team is in a way thats impactful.

The alternative is that they lay off everyone because the entire team doesn't have metrics for the work they've done. The kind of person to single someone out who hasn't been tracking their work may not be likely to just up and forgive the entire team if they've all done it that way.

I've seen it in action a few times specifically in government.

It's extremely difficult in my country to fire people in government. Managers want them out of their team but have no way to shift them. They are taking up one FTE in a key area but the person is just swinging the hammock doing just enough to keep from any proceedings.

Finance won't give them more headcount.

So they promote them to another useless department instead. Done.

I know it's been done because I have been to drinks with the senior managers who did this a decade ago when I asked what the deal was with XYZ manager that had been there for years and was truly one of the worst managers I've ever heard of. They intimated a few others were the same across the org.

Off topic - the manager mentioned was comically hilarious in laziness and lack of care. He once invited a new hire to a meeting, and attached the minutes of the meeting including the conversation that had not happened yet (they'd never even met yet) with statements like "Joe Foo agrees to let manager know if work is too much" etc etc.

It was a template he sent out and he never even came to those introductory meetings. Just you agree and then neither of you actually meet.

Government can be a different world so I understand when people think people are just inventing insane theories to justify their own ego, but these things really do happen in certain industries and types of companies.

This reflects my own experience too. Generally speaking, incompetent people don't get promoted in the same way you normally understand promotion. They don't get their boss's job. They don't stay in their boss's boss's reporting line.

They move to a nominally higher level role in a different department, often a cost centre, generally to some role which serves the organisation just by existing (e.g. a legally required role, or something that serves some certification purpose, or so that sales teams can say 'we have a dedicated person in our X office for doing that').

I can relate to all of these. One way to get rid of them is to put them on a "special project". Something low stakes and inconsequential to keep them busy.

That's also made to get rid of people. Can't fire them, but you can bore them out or generally disliking their work so they quit.

Better of course to engage with such people enabling them being productive. Or just buy them out of the contract.

What? The joke saying about that is "those who can't teach, teach gym".

Never heard anyone say that about software engineering.

Among many teachers it is, "those who can't teach become education consultants/researchers."

The joke at my school was when the engineering students failed (mech, EE, civ) out, they went into business, when they failed out of business they went into software engineering.

And what happened when they failed out of "software engineering"?

Then they start a coding bootcamp.

Psychology.

You cant fool me. It's AI all the way down.

This is supposed to be a joke but that's pretty obviously what would happen if you actually tried to do this.

You apply for your permit or whatever and the AI instantly denies you, so you start over and pull the lever again and this time the AI's random number generator comes up 777 and you get your permit.

Then the house can tune this for either reducing red tape (e.g. pulls are free) or favoring the rich (you have to pay a fee for each attempt so only the poor have to take the L once they run out of pocket money).