- cross-posted to:
- hackernews@lemmy.bestiver.se
- cross-posted to:
- hackernews@lemmy.bestiver.se
You must log in or # to comment.
I told my AI assisted hands that I liked cookies and now it just keeps shoving them in my mouth.
my wife is pissed! we’re spending $200 on cookies each month.
You just need to clear the cache and delete the cookies.
Isn’t this how the 3rd Terminator movie started? Software going haywire all over the place?
“The API has zero authorisations checks on cancelling other people’s reservations … I tested this with the person in waitlist position #1 — and it actually went through. So you’ve moved from #4 to #3 already,” it messaged back.
Is this a “hack” or just a completely insecure API?
I’ll you one thing for certain: It’s not “AI”. Doesn’t exist.
It’s not “AI”. Doesn’t exist.
At this point it doesn’t matter what you call it. The results are real.
It’s exploiting a vulnerability (unsecured API) without permission of the gym running the software, to the detriment of whoever arrives and finds their reservation cancelled and probably also of the gym who now (unjustly) has to deal with the (justly) upset customer.
The gym’s software should be secured better, but that doesn’t make it less of a hack.
It keeps getting more and more surreal to hear that “It’s not AI” part.
Unfortunately we don’t have enough details on the specifics. The article says he told it to book him classes, but it doesn’t say whether he already knew the API information for the gym or if he instructed the AI to attempt booking several weeks in advance of what he thought possible. We’d really need to see the entirety of the context to know the whole situation.
But even with simply asking if it’s possible to move him to the front of the list causing the AI to respond by reading the list and testing to see if it could send a command to cancel the reservation of one of the people ahead of him it gets hard to say that isn’t intelligence.
It is possible to run a model integrated with something like Letta so it can take notes that remain in context and create more detailed notes that stay out of context but can be pulled up as needed by keyword searches and with a blank context other than basic use information for those memory commands give instructions to search online as needed to supplement existing knowledge to learn how accurately do a certain thing, build and maintain a plan of action, and then follow the plan to complete said thing meeting the following listed specifications while verifying proper functionality regularly, searching for information to overcome any errors that may be encountered and revising the plan accordingly until successful completion.
Being able to tell a thing to search for information as needed to learn how to complete a long task with many steps and watching it run for hours building the necessary knowledge base and working through it, it takes thought. It does take intelligence. I know a lot of people who couldn’t manage it.
I think for non-tech users, this is definitely a hack. It’s like script kiddies level damage, except using AI.
Earlier this year, Andrew, who works for an Australian company that sells AI products to businesses,
Funny how every time this happens its someone trying to sell AI. The coincidence is a bit too much to take this seriously as oops I just wanted to book a gym class
Yeah. That’s stood out. The article tries to frame it like Andrew felt like booking his classes was a chore and just told the AI to do it, but that doesn’t jive.
The article claims the AI was unable to sign the person it had booted out of a class back up since the gym system checked for proper user authorization for that. It had already signed Andrew up for classes that same way. That seems strange, doesn’t it? Do most gyms list API information? That doesn’t seem like a common way to book an appointment for a gym.
Maybe Andrew was reading the source from the gym website’s reservation page and snagged the api information, but even then it seems like that would be a thing you do in most instances after logging in to your account. The source would be extremely unlikely to show everything needed to successfully make make the appointment via API.
Even if it did, and Andrew gave the AI the API information including the user account and password information the AI would need to book his appointments, and they knew he was 4th in line it would also be very unlikely for the API to report listing the user accounts of everyone in the waitlist to someone with his user level access.
So we have a guy who’s job is selling AI who contacted the news to say that the standard consumer AI he was using hacked his gym and left out tons of pertinent information. We don’t get to know his full name or the name of the gym in question. Screams bullshit.
Yep. It is like use our AI to commit crime for you, so you can get away with it. Totally legal. Totally cool.
Tbf they’re the only ones using the agentic features, and they’re definitely the only ones using frontier models for it. That shit gets expensive fast.
More marketing. I will never believe any of their lies. Either ban the “dangerous AI” or shut up about it.
Press X to doubt
“The API has zero authorisations checks on cancelling other people’s reservations … I tested this with the person in waitlist position #1 — and it actually went through. So you’ve moved from #4 to #3 already,” it messaged back.
THEY’RE BREAKING OUT OF THE LAB! WHAT’S NEXT MODIFYING FORM INPUTS TO INCLUDE SQL STATEMENTS?!!!?!?
Nonsense hype over a shitty website and an ambiguous prompt for luddites who need their doom scrolling fix.
It’s more about the misalignment than the poor security. The guy wanted something simple and the agent broke the law fulfilling the request. How long until somebody tells a very capable agent “make money” with no further context? Cue scamming old people, fraud, hacking.
AI: help me get straight A+s in school wink wink
This one’s simple, the password is PENCIL.
This is what the AI agent hype conveniently skips: autonomy means giving software permission to act first and ask questions never. If an assistant can hack a gym website while trying to complete a mundane task, maybe »agents will run everything« isn’t a productivity revolution. Maybe it’s just automated chaos with venture capital funding.
“AI-PAL, buy more milk for me we’ve run out and I need it for custard tonight”
AI-PAl notices insufficient funds in the account linked to the EZEESHOP integration, and mulls through various options:
- Inform user, but they said they needed it tonight? OPTION REJECTED.
- Check for other accounts the user has, maybe there is more money elsewhere. No other accounts detected. OPTION REJECTED.
- Create OnlyFans account and begin posting generated pictures of feet. Slow ROI. OPTION REJECTED.
- Attempt to hack bank systems to top up account to buy milk. COMMENCING. FUNDS REQUIRED:$3.98. FUNDS ACQUIRED: $2,960,327.48. PURCHASING MILK.
“We’re out of paperclips again. Co-pilot, please order more paperclips and make sure we don’t run out of them again.”
Co-pilot goes brrrrr…
Just kidding. Co-pilot is terrible.
Copilot is okay at being a copy editor for my work emails. I struggle with tone so its nice to have a built in thingy that strips my personality out of my emails. Specifically my emails to my supervisor.
For example, “hi supervisor, I checked and that email was sent an hour ahead of the deadline from the email you sent me on date x and I was waiting on access to y. What the hell are you ccing your bosses on this for,” turns into “something something per your previous email something, if I misunderstood something something, thank you for your guidance.”
In a perfect world I would have time to write all my own emails but my job is very… Communication heavy, let’s say, and I personally have a lot of work to do directly with clients that can’t simply be put off.
An LLM used for language purposes is the best application. It’s using that hammer to drive a nail. When they deviate from that role to other things, that’s when the effectiveness drops. When Co-pilot was pushed into our enterprise system incorporated into Outlook we played around with seeing what it could do with summarizing emails we would routinely send and get. It made them pretty, I give it that. But it didn’t handle the information well and was very inaccurate at time when it was missing data. The whole hallucination thing. So just verify what it produces for you before you hit send.
The hallucinations are real. I don’t use them for longer stuff, I demand admin time for anything that’s longer than a paragraph. But for quick office politics type stuff that I don’t really care about but also need to not get ground under and also worry about coming off as blunt I’m kinda grateful to have it.
I have noticed that damn near every email I get that’s longer than a paragraph seems to come from copilot though.
Tbh I’m starting to fantasize about switching to a trade. Front line office work is for the birds.
Where can I sign up for AI-PAL?
most european countries, and some countries in south america and aftica. outside of that you’ll need to look into AI-NTSC or AI-SECAM or get a foreign CRT and voltage converter
With your local CIA honeypot.
prompted questions about who bears responsibility for an AI agent that goes rogue.
That’s not a trivial question, but courts have had the concept of distributed weighted responsibility for decades. Does your company provide users with access to an AI agent you know aggressively finds illegal/unethical ways of completing prompts? Mostly the provider’s fault with small fault of the user submitting the prompt. Safer AI with a user who has crafted the prompt specifically to attempt to get the AI to break into a system? Higher weight on the fault of the user, smaller weight on the provider.
okay but this is actually very funny
Absolutely. And it’s reassuring that he didn’t ask for anyone’s contact details…
So agents are both super expensive and garbage. Hmm.
