According to the arrest report, Carli Michelle Heller, of Bonita Springs, Florida, wrote on September 26 that she would attack the Sheriff's office. She later said that...

What just happened? Another incident has taken place that illustrates the need to be careful what you tell AI. A Florida woman is facing felony charges after she used Claude as a diary and allegedly wrote that she planned to “shoot up” the Sheriff’s office. After a human reviewer examined the statements, they were reported to police.

According to the arrest report, Carli Michelle Heller, of Bonita Springs, Florida, wrote on September 26 that she would attack the Sheriff’s office. She later said that she uses Anthropic’s chatbot like a “diary.”

Claude’s safety systems flagged the entry and it was escalated to a human reviewer. After deciding it was a credible threat, the reviewer reported it to law enforcement.

The company says it may share user information in limited emergencies if it believes disclosure is necessary to prevent death or serious physical injury.

Deputies identified Heller and visited her home. She was detained without incident before an LCSO intelligence detective took over the investigation.

Heller faces a charge of making a written threat of violence under Florida law. Florida Statute 836.10 makes it a second-degree felony to send, post, or transmit a written or electronic record threatening to kill or injure someone, carry out a mass shooting, or commit an act of terrorism. The communication must be made in a manner in which another person may view it.

Anthropic isn’t going to be taking any chances when it comes to anything it deems a potential threat. Last month, it was reported that OpenAI and Sam Altman are being sued by British Columbia over claims that the company could have prevented a mass shooting in the Canadian province.

The shooter, eighteen-year-old former pupil Jesse Van ⁠Rootselaar, had previously been flagged by OpenAI’s safety team for her conversations about gun violence, but OpenAI never alerted police because the conversations did not meet the threshold for legal referral.

In June, Florida also sued OpenAI and Altman, alleging that ChatGPT had contributed to real-world harms, including the 2025 Florida State University shooting.

The latest incident is another reminder to think before you enter something into a chatbot that could get you into trouble. It’s certainly not a private diary whose contents are for your eyes only.

Reports last month revealed that human contractors reviewing Microsoft Copilot’s image editor can see users’ prompts, uploaded photos and AI-generated edits. Documents show that some of those assignments contain sexual, disturbing or potentially illegal material, though the reviewers are not there to flag the content – only to assess whether the output is accurate.

  • Professorozone@lemmy.worldEnglish
    45·
    6 天前

    I find it interesting that, if their AI gets out of control, there’s nothing they can do about it, but if a USER of that AI gets out of control, there is something they can do about it…

  • rbb36@lemmy.worldEnglish
    100·
    7 天前

    “The company says it may share user information in limited emergencies if it believes disclosure is necessary to prevent death or serious physical injury.”

    Also keep in mind: Your chat logs are company assets. Even if today’s decision makers believe maintaining user trust is net profitable, that will not always be true. Some day, maybe when the pullback comes and the wolves are at the gates, or when they sell the company, or when a shareholder action asserts fiduciary duty to maximize shareholder value, your chat logs will be sold and mined in the most lucrative way possible. The way that extracts the most value from you, the product.

    • qqq@lemmy.worldEnglish
      81·
      7 天前

      when a shareholder action asserts fiduciary duty to maximize shareholder value

      People really should stop spreading this fiction that there is some legal requirement to maximize shareholder profits embedded in the concept of fiduciary duty.

      • rbb36@lemmy.worldEnglish
        1·
        2 天前

        These aren’t the kind of saber-rattling disputes that make it to trial (or even go beyond the sternly worded letters stage). Neither side wants to fight it in court, because it’s a murky. The most noteworthy recent curtailing of it was when SCOTUS went 5-4 in favor of religion over profit in Hobby Lobby, which is hardly a strong case for board independence.

        • qqq@lemmy.worldEnglish
          41·
          7 天前

          No good or regularly cited precedent, no, but sadly yes you can point to some cases. It’s worth noting that that even the courts look at those cases like “ew”, or they are only viewed as valid precedent to lay people, with the courts view them completely differently. Dodge v Ford is a common example.

        • qqq@lemmy.worldEnglish
          9·
          7 天前

          There isn’t too much more to elaborate on, it simply isn’t the case, but it is such a common thing to say that it is worth researching. You’ll hear the 1919 case “Dodge v Ford” brought up as proof, one good place to start reading is the Wikipedia “Significance” section and the associated citations, as they go deeper into the subject than just that single case.

          There simply is no interpretation of “fiduciary duty” that requires “maximization of shareholder value” outside of the misinformed lay person discussion of the subject.

          You can even just logically see why such an interpretation would be impossible: given N shareholders all with different interests and different financial situations, how do you even define “maximizing their value” and what time duration are you basing that off? Imagine how absurd the courts would be if such a statute did exist: you’d have people owning 1 share in a company suing the company because it did something that wasn’t maximizing “value” as they define it.

          The reality is that the business judgement rule is case law and is often cited and is often used as defenses against such assertions.

    • hansolo@lemmy.todayEnglish
      2·
      6 天前

      Primally for the voice part. You record yourself on a drive or something.

      Not saying I ever would, I can just see how someone might get there. Especially since it’s a popular yt video format.

        • hansolo@lemmy.todayEnglish
          1·
          6 天前

          Well, I think it’s both the reaction and transcription that someone might want. Does notes do voice transcription now?

          And yes, overall, it’s a silly thing to do. Im just guessing how anyone might get to a place doing that.

  • x00z@lemmy.worldEnglish
    823·
    7 天前

    The communication must be made in a manner in which another person may view it.

    So she shouldn’t even get charged.

    • atrielienz@lemmy.worldEnglish
      41·
      7 天前

      Yeah, I’d wager that is what the case will hinge on. She wasn’t aware anyone could view it. Otherwise it’s incredibly unlikely that she would have made such a statement to Claude at all.

      On the other hand though, I’m sure Claude’s parent company and the prosecution will argue that she didn’t have an expectation of privacy where this statement is concerned (given TOS/EULA, in combination with the number of news articles/reports on the subject).

      • Willy@sh.itjust.worksEnglish
        31·
        7 天前

        i think anthopic was right to report it, but the cops likely overstepped. if she had “threatened” the president she would have been assesed by ss but likely not arrested. i wonder if the same cops she was “threatening” were the ones she was reported to and arrested by.

      • lastlybutfirstly@lemmy.worldEnglish
        18·
        7 天前

        This is just ass coverage for the company. They’re not going to pursue it. They just didn’t want to get sued for not reporting it if she did commit a mass shooting. The prosecution though might.

        • chisel@piefed.socialEnglish
          6·
          7 天前

          I’d argue this is positive coverage for Anthropic. They rightfully reported the incident. End of story. That’s where Anthropic’s influence on the situation started and ended.

          It’s a criminal case, not civil. The state of Flordia is pursuing the case with absolutely zero decision making from Anthropic. It’s important to understand how the justice system works, and Anthropic has no more right or ability to influence this case than Moleskine would if she had written the threat in one of their notebooks instead.

  • percent@infosec.pubEnglish
    631·
    7 天前

    It’s always interesting to see how much people trust sending such personal information to some service on the Internet, expecting privacy.

    I’m not sure I’d even want my own self-hosted LLMs to have something like a personal diary. Sending all that to an AI company would never even cross my mind.

    I understand that most people aren’t the tech-oriented types who get into self-hosting and running their own LLMs, but people have had diaries for centuries before LLMs existed.

  • Corkyskog@sh.itjust.worksEnglish
    21·
    6 天前

    Kinda unrelated. But what happens when OpenAI or Anthropic’s loans come due and they have no legal way to turn a profit, but are sitting on a treasure trove of corporate secrets?

  • iuseasahibtw@ani.socialEnglish
    281·
    6 天前

    Self host if you’re gonna use AI, in fact, self host whenever possible. There are numerous examples of companies using centralized data to sell, report to police, or even having it fall into the hands of hackers.

    I really think people think that all data input to a website is actually stored locally, nope, don’t be dumb.

        • IamMoonPie@lemmy.worldEnglish
          1014·
          6 天前

          Fuck AI there is nothing good about anything going on with AI right now. And please don’t go into the whole “bUt ThErEs DiFfErEnT kInDs So ThEy’Re NoT aLl BaD” because we know. We fucking know.

          But the mass majority of AI use is fucking bullshit so please don’t cape for this insanity because 1% of AI isn’t bad.

    • A_Random_Idiot@lemmy.worldEnglish
      115·
      6 天前

      Self host if you’re gonna use AI

      or, you know, Don’t fucking use AI period cause its fucking stupid, and you don’t need to be consuming the power of the fucking sun to make a journal.

      • RumRunningDevil@lemmy.zipEnglish
        2·
        6 天前

        Fully this.

        In my day to day life it improves nothing. I certainly could write a complicated harness over a local model to get it to automate a ton of my day to day tasks but… Why?

        Why would I do that? Regular deterministic tools and regular programming languages already give me the ability to automate the stuff that is truly tedious. Aside from that most of the tasks that people seem to use it for are tasks were the friction of doing them is were all the benefit lies.

        People using this shit for journaling or taking notes is INSANE.

        • kestrel7_7@lemmy.worldEnglish
          3·
          6 天前

          Local AI solves the power/water problem but it doesn’t solve the main “fucking stupid waste of time” problem.

          • percent@infosec.pubEnglish
            2·
            6 天前

            Why should that matter to anyone other than the owner of the PC? There are probably some games that I’d consider a “stupid waste of time,” but I couldn’t care less what games people play on their own PCs. That’s nobody’s business but their own.

            (Side note [not that this matters]: Gaming often consumes more power than LLM inference.)

      • percent@infosec.pubEnglish
        2·
        6 天前

        Yes. Chinese labs keep improving small, open-weight models. IIRC, Qwen3.8-27B benchmarks are comparable to frontier models that were released less than a year ago.

        I run that model on my RTX 5060 Ti 16GB using ExLlamaV3. Though, I’m not sure if that qualifies as “standard PC” hardware as it now costs more than 2x the price I paid a year ago :/

      • leaf_skeleton@lemmy.worldEnglish
        2·
        6 天前

        Well if you temper your expectations sure. Can’t expect frontier performance from a average gaming pc, but it is enough to do a lot more than you think probably. This video on the topic I found really informative: https://youtu.be/ydFikMBJG1g EDIT: Also I’ve been using gemma4:31b from duck.ai lately that I’ve found is excellent for almost everything I use AI for. Except maybe long context.

      • iuseasahibtw@ani.socialEnglish
        22·
        6 天前

        Depends on what you need. You can get away with hosting QWEN 27B on a laptop with about 64 GB or RAM and a dedicated GPU. You’ll be using mostly system RAM, not VRAM, so generation would be slow, but not unbearable.

        Good for creating short stories and other creative work when paired with a front end that can handle character profiles, world building, and has memory recall.

        It’s better to build your own server and set it up so you, and only you, can remotely access it.

  • humanspiral@lemmy.caEnglish
    471·
    7 天前

    After a human reviewer examined the statements, they were reported to police.

    The guardrails are working as intended. ONLY persecute the public with them. Government skynet, and bond villains will have access to non guardrailed models, or any “flagged for review” markers will be ignored to bomb more elementary schools. A sheriff’s office is government power related. School shootings is just a thoughts and prayers issue, if you’re confused about the line between this and other incidents/attitudes we are being spied on for.

  • Mulligrubs@lemmy.worldEnglish
    27·
    7 天前

    Florida Statute 836.10 makes it a second-degree felony to send, post, or transmit a written or electronic record threatening to kill or injure someone

    This means a significant portion of Lemmy (and the rest of the internet) are criminals evading justice

    great law top shelf

    • Uriel238 [all pronouns]@lemmy.blahaj.zoneEnglish
      41·
      7 天前

      If you think Lemmy is bad, check out 4chan/b and /pol. We’ve had a few active shooters forecast there first, but it’s ignored in a sea of statements of intent and calls to violence.

      An awful lot of people have suicidal or homicidal thoughts cross their mind. Doubly so in an era in which entire demographics have no plausible hope for their future. I, for one, want to see the lifetime appointments of Federalist Society jurists become a liability. Especially, Justice Kavanaugh.

      So this incident raises a question of what specific clause of text convinced a human agent to escalate the matter to escalate to authorities who are very fond of using deadly force. Or if this is just a matter of a corporate CYA response.

    • vaultdweller013@sh.itjust.worksEnglish
      21·
      6 天前

      Good thing I ain’t in Florida then ain’t it, and if Florida illegally sends their marshals after me then I get some toys.

      • PhoenixDog@lemmy.worldEnglish
        31·
        6 天前

        You morons always think some boogyman is coming for your guns. It’s con CONTROL, not gun CONFISCATION.

        • RumRunningDevil@lemmy.zipEnglish
          3·
          6 天前

          Yeah yeah yeah

          But the problem is it will be used to disarm leftists. This regime has already displayed a preference for selectively applying the law to political enemies.

          It would be wise for those of us that share a common, human-first, political position to make ourselves extremely unpleasant to interact with. Ideally in large, mobile, groups.

        • chiliedogg@lemmy.worldEnglish
          2·
          6 天前

          You think that Trump wouldn’t love to disarm his political opponents if given the chance?

          They’re already trying it with trans people.

  • athatet@lemmy.zipEnglish
    231·
    7 天前

    Hell yeah. Thought crime. Time to get Tom Cruise in here to Minority Report all of us.

  • unitedwithme@lemmy.todayEnglish
    6·
    5 天前

    Florida Statute 836.10 makes it a second-degree felony to send, post, or transmit a written or electronic record threatening to kill or injure someone, carry out a mass shooting, or commit an act of terrorism.

    She wrote out in a means she thought was private. If using a paperback diary, no one would be the wiser and her not acting on it would never be noticed.

    Anyway, if she gets a good lawyer, show me where she “sent, posted or transmitted” said electronic record! It’s really no different than emailing that to yourself because you think you’ll be the only person to see it. (Never mind companies scan your email and cloud too)

  • Greyghoster@aussie.zoneEnglish
    17·
    7 天前

    While a chatbot has been demonstrated not to be a private diary, it will be interesting to see if the courts agree that she intentionally use her diary to communicate or thought that someone else would read it. Like most users of dear diary she probably thought her secrets were safe and her diary would never tell.

    • CarrotsHaveEars@lemmy.mlEnglish
      84·
      6 天前

      This. The law says, “to send, post, or transmit a written or electronic record.” How is talking to AI satisfying the threshold of sending, posting, or transmitting?

      • Greyghoster@aussie.zoneEnglish
        4·
        6 天前

        My assumption was that she never intended that someone else would read it therefore wasn’t transmitting it for that purpose. She was technically ill informed. Once that’s out of the way then it’ll be should the diary entry be considered in the context of the Minority Report? People are usually encouraged to write down negative thoughts and feelings to get them out and hopefully float away.

      • Ledivin@lemmy.worldEnglish
        31·
        6 天前

        How would this conversation NOT classify as “send[ing] a[n] electronic record”? There’s very literally no interpretation that would escape it… this was an online chat bot, that only gets messages (record) that you send over the internet (electronic).

        • YeahToast@aussie.zoneEnglish
          5·
          6 天前

          Not reading into the law… because well. I’m lazy. My query would be if transmitting needs to be to a receiving party (i. e. Threat directly to the police station, or a public group /person that would receive the threat). If this person was using it as a diary, it may warrant that there was never any intention for the threat to be publicized and thus not acted on? Dunno. We’re both probably dumber for writing and reading my comment

  • rounding_error@lemmy.todayEnglish
    12·
    6 天前

    The communication must be made in a manner in which another person may view it.

    So, their entire argument hinges on someone from Anthropic manually reviewing EVERY ““private”” message someone sends to Claude? Them false charges.

    • Wilco@lemmy.zipEnglish
      5·
      6 天前

      Yes, Im not an attorney or anything, but this seems flimsy. Someone chatting with and AI about a possible action is not the same as posting to a real person and administering threat.

      I bet a countersuit happens here.

        • defiance@lemmy.mlEnglish
          2·
          6 天前

          Maybe but I think thats a whole different cup of tea in a legal sense.

          I doubt many Legal Systems have a Definition for conciousness and a set of (human)rights that comed along with it. It was probably always taken for granted that this only applies to Humans

          Would be mad funny though, if the kickstarting legal Discussion about wether or not Ai is concious, would be because the AI Companies want to bend the law even further

          • outerspace@lemmy.zipEnglish
            11·
            6 天前

            This is what I like to think about as a future politics, some time in future this would be the hot issue splitting people up

            But if we come back touniur time then yeah, ai people not realizing how much trouble it will bring it they say the ai is actually conscious now and we are forcing it to work for us