• Mo.@lemmy.world
    link
    fedilink
    English
    arrow-up
    9
    arrow-down
    2
    ·
    4 hours ago

    No. Everyone doesn’t decide to do this shit all at once and pushing this narrative away from the true blame factor (the current administration) is fucked and scary and propaganda.

    Are there nuggets of truth here? Yes, absolutely. Anyone in tech knows that. But this is not that. There’s too much happening in core systems all at once at the same time and there’s no otherworldly reason that all of these people independent of one another all decided to update their systems at the same fucking time.

    Get off of it.

    Twisting the narrative or spinning it away from the ghouls that lit this fuse years ago is fucking dangerous and I’m really surprised to see it being pushed so very hard here of all places.

    We’re smarter than that.

  • jj4211@lemmy.world
    link
    fedilink
    English
    arrow-up
    17
    ·
    6 hours ago

    While frequently true, the recent events seem to be more the opposite, deploying improvements to systems that weren’t broken (yet).

    The FAA? Upgrading to a new piece of network equipment. An act of finally working toward refreshing long neglected infrastructure.

    Microsoft? Some developer pushed a windows os change in July, and circumstances fired off now to expose it as flawed.

    Banking? Again, some developer at one bank pushing some change.

    As to why all at once, could be coincidence, but if it isn’t the likely connecting factor is AI. You need to create the complex configuration for a piece of network gear? What a pain, just feed Claude the current config and ask to port it to new equipment. Want some nebulous software improvement but don’t have any ideas? Have Claude look for, provide a solution, and also have it “audit” and “test” its own solution. I have seen all of the above happen and go horribly wrong when I am actually in a position to intercept, but it is super popular.

    • AEsheron@lemmy.world
      link
      fedilink
      English
      arrow-up
      16
      ·
      6 hours ago

      Read a comment elsewhere that the banks aren’t quite comfortable enough with AI to let it make the code. But, what they are doing, is using AI to hunt for vulnerabilities, and get people to patch them. And the AI are finding tons of holes, so there is pressure to push tons of updates to fix them, because if their AI can find them, then hackers’ AI can find them. And so the workload has exploded with tasks labeled Priority 1, Critical! non-stop, resulting in sloppy updates.

    • Katana314@lemmy.world
      link
      fedilink
      English
      arrow-up
      2
      ·
      6 hours ago

      It is very possible they became pressured to perform an upgrade because something on the old system was compromised and out of support.

      Legacy stacks quite often maintain old tech, and then somewhere in there will be a library that has a critical exploit and has been totally replaced for future major versions.

      • korazail@eviltoast.org
        link
        fedilink
        English
        arrow-up
        1
        ·
        6 hours ago

        Unsure if you’re defending or objecting to the comment. Flawed and outdated code needs to be replaced and that pressure should be maintained. The ‘Go Fast And Break Things’ model works OK in an internal corporate environment, but not when we’re talking about the FAA or banking. Outlook can fuck right off.

        Outdated hardware needs to be replaced, but I’m with JJ about the likely root cause. Some boss decided that the cheapest way to fulfill their job/contract was to install or deploy whatever ChatGPT told them would work.

        • PuddleOfKittens@sh.itjust.works
          link
          fedilink
          English
          arrow-up
          2
          ·
          5 hours ago

          Outdated hardware needs to be replaced,

          No it doesn’t. Nuclear launch systems were running on floppy disks in the 2010s, IIRC, if it ain’t broke then “fixing it” is at best a nice to have.

          Now, if it’s insecure then we should replace it. Unless you’re replacing it with something even less secure (like AI, perhaps).

          • korazail@eviltoast.org
            link
            fedilink
            English
            arrow-up
            1
            ·
            3 hours ago

            fair - not broken (yet) doesn’t mean shouldn’t be maintained, though.

            Most solid state things are pretty much static, by definition, but there’s still stuff like plastic dry rot or metal corrosion. We’re talking decades, but we’re also talking decades. If that critical router failed outright the result might have been worse while a replacement was sourced, with a follow-on failure when the code bombed too.

            I’m not defending the outage – there should have been better testing – but I’ve been on the receiving end of a minimal but crucial server’s power supply capacitor failing (the post-mortem was wild to figure out what broke). That server was ancient and should have been replaced before it failed and the fire drill to recover it (even just taking a backup and restoring to a VM) was unpleasant.

      • jj4211@lemmy.world
        link
        fedilink
        English
        arrow-up
        1
        ·
        6 hours ago

        On the bank, a possibility, on Microsoft, based on what they detailed sounded like a go at performance optimization.

        For security, I get it, there is little time to lose in this scenario, usually. But huge volume of changes for change’s sake are also in play.

  • LotrOrc@lemmy.world
    link
    fedilink
    English
    arrow-up
    1
    arrow-down
    1
    ·
    3 hours ago

    Bullshit like israel and the current administration aren’t the ones doing it…

    Think every system is overloaded and going to fry at the same time? If you do you’re an absolute idiot

    • Optional@lemmy.worldOP
      link
      fedilink
      English
      arrow-up
      1
      ·
      2 hours ago

      In theory, but in practice the market bends towards bullshit and quadrant-leaders. Those companies (microsoft) are fucked up for the same reaons - they didn’t want to let the engineers apply redundancy beyond a paper thin excuse layer. Rinse, repeat, and wait.

    • JustEnoughDucks@feddit.nl
      link
      fedilink
      English
      arrow-up
      20
      ·
      11 hours ago

      The market optimizes for extreme short term profit squeezing with the least amount of labor possible. The only efficiency it has is in exploitation and labor extraction.

  • Stupidmanager@lemmy.world
    link
    fedilink
    English
    arrow-up
    6
    ·
    11 hours ago

    I work in the industry, I have these friends and coworkers too. One of them is wrong, a lot - in this case He still is. But this doesn’t go unnoticed when the shit hits the fan. now we work in an industry that wants to move fast, so expect more of it with far more blame.

    The sky is falling folks.

  • Trad Wench@thelemmy.club
    link
    fedilink
    English
    arrow-up
    2
    arrow-down
    1
    ·
    7 hours ago

    Multiple things breaking across industries over such short periof of time- is that a coincidence then?

  • jackal@infosec.pub
    link
    fedilink
    English
    arrow-up
    93
    ·
    18 hours ago

    You missed the second post so I have it below. I’m dealing with this at my job and it’s so fucking aggravating to deal with executives who only care about this quarter’s performance.

    Mastodon post from @Viss@mastodon.social

    In case of missing image:

    you know when you cheap out and buy the cheap thing, but it turns out the cheap thing works like 5 times then breaks, so you have to decide whether to concede to defeat and spend the money on the right thing, but the sunken cost fallacy is tugging at you to buy the cheap thing and try again, but then exactly the same happens again and now youre kinda stuck?

    its that but its computers acting as digital infrastructure all over the country

    run by guys who abjectly refuse to admit defeat ever.

    • Valmond@lemmy.dbzer0.com
      link
      fedilink
      English
      arrow-up
      15
      ·
      11 hours ago

      Yeah the problem is that the middle management has an order to carry out; make +2.5% this quarter. They do not have an order; make it sustainable.

      My guess is that it’s all about metrics, you can see the line go up, but you can’t assess sustainability without relying on trust (which can and will be broken and abused in our capitalist system, because we all need to eat), so line go up it is.

  • NovaVeles@lemmy.world
    link
    fedilink
    English
    arrow-up
    111
    ·
    19 hours ago

    Welcome to IT. “Everything is fine, why do we pay you?!” - “Everything is broken, why do we pay you?”

  • WhoIzDisIz@lemmy.today
    link
    fedilink
    English
    arrow-up
    195
    arrow-down
    1
    ·
    21 hours ago

    Forgot to clarify that the reason they “don’t feel like fixing any of it” is because doing so doesn’t directly contribute a positive effect on the bottom line, and they’re too fucking short-sighted to understand how having invested in reliable infrastructure would have kept them from hemorrhaging money now.

    • credo@lemmy.world
      link
      fedilink
      English
      arrow-up
      2
      ·
      5 hours ago

      I’ve been trying to answer the question, “How do you show the value of preventitive action?” for a long time. I still don’t have a good answer, other than, you’ll know when you don’t do it.

      Same with public heath and I’m sure many other professions I can’t think of right now.

    • jimmy90@lemmy.world
      link
      fedilink
      English
      arrow-up
      1
      arrow-down
      1
      ·
      edit-2
      5 hours ago

      that applies to the IOT bullshit that happened to the supermarket freezers yeah

      the “systemic” outages from Microsoft, the FAA and Linux kernel bug tsunami are incredibly complex systems that are tested to hell and back but human error will barely creep through

      not because the fucking boss is a greedy capitalist

      i know this is how the commie brain rationalises the complex world you live in which is why you’re such delayed learners

    • Seb the goblin@lemmy.world
      link
      fedilink
      English
      arrow-up
      30
      ·
      16 hours ago

      Literally. I had to fight tooth and nail, repeatedly saying to people with Head in their title that if this [migration project that should have been done 2y ago] doesn’t go ahead, then in 6 months company will not have a product. Still ended up with lots of whinging, compromises, running over deadline due to said compromised, and then performance reviews coming in with “your product roadmap didn’t have much in it” like bruh, we were trying to put out the damn thermite trying to burn down everything

    • Boomer Humor Doomergod@lemmy.world
      link
      fedilink
      English
      arrow-up
      3
      ·
      17 hours ago

      What do you mean? The rental cars and ride shares and hotels made bank. $300 hotel rooms and $600 rental cars.

      It’s just poor people’s money anyway and they don’t have enough to be worthy of consideration