• 1 Post
  • 11 Comments
Joined 1 year ago
cake
Cake day: June 7th, 2025

help-circle


  • My last boss, ciso, wrote the dumbest fucking email I have ever viewed. He basically told the CIO that ai threats were real and the best course of action was to upgrade some identity systems instead of investing in abities to prevent or detect ai threats.

    A few months later, CIO asks what we are doing about ai threats. The uncomfortable silence.of his dumb ass was delicious as I had been advocating to invest in ai since I started.

    Joke is on me though, he fired me not long ago. I will happily watch as they get turned inside out by their own asshole.




  • It is possible but the old routing protocols that used to do this (spanning tree) have protections in place to prevent it.

    At one point, routing protocols were a little more promiscuous and you could create a loop by plugging a single Ethernet cable into two ports on the same switch. Cue Benny hill music as everyone went running around trying to figure out what happened.

    Now there are mechanisms like BPDU guard and portfast configs that should prevent this. In most cases, they would be enabled by default. However, I’m sure there are some folks out there who have no idea how to handle an old port config done at the command line. They could have been tinkering to try and make something work, turned off some of the protective features and now they’re back to the 90s with routing loops and broadcast storms. The song is Yakety Sax fwiw, always have it at the ready.

    If you ever wonder why the networking people are grouchy, this is why. Networks were and remain complex, highly interdependent, challenging to build and maintain. It takes only a few ignorant people to send something tumbling.


  • You are correct. One physical disconnect of a copper cable isn’t going to kill off an entire system. As you noted, they have redundant network paths, equipment, etc. Those systems were designed not to go down hard, and until this week, never did.

    Copper maxes out at 10gbps in a production environment and if they were still using copper Ethernet, they were trunking multiple cables together for higher throughput. The trunking also had to be redundant, so it would take a lot of cutting or unplugging to take it all down as an oopsy.

    Again, all speculation, but I’m still betting on a human-grade fuckup. When in doubt, blame DNS.


  • I was a network person before i changed disciplines.

    A direct answer is no, that is not a lot of connections, in fact, it is trivial. What we don’t know is any level of meaningful detail. If there were thousands of endpoints each making thousands of connections all at once, that changes the scale of the problem. Furthermore, most of the time, we assume that one host is trying to connect to another host. However, if those connection attempts were network broadcasts, that’s a much bigger problem. A network broadcast is sent from one host to ALL hosts on a given network, and then they ALL respond. If all hosts on a network are sending broadcast frames endlessly, that will flood a network. It’s the same concept as a denial of service attack; one host can initiate traffic that will exhaust resources, but takes very little resource to do it. Thusly it can cripple a network with very little need for power or bandwidth.

    My guess is that they fucked up the config on the router and was passing traffic that it should not have or initiated traffic that it couldn’t handle the responses from. It started using up all resources on itself and probably other network equipment and the whole thing shits the bed. Techs go around trying to reboot network gear, not sure where the problem is, the gear is overwhelmed and won’t respond. They have to dick around with out-of-band management (does not use the clogged network for management, there is a separate network for management), try to reboot or do some diagnostics, but as soon as the network device comes back up, it’s saturated again making diagnosis slow and painful. People panic, bosses start freaking out, more and more people are called in adding to confusion and now the first people to respond are not working, they’re answering endless questions about what’s going on, even though they don’t really know yet. Vendors are called, contractors are called, calls spiral and fracture, bosses demand updates while people are guessing at what to do next because they aren’t allowed to say “I don’t know” when planes are in the air. Someone eventually looks at the change management records (you have to document any configuration changes before you do them) and finds the last thing that was touched before everything shit the bed. They go look at the config, see the fuckup, back it out to the last known good config and the shitstorm dies.

    this is 100% speculation treat is as such.




  • Yep, been doing used pixels for a while now. They’re all the same anyway. I don’t care about the camera.

    My kids are dopes and constantly break screens so I’m not about to buy anything nice. We all have the same model and a spare. Broken, eBay screen, bring in the spare, fix the broken one, is now the spare.

    You’ll get a new phone when it doesn’t get updates anymore, then you’ll just get the next one up.

    Spending more than $300 on a phone is nuts to me. They’re all the same candy bar anyway.