• 0 Posts
  • 19 Comments
Joined 3 years ago
cake
Cake day: June 16th, 2023

help-circle




  • Yeah, the FAA incident may or may not be related, but it certainly seems just like an averted disaster at my workplace where someone proposed a Claude translated network config to go from one vendor to another and I had to manually interject and insist on a review and found that claude completely screwed up something that would have done… well pretty much exactly what happened to the FAA, had it been actually applied.

    The banking issue they flat out mentioned it was AI related, but they only explicitly admitted it was trying to remedy an AI detected security issue. Question is whether they took the AI suggested remediation and got screwed by it, or if a human boffed the remediation, and it’s tough to say without the bank coming out and being explicit about it.

    Microsoft came out kind of vague and said some change was made to Windows OS in July that caused the catastrophe that they had to unwind. Again light on detail, but given their excessive public bullishness on full on agentic AI and the sort of change described, it’s almost certainly a vibed change in pursuit of what would likely be considered a premature optimization. Huge risk, no reward sort of change for the sake of someone’s AI KPIs, probably.

    A bit grumpy because in my work, I try to be as hard on the AI skepticism as my company will take, and onboarded a pretty all-in AI coder and thought I had at least gotten them to actually make sure their change at least specifically did as they intended, and not merely passed the automated test suites. I sat with him interactively on some pull requests to make sure he actually made a plan to try out his changes and validate the behavior with his own eyes, and thought I instilled a good habit. So shocker as the first ‘big test’ after I trusted that he actually vetted some of his changes fell over. His pull request even explicitly listed the scenarios and claimed that each one was tried. He then had to admit that Claude just made up that it tested the scenarios tested but he figured it was a safe bet if Claude ‘audited’ it that it was ok if it lied about actually trying the scenario…


  • Yeah, the things is the processes at work are all driven by optimism, “won’t fix” is not an available option for a bug, and “not bothering” isn’t a valid answer to a PR. And both the owner and reporter must agree on any closure.

    So we end up with perma-issues that can’t be closed.

    I can effectively block the PR, but so long as the submitter wants it open… Doesn’t matter how senior I am.



  • Just so long as your ultimate pull request is plausibly human viable.

    Have been fighting someone because they submitted a 15,000 line pull request for ‘one’ feature and no one is going to review that mess. I gave a very quick ‘for example’ showing in the first 30 lines of changes that it’s screwing around with stuff that has nothing to do with what the sloperator said they pointed out.

    Now they are complaining that I won’t take code even though Claude audited it and said it is all good…

    (This is work, so I don’t have the luxury of just permanently ignoring them).




  • Pure slop is not copyrightable is likely the case (the couple of cases where they ruled that works with zero human involvement do not grant any human copyright). However, the moment a human tweaks it, that resultant work can qualify for copyright. Consider that Steamboat Willie Mickey is public domain, but despite being derived from that design, Fantasia Mickey is still copyrighted. In the cases where courts ruled the human didn’t have copyright, the human had loudly and proudly proclaimed that there was no human in the creative process. If they claimed they applied some transformative work to it, then that would be copyright eligible. Further, a person is not compelled to divulge the ‘raw’ materials, so even if the slop they tweaked would be public domain, they are under no obligation to show the ‘pre-human’ form of the data.

    On the matter of LLM works being derived by virtue of training, unfortunately it is looking like the courts are broadly siding with the AI companies. Not only is it not inherently infringing to train models using material without express permission, it is also the case that if the output is infringing, the user, not the provider, is on the hook for the infringement.


  • First have to get courts to recognize that the training is subject to copyright for a license to apply in the first place, which isn’t looking good.

    I fully anticipate that there absolutely will be copyright consequences for AI generated knock-offs of big software, but conveniently the well resourced AI companies won’t be on the hook, just the end users will be left holding the bag. By that point, the license goes back to working as intended already.


  • I guess the key is that ‘open source’ generally means ‘free use’ versus ‘source available’ which at least in the old days used to be a supremely common arrangement for commercial software.

    I will also agree that every other IP pretty much is effectively ‘source available’, you see a book, there’s no hidden secret form you need to construct that work. Nothing inherently stops you from reproducing a modified copy. Sure there are more convenient assets (having distinct tracks that are used to compose a song versus only the post-mix recording), but generally other IP has to decide how to deal with remix, sampling, etc, without some fundamental barrier like ‘it got compiled into machine code which is hard to modify’.





  • You are correct. Further, AI kind of fell into their lap, though they did have the core competency to sustain supporting that.

    However, even if Huang did not drive the success directly, another leader could easily tank things. Just like how when a good president can’t magic the economy into being great, a very bad president can screw it up.

    I would say the sort of potential CEO to secure leadership of the second largest market cap in the world is the most likely sort of terrible leader to do that. The sharks that circle for that sort of prize are the ones most eager to burn it all to the ground for the sake of short term self enrichment.