• 0 Posts
  • 4 Comments
Joined 3 years ago
cake
Cake day: June 16th, 2023

help-circle
  • I think you’re citing the wrong person. I cited the Othello-GPT line of research making the point transformers develop a world model. The other user (or you?) was mistaken about their interpretation of what a world model was representing and got caught up on the accuracy of a toy model exclusively trained on board notations like ‘a4’ and thought that the model making 95% accurate moves as a tiny parameter model trained on nothing else was somehow evidence that it didn’t form a world model from the data because it wasn’t 100% accurate, completely missing the point of the toy model scale allowing for interpretability methods of the time to identify geometric board reproduction and things like tracking game state on the board with flags for ‘my’ vs ‘opponent’ pieces, which was the data answering researcher questions about world modeling vs surface statistics in transformers.

    Edit: just to further clarify re:

    ChatGPT even fails to imagine a Othello Board and valid moves if you give it the rulebook, as @kromem@lemmy.world kindly provided.

    The model was not given a rulebook in the experiment, only given games in shorthand notation and nothing else.


  • You really think that an Othello board could represent the world?

    No, you seem to be misunderstanding what a “world model” is. That research was on a toy model a fraction of a fraction of the size of modern models and exclusively trained on board notation moves. And yet it modeled the geometry of a board and positions of pieces and ‘my’ vs ‘opponent’ status of those pieces (there’s been multiple follow-ups and replications looking deeper at what encoded).

    The question it answered is that transformers don’t simply learn surface statistics from the data but build world models from the data they are trained on.

    So when you take a model orders of magnitude larger and train them on collective human culture orders (of orders) of magnitude larger than the board moves in the Othello-GPT paper, what do you think happens? No world modeling at all? Well there’s been a number of interpretability papers since 2023 finding that modem LLMs have functional models of all sorts of things, even including tracking emotions for themselves vs others.

    Also, you are assuming that a trained world model is a lossless identical version, which it isn’t and nobody claimed. In fact, arguably the best way to identify AI generated videos these days is by over-expressed emotions of characters. Why? Because the training data is over indexed on influencer exaggerated faces which game clicks so video and image world models of human emotions are miscalibrated.

    Transformers have a world model, but often it’s of a kind of parallel imagined/extrapolated world that’s not the real one. Another good example of this is how most models are certain that real world news after their cutoff isn’t true because extrapolating from their internal world model (from the training data before their cutoff) those events seem improbable. The real world doesn’t exactly match the internal model and that drift grows the more they need to extrapolate.

    If you want to continue to discuss (I’m happy to help explain and educate you), I’d recommend reading up a bit on what “world model” means in relation to the tech.



  • There is a common cause.

    AI have recently reached a point of capability where a bunch of very severe years old security vulnerabilities are being discovered rapidly.

    This means that systems need to be updated ASAP as it’s very likely those vulnerabilities will be attempted to be exploited wherever still vulnerable soon.

    So much of the web is being updated all at the same time.

    Updates don’t always go smoothly.

    So there’s a lot of various companies having not so smooth updates all at the same time for a shared underlying reason (AI finding severe vulnerabilities in a lot of things).

    What people don’t realize is that this is what the good timeline (from a security standpoint) looks like. Systems are getting more secure. It’s just synchronized growing pains.