BDH-CQ: In-Context Learning with Recurrent Latent Reasoning
arxiv.org
We introduce BDH-CQ, a reasoning model that combines in-context learning with recurrent latent reasoning. Inputs presented at inference time continuously update the model's recurrent memory; the model then solves a query through iterative computation in a high-dimensional latent space, without verbalizing its intermediate reasoning. We evaluate the model on the public ARC-AGI-1 evaluation set and use controlled ARC-like interventions to study what it learns from demonstrations, how consistently it applies an inferred transformation, and which concepts remain difficult. A 150M-parameter configuration reaches 29.5% pass@2 at a computed inference cost of \$0.0007 per task. This operating point breaks through the previously reported ARC-AGI-1 cost-accuracy Pareto frontier, establishing a new state of the art in benchmark cost efficiency.
    • hoshikarakitaridia@lemmy.world
      9·
      10 days ago

      29% is probably somewhere between meh and ok-ish.

      150M is extremely small. The B stands for billion parameters, so the M stands for million parameters. So we’re talking 3% the size of recent mid-tier models.

      What this means is someone made a tiny model, and it scored way better than we thought. The conclusion is (I assume) we can make models way smaller at the same quality.

      That’s huge, because that would mean in a matter of years anyone can deploy their own model on the most dog shit hardware. This would kill AI companies, it would neuter Nvidia and it would mean PC components will decrease to suitable prices. And it also means even in poor countries people have access to AI which I personally think is one of the most noble things to finally make people more equal by giving them lots of knowledge.

      • jatone@lemmy.dbzer0.comEnglish
        31·
        10 days ago

        yes, models can be much smaller than people think. esp when they’re fit to purpose. I have a couple models I deploy in apps that perform better than the 27 gig models models and are only 20-100MB. its about training, data processing, and understanding how information theory plays into the domain space you’re solving for.

      • foremanguy@lemmy.ml
        1·
        8 days ago

        Mostly right but don’t forget that technological advancement never leads to cheaper more affordable components, and more equality for the people Just the opposite the big techs continues growing faster utilizing even more ressources

        • hoshikarakitaridia@lemmy.world
          3·
          8 days ago

          I disagree. Look at how many poor countries have been transformed just by virtue of cheap smartphones.

          Give them a device to communicate with one another and suddenly their life becomes a heck of a lot easier.