• pebbles@sh.itjust.works
    cake
    link
    fedilink
    English
    arrow-up
    1
    ·
    2 hours ago

    I would reccomend you look into sythetic data. It is a real thing and is used intentionally in basically every LLM of the past year or more. Last I heard of an LLM without sythetic data was Dots LLM1 and that wasn’t competitive when it came out.