• Womble@piefed.world
    link
    fedilink
    English
    arrow-up
    3
    arrow-down
    10
    ·
    2 days ago

    You almost certainly weren’t using a Deepseek model offline (unless you have 100s of GB of RAM/VRAM). You might have been using a small model distilled on Deepseek responses, but the smallest model made by Deepseek is about a 300B param model.

    • SteveTech@aussie.zone
      link
      fedilink
      English
      arrow-up
      13
      ·
      2 days ago

      the smallest model made by Deepseek is about a 300B param model.

      This is not correct. DeepSeek has plenty of smaller models. I just had a scroll through their huggingface, and picked out a few: DeepSeek-V2-Lite (16B), deepseek-moe-16b-chat, deepseek-llm-7b-chat, but there’s quite a few more too.

      Though DeepSeek R1 is their flagship model, which is the one people usually think of, and that’s a 684B param model.

      • Womble@piefed.world
        link
        fedilink
        English
        arrow-up
        4
        ·
        edit-2
        21 hours ago

        huh fair enough, I hadn’t seen those before, they’re from before deepseek made a name for themselves (which as you say was with R1). They are very out of date though, as is R1 (from the end of 2024 and start of 2025 respectively). the current set of models from Deepseek come in 291B and 1.6T varieties and the previous version only was released in 685B

        You’d almost certainly be better off upgrading to qwen or Gemma4 for a local model of that kind of size.