找回密码
 加入我们
搜索
      
查看: 6669|回复: 21

[显卡] 各位大佬 AMD的卡能跑本地大模型吗

[复制链接]
发表于 2025-1-16 09:38 | 显示全部楼层
7900xtx, 64G ram, amd 5900x@65w

  1. N:\>ollama run phi4:latest --verbose
  2. >>> briefly explain what is quantum physics
  3. Quantum physics, also known as quantum mechanics, is a fundamental theory in physics that describes the behavior
  4. of matter and energy at the smallest scales—such as atoms and subatomic particles like electrons and photons. It
  5. departs from classical physics by introducing concepts such as quantization, wave-particle duality, superposition,
  6. entanglement, and uncertainty.

  7. 1. **Quantization**: Many physical quantities, like energy, come in discrete units called "quanta." This means
  8. that these quantities can only take on specific, fixed values rather than a continuous range.

  9. 2. **Wave-Particle Duality**: Particles such as electrons exhibit both wave-like and particle-like properties. For
  10. example, they can interfere with themselves like waves while also being detected as discrete particles.

  11. 3. **Superposition**: A quantum system can exist in multiple states or configurations simultaneously until it is
  12. measured or observed. Once a measurement occurs, the system 'collapses' to one of the possible states.

  13. 4. **Entanglement**: Particles can become entangled such that the state of one particle is directly related to the
  14. state of another, no matter how far apart they are. Changes to the state of one entangled particle instantaneously
  15. affect its partner, a phenomenon Albert Einstein famously referred to as "spooky action at a distance."

  16. 5. **Uncertainty Principle**: Formulated by Werner Heisenberg, this principle states that certain pairs of
  17. properties (like position and momentum) cannot be simultaneously measured with arbitrary precision. The more
  18. precisely one property is known, the less precisely the other can be determined.

  19. Quantum physics has been incredibly successful in explaining a vast range of phenomena and forms the foundation
  20. for many modern technologies, including semiconductors, lasers, and quantum computing. Despite its
  21. counterintuitive nature, it remains one of the most well-validated theories in science.

  22. total duration:       6.3898724s
  23. load duration:        12.3162ms
  24. prompt eval count:    17 token(s)
  25. prompt eval duration: 118ms
  26. prompt eval rate:     144.07 tokens/s
  27. eval count:           373 token(s)
  28. eval duration:        6.254s
  29. eval rate:            59.64 tokens/s
复制代码
发表于 2025-1-16 10:13 | 显示全部楼层
悠悠lyz 发表于 2025-1-16 10:05
7900XTX单卡能跑Qwen 70B
推理可以,四五十token/s

我这最多就跑个32B的模型。你的单卡70B是怎么部署的?

  1. 运行BF16或FP16模型需要多卡至少144GB显存(例如2xA100-80G或5xV100-32G);运行Int4模型至少需要48GB显存(例如1xA100-80G或2xV100-32G)。
复制代码
您需要登录后才可以回帖 登录 | 加入我们

本版积分规则

Archiver|手机版|小黑屋|Chiphell ( 沪ICP备12027953号-5 )沪公网备310112100042806 上海市互联网违法与不良信息举报中心

GMT+8, 2025-6-12 04:36 , Processed in 0.007468 second(s), 5 queries , Gzip On, Redis On.

Powered by Discuz! X3.5 Licensed

© 2007-2024 Chiphell.com All rights reserved.

快速回复 返回顶部 返回列表