A guide to Running Llama 3–8B locally on your Local Device (Mac, Linux, and Windows)

By Jon Shrestha

May 10, 2024

I recently wanted to check out Llama 3 myself given all the hype around it. Thanks to Facebook (i still forget to call them Meta),it is open source. More reason to try them and play with this model.

They have two new models out with a third one still cooking. The models are Llama 3 with 8 billion and 70 billion parameters and 400 billion is still getting trained.

Anyway most of us don’t have the hope of running 70 billion parameter model on our personal device, forget about the 400 billion one.

Note: LLama 8B model needs a minimum of 16GB of RAM.

With that out of the way here is how you run it.

Step 1:

Download ollama from here: https://ollama.com/

Select your system. It supports macOS, Linux, and Windows.

Run the installation file and once it's installed on your system you are ready for the next step.

Step 2:

Open terminal or even better try warp (https://www.warp.dev/), if you are on Mac. It's a fun terminal alternative that I recently started using. Much better in my opinion.

Run this command:

ollama pull llama3

This will automatically pull the llama3 8 billion parameter model.

Step 3:

You are done!

Run this command to start chatting with your own local LLM model:

ollama run llama3

Enjoy!