How to run LLM on a laptop?

How to run a LLM in a laptop?

Step 1: Get a laptop with >16 GB RAM and core i7 processor (GPU too is good, but I did not use it, It took 800% from cpu, 10% from Memory 4GB, for a HelloWorld)
Step 2: Get Permission to LLAMA LLM usage from meta (Waiting Time 24 - 48 Hours, frankly unnecessary)
Step 3: Install HuggingFace-hub
Step 4: Download any LLAMA model variant from HF using huggingface-cli (7B)
Step 5: Install ctransformers
Step 6: Run the script (that you create) that opens the downloaded LLM file and use prompts on it (Check any HF LLAMA model for this)

References

relevant post. —

Did it work for you?


What do you think? Let me know your thoughts in the comments below.




Enjoy Reading This Article?

Here are some more articles you might like to read next:

  • Worthy Words #3: Shadow Work
  • Can taxes be evaded if not a salaried job?
  • Is capitalism becoming like dinosaurs?
  • How I finally partially understood game devlopment?
  • Survey of Existing Solutions in LLM