Close Menu
  • How to
  • Apps
  • Games
  • MOD APK
  • Root
  • About Us
  • Privacy Policy
  • Disclaimer
  • Contact Us
  • Write For Us
AndroidFit
  • How to
  • Apps
  • Games
  • MOD APK
  • Root
AndroidFit

How to Install and Run LLMs Locally on Android Phones

By S B Sohag How to
Share Facebook WhatsApp Twitter Pinterest Telegram Copy Link
How to Install and Run LLMs Locally on Android Phones
How to Install and Run LLMs Locally on Android Phones

In recent years, the capabilities of language models have grown exponentially, transforming the way we interact with technology. From generating human-like text to providing intelligent responses, these models have become integral to various applications. However, accessing these models often requires an internet connection, limiting their usability in offline scenarios. But what if you could harness the power of language models directly on your Android phone, without relying on the internet? In this guide, we’ll explore how to install and run Large Language Models (LLMs) locally on Android devices.

Understanding LLMs

Large Language Models, such as GPT (Generative Pre-trained Transformer) models, are deep learning models trained on vast amounts of text data. These models can understand and generate human-like text, making them incredibly versatile for tasks like language translation, text summarization, and conversation generation.

Benefits of Running LLMs Locally

Running LLMs locally on your Android device offers several advantages:

  1. Offline Accessibility: By installing LLMs on your phone, you can access their capabilities even without an internet connection, making them ideal for scenarios where connectivity is limited or unavailable.
  2. Privacy: Running LLMs locally enhances privacy by processing data directly on your device, reducing the need to send sensitive information over the internet to remote servers.
  3. Faster Response Times: Local execution of LLMs can result in faster response times compared to relying on remote servers, especially in situations where latency is a concern.

Install the MLC Chat App 

The MLC Chat app is not available on the Google Play Store. Hence, you have to sideload the app on your device. Follow the steps below.

Note: The MLC Chat app is still in the demo and is made specifically for the Galaxy S23 devices powered by the Snapdragon 8 Gen 2 chip.

1. Download and install the MLC Chat app on your device. You can download the app from the official website for free (148 MB).

2. After installing the MLC Chat app, you need to download the desired LLM model to use it for AI chatting. Make sure you are connected to the internet.

3. I recommend downloading the Phi-2 model first since it should run fine on most devices.

4. Tap on the Download button next to the model name to download it to your device. The download size and speed may vary depending on your internet connection.

5. Once the model is downloaded, you can let go of the internet and tap on the Chat icon to start using the model that you just downloaded.

Install the MLC
Install the MLC

6. You can now start chatting with the AI model locally. Note that you can give input and get output in text format only.

Model List:

  • Llama3-8B-Instruct-q3f16-MLC
  • gemma-2b-q4f16_1 (not working)
  • phi-2-q4f16_1 (recommended)
  • Llama-2-7b-chat-hf-q4f16_1
  • Mistral-7B-Instruct-v0.2-q4f16
  • RedPajama-INCITE-Chat-3B-v1-q4f16_1

Using the MLC Chat App

The MLC Chat app allows you to store LLMs on your device. At the time of writing, the MLC Chat app cannot leverage the NPU on your smartphone chip; instead, it uses the CPU to run these LLMs locally.

Moreover, the app is optimized for the Snapdragon chipsets only, particularly the Snapdragon 8 Gen 2 found on the Galaxy S23. So, if you have a MediaTek device like I do, the number of tokens per second might be low. 

For perspective, I was getting somewhere around 1.5 to 2 tokens per second (prefill) and about 4 tokens per second for decoding on my POCO X6 Pro which is powered by the MediaTek Dimensity 8300 Ultra chip. For the uninitiated, more number of tokens means faster and better performance.

tap on the Chat icon to start using your downloaded LLM
tap on the Chat icon to start using your downloaded LLM

Copy Text

Once you’ve got your response, you can tap and hold the text to select and copy it or continue the conversation with more prompts.

tap and hold the text
tap and hold the text

Reset the Chat

If you’d like to end a chat or conversation and start a new one, tap on the reset icon at the top right corner of the interface.

Reset the Chat
Reset the Chat

Conclusion

By following these steps, you can harness the power of Large Language Models directly on your Android phone, enabling offline access to advanced language processing capabilities. Whether you’re building a custom application or simply experimenting with LLMs on your device, running these models locally opens up a world of possibilities for enhancing productivity, privacy, and user experience. So why wait? Dive in and unlock the full potential of language models on your Android device today!

Also check this

  1. The 10 Best Android Developer Options Worth Tweaking
  2. 10 Best Stud Finder Apps for Android
  3. Lineage OS 18 / 18.1: Download Link and Supported Devices [2025]
  4. The 12 Top New Android 14 Features
  5. FRP Bypass APK Download 2025 Free Latest Tool All Version (With Guides)
Share. Facebook Twitter Pinterest Email Telegram WhatsApp Copy Link
Previous ArticleHow to Fix Xiaomi Phone Stuck on Redmi, Mi, Fastboot, MIUI Logo
Next Article How to Disable Call Recording Announcements on Android (Realme, Oppo, Vivo, iQOO, Xiaomi & OnePlus)
S B Sohag

Related Posts
How to

Fix “Something Went Wrong, Try Again” Error on Google Play Store (Android)

April 10, 2025
How to

FM Whatsapp vs Official Whatsapp [2025] – Which One is Best?

August 21, 2024
How to

Recover Deleted Text Messages on Android Without PC

July 9, 2024
Add A Comment
Leave A Reply Cancel Reply

Latest Posts
How to Install Google Play Store and Google Apps on Windows Subsystem for Android (WSA)
April 18, 2025
Fix “Something Went Wrong, Try Again” Error on Google Play Store (Android)
April 10, 2025
The 12 Best Offline Games on Android That Don’t Need Internet (2025 Edition)
April 2, 2025
How to Build Your Ultimate Fitness Plan with Android Apps
April 1, 2025
Android Secret Codes You Need to Know
March 29, 2025
Google’s Gemini Live AI Can Now See Your Screen—Is It Safe?
March 27, 2025
Sony Unveils Powerful New In-Ear Headphones with Glass Design: The WF-C710N
March 27, 2025
Everything About Samsung One UI 7.0 (Android 15) – February 2025 Update
February 14, 2025
Samsung Android 15 One UI 7 Update: Everything You Need to Know
February 14, 2025
Pikashow APK Download (Live IPL 2025) Latest Version
January 15, 2025
© 2025 AndroidFit
  • About Us
  • Privacy Policy
  • Disclaimer
  • Contact Us
  • Write For Us

Type above and press Enter to search. Press Esc to cancel.