Important note: 8~12GB RAM is required
One important note in advance of main topic. Check RAM capacity of a target smart phone or tablet. Executing chat, and it loads a language model to memory. Probably, more than 8GB or 12GB RAM is required to hold a model fully on memory.
The video at the end of this post shows forced termination by Android OS due to loading failure, not error or crash as programming bug. This is the case of Microsoft Surface Duo. It has 6GB RAM, but it is not sufficient to hold a language model on memory.
Although successful cases are reported on the net, some of them don't open usage model. Probably, they are recent high class model as Google Pixel 7 Pro.
Required source code and tools are
Building execution binary
Android NDK is stored at the next folder.
In this assumption, path of "android.toolchain.cmake" is
Path of source code is
Execute commands to build binary at this folder, execution file "chat" is generated under "build" folder. Series of command is
mkdir build cd build cmake -DCMAKE_TOOLCHAIN_FILE=/opt/android-ndk-r25c/build/cmake/android.toolchain.cmake -DANDROID_ABI=arm64-v8a -DANDROID_PLATFORM=android-23 -DCMAKE_C_FLAGS=-march=armv8.4a+dotprod .. make
Termux needs permission to access storage. At first, assign storage permission at "Settings" of Android.
Basically, Termux can access and control under its folder, "/data/data/com.termux/files/home". Executing the next command at Termux, Android internal storage is mounted as "storage" under there.
Copy "chat" and Alpaca model to Android smart phone, and run.
To execute "chat" from Termux, it should be located under Termux. Following procedure is required from copying files to run "chat".
- copy "chat" and Alpaca model to Android once. Example, "alpaca" folder under "Download".
- copy them under Termux.
- grant execution privileges to "chat"
- run "chat"
Series of commans is
mkdir alpaca cp storage/shared/Download/aplaca/* ./alpaca/. cd alpaca/ chmod 777 chat ./chat
But unfortunately, this process is terminated by Android OS as the movie below. This is due to memory shortage to load a model to memory. In case of Linux, it can be avoided with swap*6, but Android can't take this solution due to missing root.twitter.com