Prism confirms Apple is evaluating its LLM compression tech

PrismML CEO Babak Hassibi has confirmed that Apple and other tech companies are evaluating its LLM compression technology.

While saying discussion is at an early stage, Hassibi told CNBC, “They’re really evaluating our technology right now.., things are progressing nicely.”

Putting LLMs on a diet, to run them on device

The company makes a tech that can shrink AI models enough to run directly on an iPhone. This week it released a compressed version of Alibaba’s open-source Qwen model, reducing the size of the model from 54GB to 4GB while allowing all 27 billion model parameters to run on an iPhone. 

He told CNBC that Apple and others are evaluating what it does, measuring speed, energy efficiency, and performance.

Please subscribe to my human-curated daily Apple news summary on Substack

Prism claims its compressed models use up to 15x memory, up to 6x less energy and generate responses six to eight times faster than full weight models running on conventional hardware. There is a slight hit on performance in some fields. 

You can try Prism yourself.

The company has released two versions of its compressed Qwen models for use on iPhones and Macs and intends working on Google’s open-source Gemma model next.

What’s most important about this tech is that it could permit Apple to introduce support for full weight AI models on the device, enabling rock solid privacy while unleashing he full positive impact of AI across its platforms. We have to hope talks succeed and Apple doesn’t lose this opportunity to a competitor. 

You can follow me on social media! Join me on BlueSky, LinkedIn, Mastodon, and subscribe to the human-curated daily Apple news briefing at The Core.