On-device 0.5B LLMs, voice/text in, action out, outperform GPT-4.
A 0.5 billion-parameter language model developed for high-performance function calling on edge devices.
Uses a novel 'functional token' strategy to reduce context length by 95%.
Demonstrates a 140x faster inference speed compared to the RAG solution, and is 4x faster than GPT-4o.
Achieves 98%+ function call accuracy, surpassing previous models and matching the performance of GPT-4.
On-device language model for super agent: Octopus-v2-2B, an advanced open-source language model with 2 billion parameters, represents Nexa AI's research breakthrough in applying large language models (LLMs) for function calling. Octopus-v2-2B introduces a unique functional token strategy for both its training and inference stages. This approach not only allows it to achieve performance levels comparable to GPT-4 but also significantly boosts its inference speed beyond that of RAG-based methods.
Explore our collection of 200+ Premium Webflow Templates
