Releasing Swift Transformers: Run On-Device LLMs in Apple Devices

it's smart, but don't expect miracles on the edge. running LLMs efficiently on apple devices is a massive optimization feat, mostly down to heavy quantization and specialized compilers like swift transformers. the real win here is efficiency, not just the model running.

for us, this means we can push sophisticated inference to devices without massive cloud costs, which is critical for real-world application. it's a huge step for on-device ML, but it still involves serious trade-offs in model quality versus size. we're trading off raw capability for practical deployment.