Tag: GPT

Hugging Face releases tools allowing running LLMs on devices

by Danis Tazetdinov

Hugging Face releases Swift Transformers – package to implement transformers APIs in Swift.

Overall, couple of products were released and updated:

  • swift-transformers, an in-development Swift package to implement a transformers-like API in Swift focused on text generation. It is an evolution of swift-coreml-transformers with broader goals: Hub integration, arbitrary tokenizer support, and pluggable models.
  • swift-chat, a simple app demonstrating how to use the package.
  • An updated version of exporters, a Core ML conversion package for transformers models.
  • An updated version of transformers-to-coreml, a no-code Core ML conversion tool built on exporters.
  • Some converted models, such as Llama 2 7B or Falcon 7B, ready for use with these text generation tools.

Package currently supports following models:

  • Llama 2;
  • Falcon;
  • StarCoder models, based on a variant of the GPT architecture;
  • GPT family, including GPT2, distilgpt, GPT-NeoX, GPT-J.

As of now package is in active development. It is not expected to be used in production projects without appropriate testing.

GitHub:

Hugging Face: Releasing Swift Transformers: Run On-Device LLMs in Apple Devices

Apple restricts generative AI app age rating to 17+

by Danis Tazetdinov

Apple requires BlueMail app developers to set its rating to 17+ due to use of generative AI models which potentially could provide responses not suitable for children.

This rule could be then extended to all apps working with generative models, especially GPT models which are getting traction recently.

References: