Aleph Alpha Announces Launch of a Pioneering New Tokenizer-free (T-Free) Architecture Innovation for Next-Generation Sovereign LLMs and a New Powerful Collaboration with AMD and Schwarz Digits

22.01.2025, Davos

• Aleph Alpha introduced a groundbreaking tokenizer-free (T-Free) LLM architecture that enables superior efficiency and effectiveness for fine-tuning and customization of AI across different languages, alphabets and specialized industries.
• This innovation addresses the limitations of conventional LLMs and unlocks new possibilities for sovereign AI solutions for governments and enterprises.
• The collaboration with AMD and Schwarz Digits strengthens Aleph Alpha’s new LLM architecture with high-performance computing and a sovereign cloud solution.

Aleph Alpha, a leading AI technology solutions provider headquartered in Germany, has announced a new architecture innovation for LLMs to address one of the most critical challenges in AI.

Teaching today’s popular closed- or open-source LLMs new languages or unique industry knowledge (often crucial for enterprises and governments) tends to produce underwhelming results and fine-tuning often proves ineffective. A key reason for this is that the patterns these LLMs learn are based on the tokenized version of the text they were trained on. If new text differs considerably from the original training data, it cannot be efficiently tokenized.

“Our tokenizer-free model architecture is a key solution to tackle a major challenge in AI: fine-tuning and creating sovereign models for diverse alphabets, low-resource languages and highly specific industry needs – areas where customization efforts have fallen short so far. Paired with the groundbreaking efficiency of the latest AMD AI technology, this marks a transformative leap, unlocking strategic opportunities previously out of reach,” says Jonas Andrulis, Founder & CEO, Aleph Alpha.

Aleph Alpha’s new LLM architecture does not require a tokenizer, extending LLM capabilities beyond the original training data. This approach solves a key problem of current Language Model Architectures, which most commonly can only be trained efficiently on data that uses similar tokens to their original training data. Fine-tuning current models on out-of-distribution data requires very large data sets (often not available for less common languages) and massive amounts of compute, resulting in a model with inferencing costs and an environmental footprint at an unsustainable level.

This groundbreaking development is set to lay the foundation for a new generation of sovereign AI solutions that enable governments and enterprises to build their own sovereign GenAI solutions based on their native languages, cultural specifications, and specialized knowledge and terminology. Consequently, this will democratize access to this technology and promote economic and cultural plurality.

“I founded Aleph Alpha with the mission to empower the sovereignty of countries and companies around the world in the era of AI. For our customers, this means open-source integration, maximal transparency and trustworthiness for a new era of human-machine collaboration, future-proof transformative AI solutions and free choice of execution environment,” says Jonas Andrulis, Founder & CEO, Aleph Alpha.

By combining Aleph Alpha’s cutting-edge GenAI technology with the performance and efficiency of AMD Instinct™ MI300 Series accelerators, and optimizations for the open AMD ROCm™ software stack, the company has achieved significant advancements in GenAI performance. Through leveraging the significant memory capacity of the AMD Instinct accelerators, this turnkey solution offers both performance and efficiency advantages, particularly for low-resource languages.

For example, using Finnish, this innovative AI architecture can deliver a 70 percent reduction in training cost and carbon footprint compared to alternative options (see the figure below, which presents a comparison of single-node performance measurements between T-Free and state-of-the-art tokenizer-based models).

“This collaboration brings more than AI – it delivers resiliency and innovation to the European AI ecosystem,” says Keith Strier, SVP, Global AI Markets, AMD. “We are thrilled to collaborate with Aleph Alpha and Schwarz Digits to boost Europe’s native AI capabilities and create a new AI trifecta for governments: a hyper-transparent GenAI platform, developed and trained within Europe, delivering exceptional efficiency on our AMD AI infrastructure.”

In addition to a high-performance compute platform based on AMD Instinct GPUs and the open AMD ROCm software stack, Aleph Alpha is also leveraging the AMD SiloAI team in Helsinki, which helped demonstrate the revolutionary multilingual capabilities of its new model architecture. This initiative is further strengthened by the data center and technical delivery capabilities of STACKIT, the sovereign cloud of Schwarz Digits, the IT and Digital Division of Schwarz Group, Europe’s largest retailer with 575,000 employees, based in Germany.

Aleph Alpha’s Pharia Operating System’s AI capabilities, now optimized on AMD, create a full-stack sovereign offering that combines AI software and hardware. This will deliver significant benefits in terms of computing costs and environmental footprint. In addition to the new Pharia LLM generation of tokenizer-free foundation models pioneered by Aleph Alpha, PhariaAI incorporates several unique innovations that enable unprecedented levels of explainability, transparency, and efficiency, facilitating the rapid integration of trustworthy GenAI into industry and government operations.

Our latest peer-reviewed research paper on tokenizer-free (T-Free) language modelling is now available here.

For more technical details, see our Blog here.