IBM launches Granite R2 multilingual AI embeddings

IBM has launched two open AI models for searching information across more than 200 languages, including nine programming languages. The new granite-embedding-97m-multilingual-r2 and granite-embedding-311m-multilingual-r2 models turn text into numerical representations that let an application find related documents, even when they are written in another language.

This is what many semantic search and RAG systems use. RAG is a technique that lets a chatbot consult documents before answering. For example, a question in Spanish could locate a manual written in German or an answer in English inside a multilingual database.

A small model with unusual results

The 97-million-parameter model is the more notable of the two. In the MTEB Multilingual Retrieval benchmark, which measures the ability to find relevant text in 18 languages, it scored 60.3 points.

According to IBM, that is the highest score among open models with fewer than 100 million parameters. The multilingual-e5-small, one of its competitors at that size, scored 50.9 points.

Size matters because it determines how much space the model takes up and what resources it needs to run. The 97-million-parameter model's weights take up about 195 MB, while its ONNX version, prepared for more efficient CPU inference, weighs 98 MB.

That makes it suitable for modest servers, edge devices or applications that need to process many queries with low latency. IBM says it can encode more than 2,500 documents per second on an NVIDIA H100 GPU using 512-token chunks.

More capacity for long documents

The 311-million-parameter model scores 65.2 points on the same benchmark. According to data presented by IBM, that is the second-highest score among open models with fewer than 500 million parameters.

Both models accept up to 32,768 tokens, compared with 512 tokens in the previous Granite R1 generation. In practice, they can analyze a much larger portion of a contract, technical report or article before turning it into a representation for search.

The improvement is especially clear in LongEmbed, a benchmark focused on long documents. The 97-million-parameter model scored 65.6 points and the 311-million-parameter model reached 71.7, compared with 34.3 and 37.7 for their respective R1 predecessors.

They also include specialized training with code in Python, Go, Java, JavaScript, PHP, Ruby, SQL, C and C++. This makes it possible, for example, to search for an equivalent function in another language or find related snippets within a codebase.

Which model should you choose?

The 97-million-parameter version is designed to prioritize speed, size and CPU deployment. It generates 384-dimensional vectors, a compact representation that reduces the cost of storing and comparing thousands or millions of documents.

The 311-million-parameter version generates 768-dimensional vectors and adds support for Matryoshka. This technique lets you truncate the vector to 512, 384, 256 or 128 dimensions without running the model again.

According to IBM, reducing it from 768 to 256 dimensions lowers the multilingual retrieval score only from 65.2 to 64.7. In return, the index needs approximately one-third of the space and similarity searches become cheaper.

In practical terms:

• Choose granite-embedding-97m-multilingual-r2 if you need low resource use, high speed or deployment on devices with limited resources.
• Choose granite-embedding-311m-multilingual-r2 if you prioritize quality, long documents or search across many language pairs.
• If your data is almost exclusively in English, the English-specialized Granite R2 models may deliver better results with a smaller footprint.

What changes for developers

The models are released under the Apache 2.0 license, an open license that makes them easier to use and modify, including in commercial products. They work with sentence-transformers and transformers, and include weights for ONNX and OpenVINO, two formats designed to speed up execution on CPUs and other devices.

They can also be integrated with LangChain, LlamaIndex, Haystack and Milvus by changing only the model name. They do not require special instructions before each query, so an application that already uses a standard embeddings interface can test them without redesigning its code.

For end users, the change will not be a new feature visible on a screen. It will be search that better understands questions in different languages, finds information in long documents and can run without sending data to an external API.

Granite Embedding Multilingual R2 does not eliminate differences between languages: IBM specifically trained 52 languages, while the remaining more than 200 receive general coverage. Even so, the combination of an open license, 32K context and a compact model could make multilingual search less dependent on large-scale infrastructure.