Saving private meaning: No Language Left Behind -NLLB-200 hands-on

Recently I wrote my BSc thesis that talks about tree recognition, from satellite imagery, with different deep learning models and to share my work with everyone I’ve published all the latex code in a GitHub repo here.
I wrote the whole document in Italian, my first language, given also that my university is located in north of Italy, after that I translated sources files thanks to an agent (Claude Code) and reviewed the translation manually: that agent is very popular and useful nowdays not only to write code but also to perform those operation, this time very basic, to work with files locally on your machine. That requires a pro subscription or at least an API key, and it uses a serious amount of tokens in input but also in output (that are surely more pricey). So I was wondering about a method to perform that task locally, without using precious token and money: private AI is now a reality, even with a consumer grade GPU that in low-midrange tier, that now sports easily 16GB of VRAM.

The Meta NLLB-200 3.3B

So I searched a lot and compared various options, but this one caught my attention (no I’m not a transformer :)) more than others: the Meta NLLB-200 with 3.3 bilion parameters.
It use a shared multilanguage vocabulary and the languages are identified thanks to special tokens (eg. eng_Latn, ita_Latn) that helps the model to return the correct translation.
That’s a transformer based deep learning model, with encoder-decoder architecture, that has 24 encoding layers and 24 decoding layers. We can analyze the structure per block.

Encoding

The source text is tokenized as usual with all transformer architecture, with positional encoding information.
QKV vectors are generated from linear input projection, then the attention score is calculated using Softmax $softmax(QK^T/√d) × V$ and applied to previously calculated scores.
With 16 heads, each head uses a 128-dimensional subspace and the concatenated output returns to 2048 dimensions (per token). After add&norm (addition and normalization) the data, the result passes to a feed-forward network with linear transformation -> ReLU -> linear transformation, followed by another add&norm.
The process is repeated by all 24 encoder layers and the KV, generated from encoder output after all 24 layers, are reported in a cross-attention layer that is present in the decoder layer.

Now I’m sure I lost you. The whole thing seems so cryptic, so lets try to break down a little bit.

The text in input is broken in tokens (or chunks of words) that become a vector of 2048 numbers, in addition to that the vector is provided with positional informations about the sequence of the words inside the input: in a sentence the order of the words matter, so its important to mantain the correct sequence of the chunks.
What is really happening in the encoder?
Multi-head attention is the mechanism that, in a transformer based model, can isolate what tokens are relevant. In every transformer model each vector, thanks to a linear projection, is transformed in Query (or Q, so what I’m looking for), Key (or K, what I’m offering) and Value (or V, the real content), let’s try to visualize these operations:

Sentence: “I eat pizza” need to be translated in Italian “Io mangio pizza”
Tokens (simplified): $[t_{0} = “I”, t_{1} = “eat”, t_{2} = “pizza”]$

We add a position vector to inject order informatiom
$x_{i} = Embedding (t_{i}) + PositionalEncoding (i)$

Now the vector is submitted to the heads, let’s assume that we have only two to simplify the whole thing. These heads are trained (the NLLB-200 model is already trained).
The heads has their own weight matrices $W^{Q}, W^{K},W^{V}$ that transform the input ${x}$ into QKV. The weight are matrices of numbers, that were previously defined and valorized while the model was trained and can isolate the correct part of the sentence. So for each token the calculation is made with a dot product:
$Q_a^i=x_iW_Q^A$
$K_i^A=x_iW_K^A$
$V_i^A=x_iW_V^A$

These are more practical examples:
Head1: match subjects and verbs, trained to connect actions and subjects.
Token “I” ( $x_0$ ):
– $\bold{Q_1}$ : looking for verbs relevant to “I”
– $\bold{K_1}$ : “I am a Subject/Pronoun“ (this vector is aligned to answer “Who?” queries)
– $\bold{V_1}$ : Content “First Person Singular” (the information to be transferred)

Token “eat” ( $x_2$ ):
– $\bold{Q_1}$ : “Where is my Subject?” (this vector points in the direction of subject-like Keys)
– $\bold{K_1}$ : “I am a Verb”
– $\bold{V_1}$ : Content “action of eating”

Head2: match context or object, the matrices in $W_2$ are different because they ignore grammar and focus on semantic relationship.
Token “eat” ( $x_1$ ):
– $\bold{Q_2}$ : looking for edible objects
– $\bold{K_2}$ : “I am an action”
– $\bold{V_2}$ : Content “Eating”

Token “pizza” ( $x_2$ ):
– $\bold{Q_2}$ : looking for context,
– $\bold{K_2}$ : “I am a some edible thing“ (This vector is aligned to edible query),
– $\bold{V_2}$ : Content “Dish/Food” (The core information to be transferred)”

Let’s summarize the previous operation, that we just saw, in a function called $Attention()$ : now two operations are applied to each vector:
Add: $x’ = x + Attention(x)$ thats the original word + the new generated vectors QKV to ensure that the original information won’t lost, this is called residual connection.
Norm: $Output = LayerNorm(x^{‘})$ math normalization with mean 0 and variance 1.

And now the FFN, or feed-forward network, that’s the “reasoning” phase.
Each vector pass through three distinct layers:
Linear(Expand): the vector is expanded in a much larger space, from 2048 values to 8192. If we take for example te vector related to “eat”, the features generated with the expansion can be “It is a physical action?”, “Does it involve food?”, etc.
ReLU(Activation): this function ( $max(0,x)$ ) act as a filter, that keeps only positive matches and exclude all irrelevant noise, in example:
– Feature “Is it related to food?”: Positive (Keep).
– Feature “Is it related to vehicle repair?”: Negative (Discard)
Linear(Compression): all the filtered feature that are now relevant are compressed down to the original 2048 dimension. So the vector is no longer just the “eat” word but it’s a concept, like “Activity of consuming food by a person”.

The last layer of the encoder is again an add&norm. All this procedure repeats until the encoder layer 24th.
Now the final encoder outputs (after 24 layers) become the fixed Keys and Values that the decoder’s cross-attention will query in all its 24 layers.

Decoding

The decoding layer is very similar to the encoding one, there are only two main differences, given to the fact that the output of the decoder should generate the final result, so a text:

1. Masked Self-Attention: the decoder looks at what is already written, which comes from the target text input. You have to imagine this process done token by token: when “I” is processed there is nothing already written (technically there is a [Start] signal, but for simplicity we can ignore it), but when “pizza” is processed, the decoder has the previous “I eat” as target text. This mask prevents the model from looking ahead during training, because the decoder receives the entire target sequence and should consider only the current and previous tokens.
2. Cross-Attention: this is really the new component that is placed between Self-Attention and FFN, this is connecting the languages so the language of the input words with the target language that is the traslated output one. The input of this layer are
– Query (Q): the word currently being translated that comes from the decoder (e.g. “eat”)
– Key (K) and Value (V): that’s the encoder output, the fully processed sentence (e.g. “I eat pizza”) these parameters act like a memory.
– Interaction: the decoder uses its current state as a query and compares it with all the encoder outputs (keys). When it finds strong matches, it uses the corresponding values to retrieve the most relevant information for generating the next word in the translation. For example, when processing the position where “mangio” will be generated, the decoder computes higher attention scores corresponding to the encoded representation of “eat”. It’s important to note that the word “mangio” cannot be retrieved from the encoder, which only contains the processed English input. Instead, the association between “eat” and “mangio” is encoded in the model’s weights, learned during the training procedure on millions of parallel sentence pairs.
– Result: refined token representations that align source and target meanings across languages.

After the final decoder layer, the softmax function converts the hidden state into token probabilities. The highest probability token becomes the next word in the translation.

Hands on

The model can be downloaded from the amazing Huggingface platform here.
It’s really easy to use thanks to Hugginface transformers library, that can help to load and launch the inference with few lines of code:

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

def translate_nllb(text: str) -> str:
    """Translate Italian to English using NLLB model."""

    # Setup
    device = "cuda" if torch.cuda.is_available() else "cpu"
    model_name = "facebook/nllb-200-3.3B"
    src_lang = "ita_Latn"  # Italian
    tgt_lang = "eng_Latn"  # English

    print(f"Loading NLLB model on {device}...")
    tokenizer = AutoTokenizer.from_pretrained(model_name, src_lang=src_lang)
    model = AutoModelForSeq2SeqLM.from_pretrained(model_name).to(device)

    # Translate
    print("Translating...")
    inputs = tokenizer(text, return_tensors="pt", padding=True, truncation=True, max_length=512).to(device)
    translated = model.generate(
        **inputs,
        forced_bos_token_id=tokenizer.convert_tokens_to_ids(tgt_lang),
        max_length=512
    )
    result = tokenizer.decode(translated[0], skip_special_tokens=True)

    return result

I ran these translations on my desktop with an Nvidia GPU via CUDA. At 3.3B parameters, the model fits comfortably on an 8GB VRAM card in FP16 precision, or even on a 4GB card with INT4 quantization (though with some quality trade-off). GPU inference processes ~500 tokens in 8-15 seconds. The same workload on a consumer-grade CPU takes 1.5-3 minutes, making GPU roughly 10x faster – a significant difference when translating large documents.

Results

Yeah sure, thanks for the explanation, but does it works?
Good question. Given that I have the approved translation of the thesis in English, I tried to extract some excerpt from it in order to get the translation from Italian through NLLB-200 an do some sort of comparison between the two versions. These translations are from the first introductory chapter of the document:

TRANSLATED (English)	NLLB-200	Analysis
This research aims to analyze and build a method to identify the presence of trees and green areas from satellite images of urban areas, using machine learning models.	This research aims to analyze and build a method to identify the presence of trees and green areas from satellite images of urban areas, using machine learning models.	Perfect match
In recent years, in fact, the advent of artificial intelligence has begun to change the way we work and study, allowing us to carry out research and also generate audio/video media from a simple text typed in a prompt.	In fact, in the last few years, the advent of artificial intelligence has begun to change the way we work and study, allowing us to conduct research and even generate audio/video media from a simple text typed into a prompt.	Very close ENG: “In recent years, in fact, the advent… carry out research and also generate…” NLLB-200: “In fact, in the last few years, the advent… conduct research and even generate…”
These types of artificial intelligence however, based on LLM models however very revolutionary, are general purpose and work on natural language.	These types of artificial intelligence however, based on LLM models although very revolutionary, are general purpose and work on natural language.	Close, maybe NLLB has better word choice here ENG: “based on LLM models however very revolutionary” NLLB-200: “based on LLM models although very revolutionary”
The idea is to use machine learning models that, using an artificial vision algorithm, are able to quickly identify the presence of objects, such as foliage or the shape of trees, for urban planning purposes or to feed green access indicators, such as that determined by the rule of 3-30-300.	The idea is to use machine learning models that, by exploiting an artificial vision algorithm, are able to quickly identify the presence of objects, such as hedges or the shape of trees, for urban planning purposes or to feed indicators of green access, such as that determined by the 3-30-300 rule.	Very close, the original translation is better in my opinion ENG: “foliage“, “using an artificial vision” NLLB-200: “hedges“, “by exploiting an artificial vision”

So the translations are good.
Yeah yeah, you are simply comparing four paragraph don’t you? If you are a real developer write a script that compare the whole document!
Unfortunately I can’t. This model can really translate text from a language to another, but we are talking about natural language: my thesis is written with LaTex, it cannot be translated from this model because there are a lot of code instructions to generate the document. If I launch the translation on the whole document maybe it won’t compile at all.

So why GPT & co. are more efficent than this model? Because they are multi purpose, decoder only, LLM models and they can recognize anything, so they can threat the code correctly as well as the text translation.

This test can clearly give us an overview of the current state of the art: aside of all those generative AI giants, there are a lot of models and solution that are perfectly working and usable to do something specific (e.g. image recognition, object detection, language translation and so on) and less power and capabilities eager than the usual bigger names.
And that’s actually the point.

The AI landscape isn’t just about GPT-4, Claude, or Gemini. Specialized models like NLLB-200 offer a practical alternative for specific tasks:

Zero API costs – runs entirely locally
Privacy – your data never leaves your machine
Efficiency – 3.3B parameters vs 100B+ in general LLMs

Yes, you’ll need to extract and preprocess text if you’re working with structured documents. But for similar translation tasks – converting articles, documentation, or plain text files – NLLB-200 delivers quality results without burning through API credits.

The takeaway? Don’t default to expensive general-purpose models for every task. The open-source ecosystem is rich with specialized tools that can solve specific problems more efficiently. Sometimes a focused 3.3B parameter model running on your own GPU is exactly what you need.

Cheers!

Cover image: Midjourney
Model graph: Matplotlib (Python)
Code: Me + Claude Code (Sonnet 4.5)
Text: Me 🙂 (grammar corrections made with Haiku 4.5)