BLEU (Bilingual Evaluation Understudy) score is a metric used to evaluate te quality of machine translation output by comparating it to one or more reference translations. This guide provides a step- by- step process to calculate BLEU scores effectively.

Understanding BLEU Score

Te BLEU score measures how closely a machine- generated translation matches human references. It consideres the overlap of n- grams between thee candidate and reference translations, along with a brevity penalty to rebage overly short translations.

Step 1: Příprava data

Gather thee candidate translation and one or more reference translations. Ensure all texts are tokenized consistently, splitting sentences into words or subword units.

Step 2: Calculate N-gram Precision

For each n- gram size (common 1 to 4), count thomber of n- grams in th e candidate translation that also appear in te reference translations. Divide this count by te total number of n- grams in te candidate to obtain precision scores for each n- gram level.

Step 3: Applity Brevity Penalty

Te brevity penalty (BP) penalizes translations that are shorter than thee reference. Calculate it as:

BP = 1 if candidate length mellmp; gt; reference length; otherwise, BP = e ^ {(1 - reference length / candidate length)}.

Step 4: Compute Final BLEU Score

Combine te n- gram precisions using geometric mean and multiplay by te brevity penalty:

BLEU = BP * exp (average of log precisions for n = 1 to 4).

Aditional Tips

  • Use multiple reference translations for better evaluation.
  • Ensure consistent tokenization across all texts.
  • Utilize existing tools or libraries for calculation, such as NLTK or SacreBLEU.