Get Another Text

Years ago I had the questionable idea to ignore the native Laravel localization features and adopt the more supported Gettext way, involving tools to automatically extract strings from the code and populate files in a widely recognized format. It seems I lost my bet: the Laravel native format got more traction, become even supported by crowd-sourced translation platforms, while a number of "laravel-gettext" PHP packages got published and abandoned one after the other leaving me with a choice: implement my own package and maintain it on my own, or desist and move to the Laravel format.

Yesterday I've spent all the day to migrate an application of mine (Spunta, a checklist manager), and here I want to share notes and considerations.

I had to convert my PO files in array-formatted files, keeping the match between different translations. So I generated a simple CSV to keep the strings in tabular form, with the following script using this utility library:

<?php

require('vendor/autoload.php');

use Gettext\Loader\PoLoader;

$loader = new PoLoader();

$master = $loader->loadFile('/path/to/resources/lang/i18n/en_US/LC_MESSAGES/messages.po');

$translations = [
    'it' => $loader->loadFile('/path/to/resources/lang/i18n/it_IT/LC_MESSAGES/messages.po'),
];

$output = fopen('out.csv', 'w+');
$row = ['file', 'id', 'status', 'en'];
foreach($translations as $lang => $trans) {
    $row[] = $lang;
}

fputcsv($output, $row);

foreach($master as $orig) {
    $str = $orig->getTranslation();
    $row = ['commons', '', '', $str];

    foreach($translations as $trans) {
        $t = $trans->find(null, $str)->getTranslation();
        $row[] = $t;
    }

    fputcsv($output, $row);
}

fclose($output);

In the above CSV file, I used to first three columns to keep (in order) the name of the file where to save the translation (by default, it was commons.php), the string identifier, and a convenience column where to annotate the status of string replacement in the many templates. Because, from here, it comes the hard, boring, time consuming part: assign an identifier to each string, and replace the strings in the templates, using that identifier in place of the string in the main language.

My suggestion here is to proceed template after template (instead of string after string), take advantage of the full review to normalize some string (in particular in capitalization, or suppressing the different forms of the same message), and organize the messages in groups and subgroups related to context and reference entities (using the array hierarchy of Laravel's translations files).

Once I've fixed all the strings in the CSV file, with this other script I've reassembled it in PHP files to be then moved in resources/lang/ folder:

<?php

function set_value(&$root, $compositeKey, $value) {
    $keys = explode('.', $compositeKey);
    while(count($keys) > 1) {
        $key = array_shift($keys);
        if (!isset($root[$key])) {
            $root[$key] = array();
        }

        $root = &$root[$key];
    }

    $key = reset($keys);
    $root[$key] = $value;
}

$f = fopen('out.csv', 'r');

$files = [];

$desc = fgetcsv($f);

for($i = 3; $i < count($desc); $i++) {
    $lang = $desc[$i];

    if (file_exists($lang) == false) {
        mkdir($lang, 0755);
    }

    $files[$lang] = [];
}

while($row = fgetcsv($f)) {
    $filename = $row[0];
    $key = $row[1];

    for($i = 3; $i < count($row); $i++) {
        $lang = $desc[$i];

        if (isset($files[$lang][$filename]) == false) {
            $files[$lang][$filename] = [];
        }

        set_value($files[$lang][$filename], $key, $row[$i]);
    }
}

fclose($f);

foreach($files as $lang => $filenames) {
    foreach($filenames as $filename => $data) {
        $path = sprintf('%s/%s.php', $lang, $filename);
        $content = sprintf("<?php\n\nreturn %s;", var_export($data, true));
        file_put_contents($path, $content);
    }
}

Finally I've found an undocumented feature of Laravel, to attach a custom callback invoked when a string is not found in translations. I've added the following lines of code in AppServiceProvider::boot() method to log the missing items, so to be informed of possible mistakes and errors:

app('translator')->handleMissingKeysUsing(function($key, $replace, $locale, $fallback) {
    \Log::error('Missing translation key ' . $key . ' for locale ' . $locale);
});

I've yet to decide if move other projects to the Laravel localizations format (in particular: GASdotto, involving more than 1000 strings...), but I'm pretty sure I will adopt it from the beginning when starting a new projects.