Magento Translate System

In this blog post we will look into the details of magento translate system.

Basics

Magento translate system is used to setup a single website in multiple languages.
To setup a simple multi language system, we can create either a new store view or a new store or a new website from magento admin. For this go to System -> Manage Stores and create a new store view/store/website. Next go to System -> Configuration -> General -> Locale and set the language you want the new store to be in. We will not cover in detail how to set this up properly as its not relevant to the current blog.

There are two ways using which we can setup translations

1. Using translation csv files
2. Inline Translate Feature

Translate csv file are located in app/locale/en_US by default. You can further add translational files for each language as well. In this folder translation files are setup per module, also for each module in the config.xml file we can define the translation file to use

<frontend>
   <translate>
        <modules>
                <Mage_Catalog>
                    <files>
                        <default>Mage_Catalog.csv</default>
                    </files>
                </Mage_Catalog>
        </modules>
   </translate>
</frontend> 

similarly we can define a different translate file to “adminhtml” area as well.
There is a second location where translate file is located: “app/design/frontend/default/{theme}/locale/en_US/translate.csv”. In this location we can have only a single translate file and not multiple files.

TIP: Translate files are simple key value pair files. Key is the input string and value is output in the desired language.

Inline Translate feature needs to be enabled from “System -> Configuration -> Advanced -> Developer -> Translate Inline”. Here you can turn on translate for admin or frontend. Turning this on allows us to directly edit/save translations from magento web ui itself. These translations are saved in magento database in table “core/translate”

Using Translation In Custom Modules and Themes

We have seen above how to include a custom translation file for our theme and module. To enable translation in our template files we need to use the function __(''); provided by magento. The function “__()” can take in n-number of arguments and works same the as php’s sprintffunction.
e.g

echo __('Having Fun?');
echo __('Having Fun? %s','Ya');
echo __('Having Fun? %s %d','Ya',date('h'));

This function is available in .phtml files, block files, controllers, helpers. So basically it available everywhere except model files.

We can also use translation in our configuration files
e.g

<checkout translate="label" module="checkout">
            <label>Checkout</label>

This says to use the translate file of checkout module to translate the label “Checkout”

Digging Deep

Lets now look at the code level on how magento implements translations.

Files to Look at:
1. Mage_Core_Model_Translate
2. Mage_Core_Model_Locale

Lets first look at the definition of the function “__()”

    public function __()
    {
        $args = func_get_args();
        $expr = new Mage_Core_Model_Translate_Expr(array_shift($args), $this->getModuleName());
        array_unshift($args, $expr);
        return $this->_getApp()->getTranslator()->translate($args);
    }

This function definition is located in files
“Mage_Core_Block_Abstract”,”Mage_Adminhtml_Controller_Action”, “Mage_Core_Controller_Front_Action”, “Mage_Core_Helper_Abstract”
This function ultimately calls the “translate()” function in “Mage_Core_Model_Translate”

There is an “init()” function the class “Mage_Core_Model_Translate” which initializes all translations, so lets look at that function in detail. The function is divided into 3 parts “_loadModuleTranslation” , “_loadThemeTranslation”, “_loadDbTranslation”. The purpose of each method is evident from its name.

1. _loadModuleTranslation():

        foreach ($this->getModulesConfig() as $moduleName=>$info) {
            $info = $info->asArray();
            $this->_loadModuleTranslation($moduleName, $info['files'], $forceReload);
        }
        protected function _loadModuleTranslation($moduleName, $files, $forceReload=false)
        {
             foreach ($files as $file) {
                 $file = $this->_getModuleFilePath($moduleName, $file);
                 $this->_addData($this->_getFileData($file), $moduleName, $forceReload);
             }
             return $this;
        }
        protected function _getModuleFilePath($module, $fileName)
        {
            $file = Mage::getBaseDir('locale');
            $file.= DS.$this->getLocale().DS.$fileName;
            return $file;
         }

The above function reads configuration information for each module from “frontend/translate/module” or “adminhtml/translate/module” depending on the area. Loads the file from the ‘locale’ folder and from the current locale set for the store, and add the translation using “_addData()” function.

TIP: It can be seen here that for each module we can specify multiple files in the “files” tag in config.xml

2. _loadThemeTranslation():

    protected function _loadThemeTranslation($forceReload = false)
    {
        $file = Mage::getDesign()->getLocaleFileName('translate.csv');
        $this->_addData($this->_getFileData($file), false, $forceReload);
        return $this;
    }

In this we can see that translate.csv file is loaded for the current theme.

3. _loadDbTranslation():

    protected function _loadDbTranslation($forceReload = false)
    {
        $arr = $this->getResource()->getTranslationArray(null, $this->getLocale());
        $this->_addData($arr, $this->getConfig(self::CONFIG_KEY_STORE), $forceReload);
        return $this;
    }

In this Inline Translations are loaded from database table “core_translate” and then added to the object using _addData() function.

Next lets look at the “_addData()” function

    protected function _addData($data, $scope, $forceReload=false)
    {
        foreach ($data as $key => $value) {
            if ($key === $value) {
                continue;
            }
            $key    = $this->_prepareDataString($key);
            $value  = $this->_prepareDataString($value);
            if ($scope && isset($this->_dataScope[$key]) && !$forceReload ) {
                /**
                 * Checking previos value
                 */
                $scopeKey = $this->_dataScope[$key] . self::SCOPE_SEPARATOR . $key;
                if (!isset($this->_data[$scopeKey])) {
                    if (isset($this->_data[$key])) {
                        $this->_data[$scopeKey] = $this->_data[$key];
                        /**
                         * Not allow use translation not related to module
                         */
                        if (Mage::getIsDeveloperMode()) {
                            unset($this->_data[$key]);
                        }
                    }
                }
                $scopeKey = $scope . self::SCOPE_SEPARATOR . $key;
                $this->_data[$scopeKey] = $value;
            }
            else {
                $this->_data[$key]     = $value;
                $this->_dataScope[$key]= $scope;
            }
        }
        return $this;
    }

The add data function store data in two arrays “_data” and “_dataScope”. $scope is an important variable in this function
$scope => “frontend”, “adminhtml” in case of module translate files
$scope => false in case of theme translate files
$scope => store key in case of database translate files

Its important to see how data is stored in the “$this->_data” array. To understand lets assume an translate key,value pair
“Add To Cart”,”Add To Basket”
So first this is stored as
$this->_data[‘Add To Cart’] = ‘Add To Basket';

But if this same translate key is found in another module file then in that case
$this->_data[‘module1::Add To Cart’] = ‘Add To Basket';
$this->_data[‘module2::Add To Cart’] = ‘Add To Basket Module2′;
$this->_data[‘Add To Cart’] = ‘Add To Basket Module';

and if developer module is turned on non module translate is removed which means, $this->_data[‘Add To Cart’] is removed.

The final important function which is actually used to translate the string is

    protected function _getTranslatedString($text, $code)
    {
        $translated = '';
        if (array_key_exists($code, $this->getData())) {
            $translated = $this->_data[$code];
        }
        elseif (array_key_exists($text, $this->getData())) {
            $translated = $this->_data[$text];
        }
        else {
            $translated = $text;
        }
        return $translated;
    }

As its easy to see here, first translate string is search according to code (code here is module::text). If code is not found then it searches according to text.

Magento Locale Class

There is another important class which we need to look at “Mage_Core_Model_Locale”. This is class which is used to get all locale based information like language codes, timezone information, currency etc and it gets its data from “Zend_Locale” class.

“Mage_Core_Model_Locale_Config” this class contains all the locale and currency you see in admin system configuration.

There are two important function in this “emulate” and “revert”

    public function emulate($storeId)
    {
        if ($storeId) {
            $this->_emulatedLocales[] = clone $this->getLocale();
            $this->_locale = new Zend_Locale(Mage::getStoreConfig(self::XML_PATH_DEFAULT_LOCALE, $storeId));
            $this->_localeCode = $this->_locale->toString();
            Mage::getSingleton('core/translate')->setLocale($this->_locale)->init('frontend', true);
        }
        else {
            $this->_emulatedLocales[] = false;
        }
    }

In this function we can pass a store and it will load all translate information for that particular locale. As we can see it reinitialize the entire ‘core/translate’ object with new translate information as per store locale.

    public function revert()
    {
        if ($locale = array_pop($this->_emulatedLocales)) {
            $this->_locale = $locale;
            $this->_localeCode = $this->_locale->toString();
            Mage::getSingleton('core/translate')->setLocale($this->_locale)->init('adminhtml', true);
        }
    }

The revert function undo’s the effect of emulate. It loads the previous locale and its translation.
But it can be seen here that “emulate()” function loads the ‘frontend’ area while revert loads the ‘adminhtml’ area.
Not sure if this a bug by magento or this is how it was intended because i was not able to figure out the reason for this.

Magento Events and Observers

In this post we will see how magento manages events and observers.

Basics

Magento has a very powerful event/observer pattern which we can use very effectively in our custom modules. The basic principal of event/observer pattern is, certain events are fired/dispatched by magento through out its code base. These events have specific class objects attached as function parameters. Observers (class functions) can be setup which run when these events are fired and receive the class objects. The observer can manipulate, extract data from these class object and do additional processing required. So this help us to extend magento functionality without changing the magento core. So in practice it works like this, there are various events dispatched throughout magento using

Mage::dispatchEvent('controller_action_predispatch', array('controller_action' => $this)); 

Here ‘controller_action_predispatch’ is the name of the event and $this is class object which is passes as a parameter.

Next we define observer for this events in our module config.xml file e.g

<global>
        <events>
            <log_log_clean_after> <!-- event name -->
                <observers>
                    <catalog_product_compare_item>  <!-- unique event code -->
                        <class>catalog/observer</class>
                        <method>catalogProductCompareClean</method>
                    </catalog_product_compare_item>
                </observers>
            </log_log_clean_after>
        </events>
</global>

This way we can specify which observer method to fire for any event.

Magento Event Processing

Files To Refer
app/code/core/Mage/Core/Model/App.php

Entire core functionality for magento event handling is written in class “Mage_Core_Model_App” and function “dispatchEvent()”

So the way this function works, when a event gets dispatched from magento it first checks in the configuration object if any observers have been added for that event. If observers are found, it appends them all into an array.

Finally each observer is called serially one after the other. The code event/observer is quite simple in magento and can easily be understood looking at this function.

This how events are defined in config.xml file in any module

    <global>
        <events>
            <customer_login> <!-- Actual Event Name -->
                <observers>
                    <catalog> <!-- Unique Name For Each Observer -->
                        <type>model</type>
                        <class>catalog/product_compare_item</class>
                        <method>bindCustomerLogin</method>
                    </catalog>
                </observers>
            </customer_login>
</events>
    </global>

The first tag “global” is the event area. There are 3 possible event areas, “global”, “frontend” , “admin”.
The “type” tag in event has various option
1. disabled : observer method won’t run
2. object: this will create a singleton object of the “class” mentioned and fire the method
3. model: this will create a new object of the “class” and fire the method

Event Observer Arguments

The “class” which you defined as the event observer gets “Varien_Event_Observer” as a parameter in the method. It exposes the following methods

public function bindCustomerLogin(Varien_Event_Observer $event){

      $event->getEvent()->getObject1();  //the various class object can be accessed like this
      $event->getEvent()->getObject2();
}

Troubleshooting Events

If event created by you is not firing you can debug it at “Mage_Core_Model_App” class “dispatchEvent()” function.
First check the area of your event (global, frontend, adminhtml) and see if it loads in the dispatchEvent() function.
Next in this line

foreach ($eventConfig->observers->children() as $obsName => $obsConfig) {

So if your event shows up, if not then there is some problem with the config.xml for event.
Next check the

$this->_callObserverMethod($object, $method, $observer);

this function throws an exception but only when developer module is turned on. So you check here if an exception is thrown. By checking these 3 area’s you should come to know why your event is not firing.

Magento Cron System

In this blog post we will see in detail how magento cron system works.

Basics

Magento Cron System is used to run tasks on recurring basis as with any cron based system. “cron.php” located in magento base folder is the main entry for magento cron system. To start magento cron you need to put “http://domainname.com/cron.php” in your web server cron job and set it to run every 15min.

There are various default cron job setup already in magento, so its recommend that you setup cron in magento. The default cron jobs in magento are related to log cleaning, newsletter, product stock notifications, currency updates.

Adding Cron To Your Custom Module

To add cron job to your custom module add below xml to the config.xml

  <default>
    <crontab>
        <jobs>
            <job_unique_name>
                <schedule>
                    <cron_expr>0 2 * * *</cron_expr>
                </schedule>
                <run>
                    <model>test/observer::cronMethod</model>
                </run>
            </job_unique_name>
        </jobs>
    </crontab>
  </default>

“job_unique_name” can be any unique name which you want to give.
“cron_expr” can take any valid cron expression
“test/observer” should be a valid model in your module
“cronMethod” is a function inside that model

TIP: You can also specify “always” in cron_expr, to execute cron job always.

Magento also uses a table “cron_schedule” where is stores the status of all cron jobs. Details stored are job_code, scheduled_at, executed_at, finished_at, status. This is also a useful table which you can use debug your cron jobs.

There are various settings options for cron in admin which you can configure located at “Sytem -> Configuration -> Advanced -> System -> Cron”. These settings are

Generate Schedules Every: After how many minutes should we generate a new cron entry in database
Schedule Ahead for: How many minutes ahead should the cron entry be done in database.
Missed if Not Run Within: Mark status as missed if cron has not run with these many minutes
History Cleanup Every: Delete cron entry after how many minutes
Success History Lifetime: After how many minutes to delete a cron entry with status success
Failure History Lifetime: After how many minutes to delete a cron entry with status failure

TIP: Magento has a very good design relating to cron by having a single entry point for various cron functions. The major drawback for this is, if any of the cron jobs have an error all other cron jobs will also not execute since all are executed serially.

Magento Cron Job In Detail

Now lets dig into magento source code and see how cron is implemented.

Files to Refer
1. “cron.php”
2. “Mage_Cron_Model_Observer”
3. “app/code/core/Mage/Cron/etc/config.xml”

The main entry point from cron is “cron.php”
There we see

Mage::getConfig()->init()->loadEventObservers('crontab');
Mage::app()->addEventArea('crontab');
Mage::dispatchEvent('always');
Mage::dispatchEvent('default');

So magento first loads all event observers in the crontab area, and then dispatches “always” and “default” event.
Both these events are defined in “app/code/core/Mage/Cron/etc/config.xml” file

<crontab>
        <events>
            <default>
                <observers>
                    <cron_observer>
                        <class>cron/observer</class>
                        <method>dispatch</method>
                    </cron_observer>
                </observers>
            </default>
            <always>
                <observers>
                    <cron_observer>
                        <class>cron/observer</class>
                        <method>dispatchAlways</method>
                    </cron_observer>
                </observers>
            </always>
        </events>
    </crontab>

Lets look at the “dispatch()” function “Mage_Cron_Model_Observer” class. As we can see here it reads jobs from
‘crontab/jobs’, ‘default/crontab/jobs’ xpaths and processes these jobs.

Magento URL Rewrites and Controller Overriding

In this blog post we will see in detail how magento implements url rewriting and controller overriding

Controller Overriding

In the previous blog post we saw an important function in “Mage_Core_Controller_Varien_Front” class

$this->_getRequestRewriteController()->rewrite();

Lets look at this function in detail now. The “_getRequestRewriteController()” returns a class object as below

$className = (string)Mage::getConfig()->getNode('global/request_rewrite/model');
        return Mage::getSingleton('core/factory')->getModel($className, array(
            'routers' => $this->getRouters(),
        ));

Since the $className comes from xpath ‘global/request_rewrite/model’ it can also be a custom class (but that is a separate discussion). By default is defined in “Mage/Core/etc/config.xml”

<request_rewrite>
            <model>core/url_rewrite_request</model>
        </request_rewrite>

So this functions return an object of class “core/url_rewrite_request”. If you open this file you see the “rewrite()” function defined there. The “rewrite()” function has two important functions “$this->_rewriteDb();” and “$this->_rewriteConfig();”

_rewriteDB()

The _rewriteDb() function check if any url rewrite rules have been setup in (Admin -> Catalog -> URL Management) section. If applied it redirects to the new url.
Lets see in detail how the rewriteDB works. The function first generates different URL’s to compare in database.
It generated maximum there request cases
1. request with slash and query string
2. request without slash with query string
3. request without slash and without query string
so if URL is www.domain.com/test/index/index?abc=123

the request cases generated would be
1. test/index/index/?abc=123
2. test/index/index?abc=123
3. test/index/index

Next magento checks the table ‘core_rewrite_url’ with these request paths, if any of them matches it redirects to the target_path set in database.

_rewriteConfig()

The _rewriteConfig() function check for any url rewrite written in config.xml file. It looks for “global/rewrite” xpath and then check all “from” and “to”.

<rewrite>
        <fancy_url>
                  <from><![CDATA[catalog/index/index]]></from>
                  <to><![CDATA[manufacturer/index/index/manufacturer/$1/]]></to>
                      <complete>1</complete>
            </fancy_url>
        </rewrite>

More details about config.xml can be read here

The code written in the rewriteConfig is

protected function _rewriteConfig()
    {
        $config = $this->_config->getNode('global/rewrite');
        if (!$config) {
            return false;
        }
        foreach ($config->children() as $rewrite) {
            $from = (string)$rewrite->from;
            $to = (string)$rewrite->to;
            if (empty($from) || empty($to)) {
                continue;
            }
            $from = $this->_processRewriteUrl($from);
            $to   = $this->_processRewriteUrl($to);

            $pathInfo = preg_replace($from, $to, $this->_request->getPathInfo());
            if (isset($rewrite->complete)) {
                $this->_request->setPathInfo($pathInfo);
            } else {
                $this->_request->rewritePathInfo($pathInfo);
            }
        }
        return true;
    }

Important Note: This uses ‘preg_replace’ function. So its possible to define regular expressions as well in our config.xml to rewrite controllers.

Action Overriding

In the preDispatch() function of “Mage_Core_Controller_Varien_Action” class, which is extended by all controller we have “$this->_rewrite()” function.

The rewrite() function looks for config node

$rewrite = Mage::getConfig()->getNode('global/routers/'.$route.'/rewrite/'.$controller);

So in config.xml we can define if want to rewrite a particular action of a controller as well.

    <!-- this will rewrite all actions of a controller -->
    <global>
        <routers>
            <customer>
                <rewrite>
                    <account>
                        <to>test/index/index</to>
                    </account>
                </rewrite>
            </customer>
        </routers>
     </global>

or

     <!-- this will rewrite only single actions of a controller -->
     <global>
        <routers>
            <customer>
                <rewrite>
                    <account>
                            <login>
                                <to>test/index/index</to>
                            </login>
                        </actions>
                    </account>
                </rewrite>
            </customer>
        </routers>
     </global>

So these are the three ways of rewriting URL in magento which we have seen in detail.

Magento Controller Initialization and Flow

In this blog post we are going to see in detail how magento loads controller, action based on URL
Files to Refer
1. Mage/Core/Model/App.php
2. Mage/Core/Controller/Varien/Front.php
3. Mage/Core/Controller/Varien/Router/Standard.php

As we have seen before, the “run()” function in Mage_Core_Model_App class is where the control comes when we open any URL.
Here we see the line

$this->getFrontController()->dispatch();

which is responsible for loading the controller and calling the action.

The function “getFrontController()” create a object for class Mage_Core_Controller_Varien_Front and calls the “init()” method.

Lets see working of “init()” function in detail

First important line in “init()” function is

$routersInfo = Mage::app()->getStore()->getConfig(self::XML_STORE_ROUTERS_PATH);

This basically returns

Array
(
    [admin] => Array
        (
            [area] => admin
            [class] => Mage_Core_Controller_Varien_Router_Admin
        )

    [standard] => Array
        (
            [area] => frontend
            [class] => Mage_Core_Controller_Varien_Router_Standard
        )

)

This data is defined in “app/code/core/Mage/Core/etc/config.xml” file
So this returns two router classes for admin and frontend area.
Next we see

$router = new $routerInfo['class'];
                if (isset($routerInfo['area'])) {
                    $router->collectRoutes($routerInfo['area'], $routerCode);
                }
                $this->addRouter($routerCode, $router);

This code creates a new object of each of router class and calls the “collectRoutes()” function in it.
So lets open the class “Mage_Core_Controller_Varien_Router_Standard” and see the “collectRoutes()” function.
The first line here is to create the xpath “frontend/routers/args/module”
This is the same routes we define in our module config.xml files

<routers>
            <checkout>
                <use>standard</use>
                <args>
                    <module>Mage_Checkout</module>
                    <frontName>checkout</frontName>
                </args>
            </checkout>
        </routers>

There is also a check for “routerName”

$use = (string)$routerConfig->use;
if ($use == $useRouterName) {

here the “use” tag in xml is checked to standard or admin.

Finally this function is called

$this->addModule($frontName, $modules, $routerName);

which adds the route and module to an array for further use.

Now lets go back to our original function

$this->getFrontController()->dispatch();

Above we have seen the first part i.e “getFrontController()” now lets look at the “dispatch()” function.

If you open the class “Mage_Core_Controller_Varien_Front” class, you will see the “dispatch()” function there.

The first important function here is

$this->_checkBaseUrl($request);

this checks if the current opened in the browser is same as the base url set in system configuration or not.

Next important line we have

$this->_getRequestRewriteController()->rewrite();

We will look this line in detail a little later, but this line mainly check for any URL rewriting we might have done at database level (Admin -> Catalog -> URL Management) or any URL rewrites done in config.xml

Next for each router we have “match()” function getting called.
If you look at “match()” function in Mage_Core_Controller_Varien_Router_Standard. Lets see in detail how the match() function works

Here the url is broken upto 3 parts; modules, controller, action.

        $path = trim($request->getPathInfo(), '/');

        if ($path) {
            $p = explode('/', $path);
        } else {
            $p = explode('/', $this->_getDefaultPath());
        }

Next we have

$module = $p[0];
$modules = $this->getModuleByFrontName($module);

above we see based on the first url part, module name is determined.
Next

$controller = $p[1];
$action = !empty($p[2]) ? $p[2] : $front->getDefault('action');

So above controller and action are set.
Next

$this->_checkShouldBeSecure($request, '/'.$module.'/'.$controller.'/'.$action);

This if the URL should be secure (https) or not. If it should be it redirects to https url.
Next

$controllerClassName = $this->_validateControllerClassName($realModule, $controller);

This function generates the controller file path (full directory path) using module name and controller name. Check if the file exists and class is already loaded or not. If not loading “include()” the file.

After this the action function is checked, if its present in the controller or not. Finally all parameters are set into the request object using “setParam()” function and then “dispatch()” function is called in the controller.

We know that every controller in magento extension the Mage_Core_Controller_Front_Action class which in turn extensions the Mage_Core_Controller_Varien_Action class. In the “Mage_Core_Controller_Varien_Action” class you will find the “dispatch()” method. Lets see the “dispatch()” function in detail

$actionMethodName = $this->getActionMethodName($action);
if (!method_exists($this, $actionMethodName)) {
  $actionMethodName = 'norouteAction';
}
$this->preDispatch();
$this->$actionMethodName();
$this->postDispatch();

Above is the entire life cycle of magento calling action of a controller when we open a URL in the browser