Import from cli/cronjob

Back in the days Vinai Kopp gave a talk about ImportExport module.

In the talk is a code snippet which helps to run importExport from CLI or as a cronjob. Unfortunately it is a screenshot and I typed the snippet already a couple of times. Time to make sure it is copy pastable:

<?php

require 'html/app/Mage.php';

Mage::app();

/** @var $import Mage_ImportExport_Model_Import */
$import = Mage::getModel('importexport/import');
$import->setEntity(Mage_Catalog_Model_Product::ENTITY);
$file = './import.csv';

if (!$file || !file_exists($file)) {
    echo 'File does not exist.';
    die();
}

$validationResult = $import->validateSource($file);

if ($import->getProcessedRowsCount() <= 0 || !$validationResult) {
    printf(
        'File %s contains %s corrupt records (from a total of %s)',
        $file,
        $import->getInvalidRowsCount(),
        $import->getProcessedRowsCount(),
        );
    foreach ($import->getErrors() as $type => $lines) {
        $lines = implode(', ', $lines);
        printf("\n:::: $type :::: \nIn Line(s) $lines \n");
    }
    die();
}

$import->importSource();
$import->invalidateIndex();

Grid Loader doesn't hide after request finished

I had today the problem, that the loader doesn't hide after a a ajax response is loaded:

My problem is the following error:

"TypeError: Cannot read property 'down' of null
    at Object.bindFieldsChange (http://demo.dev/js/mage/adminhtml/grid.js:280:63)
    at Object.initGrid (http://demo.dev/js/mage/adminhtml/grid.js:80:14)
    at Object.initGridAjax (http://demo.dev/js/mage/adminhtml/grid.js:93:14)
    at http://demo.dev/js/prototype/prototype.js:391:23
    at klass.respondToReadyState (http://demo.dev/js/mage/adminhtml/loader.js:95:68)
    at klass.onStateChange (http://demo.dev/js/prototype/prototype.js:1545:12)
    at XMLHttpRequest.<anonymous> (http://demo.dev/js/prototype/prototype.js:391:23)"

We can't see the error, because magento is catching it and doesn't show it to us.

For debugging purposes we can change the js/mage/adminhtml/grid.js to add this:

    // line 174
    onException: function (response, exception) {
        console.log(response);
        console.log(exception);
    },

so it looks afterwards like this:

reload : function(url){
    if (!this.reloadParams) {
        this.reloadParams = {form_key: FORM_KEY};
    }
    else {
        this.reloadParams.form_key = FORM_KEY;
    }
    url = url || this.url;
    if(this.useAjax){
        new Ajax.Request(url + (url.match(new RegExp('\\?')) ? '&ajax=true' : '?ajax=true' ), {
            loaderArea: this.containerId,
            parameters: this.reloadParams || {},
            evalScripts: true,
            onException: function (a,b,c,d,e) {
                console.log(e);
            },

Solution

In my case the problem is a missing ID on the grid

class My_Module_Block_Adminhtml_Tab_Grid
    extends Mage_Adminhtml_Block_Widget_Grid
    implements Mage_Adminhtml_Block_Widget_Tab_Interface
{
    public function __construct()
    {
        parent::__construct();
        $this->setId('transactionMailFilesGrid'); // <-- this was missing

Tab is rendered in left block instead of content

I want to add tabs to a custom form in the backend.

There are a couple of great tutorials how to add tabs, like from Inchoo or Erfan Imani.

But I wanted this:

And got this:

The problem is simple - if you know it (costs me an hour and a few hairs):

The tabs are rendered with JS. So they live originally in left, but they are shown in content, to be more specific in the node, defined by:

class My_Module_Block_Adminhtml_Project_Edit_Tabs extends Mage_Adminhtml_Block_Widget_Tabs {

    public function __construct()
    { 
        // ...
        $this->setDestElementId('project_form');
    }
}

And this id needs to be the same as in

class My_Module_Block_Adminhtml_Project_Edit_Form extends Mage_Adminhtml_Block_Widget_Form
{
    protected function _prepareForm()
    {
        $form = new Varien_Data_Form(
            [
                'id'     => 'project_form',
            ]);
    }
}

Hope this helps anyone.

Magento doesn't trigger setup script

I found a nice new, rarely trigger Magento bug.

In one of our projects, we have an example file to create new products:

data-upgrade-example-new-product-import.php

Beside this we had data scripts:

data-install-1.0.1.php
data-upgrade-1.0.0-1.0.1.php
data-upgrade-1.0.5-1.0.6.php

Unfortunately data-upgrade-1.0.5-1.0.6.php didn't fire.

It took me a while, but I found the problem:

// \Mage_Core_Model_Resource_Setup::_getModifySqlFiles
protected function _getModifySqlFiles($actionType, $fromVersion, $toVersion, $arrFiles)
{
    $arrRes = array();
    switch ($actionType) {

        // ... 

        case self::TYPE_DB_UPGRADE:
        case self::TYPE_DATA_UPGRADE:
            uksort($arrFiles, 'version_compare');
            foreach ($arrFiles as $version => $file) {
                $versionInfo = explode('-', $version);

                // In array must be 2 elements: 0 => version from, 1 => version to
                if (count($versionInfo)!=2) {
                    break;
                }

in $arrFiles we find an array with all files in the data dir which match a certain regex in \Mage_Core_Model_Resource_Setup::_getAvailableDataFiles. In short, when the files starts with data-.

The problem is, that data-upgrade-example-new-product-import.php doesn't meet the if (count($versionInfo)!=2) check and then break is called, which kills the complete loop, but should only be continue;.

So either we hack the core or rename the data-upgrade-example-new-product-import.php, I decided for renaming.

Empty order emails (no subject, no body)

One of my customers had a problem this week:

Emails looked like this:
empty email, no body, no subject

It was literally empty. The server filled a few head fields, but there was nothing left of the content we want to send (new order email).

It took a while, but finally I think I found the problem (unfortunatelly not sure about the cause).

Magento saves stuff while going down this trace:

\Mage_Core_Model_Abstract::save
\Mage_Core_Model_Resource_Db_Abstract::save
\Mage_Core_Model_Resource_Db_Abstract::_prepareDataForSave
\Mage_Core_Model_Resource_Abstract::_prepareDataForTable
\Varien_Db_Adapter_Pdo_Mysql::describeTable

This method should return an array like this:

...
[created_at] => Array
        (
            [SCHEMA_NAME] =>
            [TABLE_NAME] => core_email_queue
            [COLUMN_NAME] => created_at
            [COLUMN_POSITION] => 8
            [DATA_TYPE] => timestamp
            [DEFAULT] =>
            [NULLABLE] => 1
            [LENGTH] =>
            [SCALE] =>
            [PRECISION] =>
            [UNSIGNED] =>
            [PRIMARY] =>
            [PRIMARY_POSITION] =>
            [IDENTITY] =>
        )

    [processed_at] => Array
        (
            [SCHEMA_NAME] =>
            [TABLE_NAME] => core_email_queue
            [COLUMN_NAME] => processed_at
            [COLUMN_POSITION] => 9
            [DATA_TYPE] => timestamp
            [DEFAULT] =>
            [NULLABLE] => 1
            [LENGTH] =>
            [SCALE] =>
            [PRECISION] =>
            [UNSIGNED] =>
            [PRIMARY] =>
            [PRIMARY_POSITION] =>
            [IDENTITY] =>
        )
...

But it returned

[data] => a:9:{s:10:"message_id";a:14:{s:11:"SCHEMA_NAME";N;s:10:"TABLE_NAME";s:16:"core_email_queue";s:11:"COLUMN_NAME";s:10:"message_id";s:15:"COLUMN_POSITION";i:1;s:9:"DATA_TYPE";s:3:"int";s:7:"DEFAULT";N;s:8:"NULLABLE";b:0;s:6:"LENGTH";N;s:5:"SCALE";N;s:9:"PRECISION";N;s:8:"UNSIGNED";b:1;s:7:"PRIMARY";b:1;s:16:"PRIMARY_POSITION";i:1;s:8:"IDENTITY";b:1;}s:9:"entity_id";a:14...

As you might guess, this is a serialized array.

When we have a look into \Varien_Db_Adapter_Pdo_Mysql::loadDdlCache we see, that the schema is cached, but I think there is no way, the serialized data are returned:

// lib/Varien/Db/Adapter/Pdo/Mysql.php:1548
$data = $this->_cacheAdapter->load($cacheId);
if ($data !== false) {
    $data = unserialize($data);

To be continued...