ALL countries in checkout, although allowed_countries set correctly

We had the problem, that we have a short list of countries we ship to:

Country list configuration in the backend shows a filtered list

But the country list for the invoice and shipping address in the checkout is complete.

Country select in the checkout shows all countries to choose from

Config

The configuration is fine as you can see above. I checked the cache, because I thought it might be wrong - but the config cache too was fine.

Cache

Digging into the issue shows, that more caches are involved:

\Mage_Checkout_Block_Onepage_Abstract::getCountryOptions
public function getCountryOptions()
{
    $options    = false;
    $useCache   = Mage::app()->useCache('config');
    if ($useCache) {
        $cacheId    = 'DIRECTORY_COUNTRY_SELECT_STORE_' . Mage::app()->getStore()->getCode();
        $cacheTags  = array('config');
        if ($optionsCache = Mage::app()->loadCache($cacheId)) {
            $options = unserialize($optionsCache);
        }
    }

    if ($options == false) {
        $options = $this->getCountryCollection()->toOptionArray();
        if ($useCache) {
            Mage::app()->saveCache(serialize($options), $cacheId, $cacheTags);
        }
    }
    return $options;
}

So I checked the cache entries for DIRECTORY_COUNTRY_SELECT_STORE_DE and found a complete list of countries. So the assumption is: Something "poisons" the cache.

So, what is writing into this cache and maybe we have a broken extension - we don't.

\Mage_XmlConnect_Model_Simplexml_Form_Element_CountryListSelect::_getCountryOptions
\Mage_XmlConnect_Block_Checkout_Address_Form::_getCountryOptions
\Mage_Directory_Block_Data::getCountryHtmlSelect
\Mage_Checkout_Block_Onepage_Abstract::getCountryOptions
\Mage_Catalog_Model_Product_Attribute_Source_Countryofmanufacture::getAllOptions

If we look further into it, we can assume the problem is not from here:

\Mage_XmlConnect_Block_Checkout_Address_Form::_getCountryOptions
\Mage_XmlConnect_Model_Simplexml_Form_Element_CountryListSelect::_getCountryOptions
\Mage_Checkout_Block_Onepage_Abstract::getCountryOptions
\Mage_Directory_Block_Data::getCountryHtmlSelect

because all of them filter the country list via Mage::getModel('directory/country')->getResourceCollection()->loadByStore();, therefore the filter of the config is applied.

\Mage_Catalog_Model_Product_Attribute_Source_Countryofmanufacture

The source model doesn't do correct filtering.

How is the cache filled with wrong data?

This is the tricky part. The only place were I know the source model is used is the backend. But Mage::app()->getStore()->getCode() is always admin. Even when you edit data for another store view.

After digging a while I found out, that when I place a configurable product into the cart one of our extensions reads the country of manufacturer in the frontend and poisons the cache.

I fixed the problem with a rewrite:

class Project_Bugfix_Model_Catalog_Product_Attribute_Source_Countryofmanufacture
    extends Mage_Catalog_Model_Product_Attribute_Source_Countryofmanufacture
{
    /**
     * Get list of all available countries
     *
     * @return mixed
     */
    public function getAllOptions()
    {
        $cacheKey = 'DIRECTORY_COUNTRY_SELECT_STORE_' . Mage::app()->getStore()->getCode();
        if (Mage::app()->useCache('config') && $cache = Mage::app()->loadCache($cacheKey)) {
            $options = unserialize($cache);
        } else {
            if (Mage::app()->getStore()->isAdmin()) {
                $collection = Mage::getModel('directory/country')->getResourceCollection();
            } else {
                $collection = Mage::getModel('directory/country')->getResourceCollection()->loadByStore();
            }
            $options = $collection->toOptionArray();

            if (Mage::app()->useCache('config')) {
                Mage::app()->saveCache(serialize($options), $cacheKey, ['config']);
            }
        }
        return $options;
    }   
}

Custom Product Attributes, Source Models and Flat Tables

How to add an attribute to product_flat_table

Either one of the following attribute settings is true:

  • backend_typ = 'static'
  • is_filterable > 0
  • used_in_product_listing = 1
  • is_used_for_promo_rules= 1
  • used_for_sort_by = 1

Or you add it manually by observing this event:
catalog_product_flat_prepare_columns

Attribute is not added - source models

If you attribute is still not added it might be, because \Mage_Eav_Model_Entity_Attribute_Abstract::getFlatColumns is called on the attribute, which checks wether a source model exists.

public function getFlatColumns()
{
    // If source model exists - get definition from it
    if ($this->usesSource() && $this->getBackendType() != self::TYPE_STATIC) {
        return $this->getSource()->getFlatColums();
    }
    // ...
}

As you can see, getFlatColums is called on your source model, which default implementation is:

// \Mage_Eav_Model_Entity_Attribute_Source_Abstract::getFlatColums
public function getFlatColums()
{
    return array();
}

Therefore your attribute is not added.

And implementation like this might help:

public function getFlatColums()
{
    $attributeCode = $this->getAttribute()->getAttributeCode();
    $column        = [
        'unsigned' => false,
        'default'  => null,
        'extra'    => null,
    ];

    if (Mage::helper('core')->useDbCompatibleMode()) {
        $column['type']    = 'varchar';
        $column['is_null'] = true;
    } else {
        $column['type']     = Varien_Db_Ddl_Table::TYPE_VARCHAR;
        $column['nullable'] = true;
        $column['comment']  = 'Seals column';
    }

    return [$attributeCode => $column];
}

*Update*

How to fill the field

After the final win that the column is created on the flat tables, it was NULL after reindexing. Something is missing and this is:

\Mage_Eav_Model_Entity_Attribute_Source_Abstract::getFlatUpdateSelect

This method has to return a Zend_Db_Select due to this:

\Mage_Catalog_Model_Resource_Product_Flat_Indexer::updateAttribute
// ...
$select = $attribute->getFlatUpdateSelect($storeId);
if ($select instanceof Varien_Db_Select) {
    if ($productIds !== null) {
        $select->where('e.entity_id IN(?)', $productIds);
    }
}
// ...

An example implementation which works for me:

public function getFlatUpdateSelect($store)
{
    return Mage::getResourceSingleton('eav/entity_attribute')
        ->getFlatUpdateSelect($this->getAttribute(), $store);
}

Magento 1 Update - How to

Rico Neitzel told me back in the days, that during Magento updates files gets deleted, therefore it is a bad idea, therefore you want to run a patch (not the ones from Magento, but a patch between the versions) over your Magento installation.

After fiddeling around for a while it is a good idea to document, what I have just done (I updated from 1.9.2.2 to 1.9.3.3):

  1. Get a trusted git repository with all Magento versions you need, I can recommend https://github.com/OpenMage/magento-mirror
  2. Clone it

    git clone https://github.com/OpenMage/magento-mirror.git
    
  3. Make sure your current installation is free of core hacks and all files exists which might get patched

    magento-mirror$ git checkout 1.9.2.2
    production$  rsync --progress -v -r ../magento-mirror/* .
    
  4. You might have now too much files - either you have them already in your gitignore - if not, time to add them.

  5. Create a patch file (diff-index is needed for binary patches)

    magento-mirror$ git checkout 1.9.3.3
    magento-mirror$ git diff-index 92a1142a3 --binary > patch.patch
    
  6. Apply the patch

    production$ git apply --binary ../magento-mirror/patch.patch
    
  7. Now you have a working copy with all the changes. You can review them, commit them, etc.

array_diff(array, string) does(n't) work as expected

This is an old problem, quite a few security researchers already wrote about. But repeating things, helps learning them (especially me).

In MageSetup we read posted agreements (terms and conditions) and compare them with all the agreements we expect.

This looks like that:

$requiredAgreements = $this->_getCustomerCreateAgreements();
$controller = $observer->getEvent()->getControllerAction();
$postedAgreements = array_keys($controller->getRequest()->getPost('agreement', array()));

if ($diff = array_diff($requiredAgreements, $postedAgreements)) {  
    // ERROR
}

Today I discovered a nasty bug: if you submit no array for agreement, you end up with:

array_keys(1);
array_diff([1,2,3,4], null);

Which throws two warnings:

Warning: arraykeys() expects parameter 1 to be array, string given Warning: arraydiff(): Argument #2 is not an array

If developer mode is on, the warning is transformed into an exception and thrown, which "solves" the problem. But if dev mode is off, array_keys returns null first, and then array_diff() does the same.

So we don't have the expected check against our $requiredAgreements.

Please make sure, you are handling arrays, before passing them into array_* functions!

Magento core_config_data not loaded in backend

Guest Author: Rico Neitzel, http://buro71a.de/

I sometimes have the following strange behaviour in Magento:

Config not loaded in Backend

  1. I do have valid config entries in core_config_data Table
  2. The backend in System -> Config doesn't reflect these config values from the table

On further investigation I saw that this usually only applies to the General Tab and the General Section.

My finding was astonishing! There had been a core_config_data Entry with the following data:

ID scope scope_id path value
123 default 0 general NULL

How did that happen?

I'm pretty sure that I accidentally clicked the "Add new row" Button in my SQL-Tool. That prefilled the fields with its (in SQL defined) default values. The main issue here is the default value for the path field: general

What did it do?

When Magento loads the configuration and applies the database config values this NULL value for general will overwrite all general/* settings and so the backend cannot find anything although it's stored in the DB.

How to solve it?

Dead simple: Remove that broken general entry from the core_config_data table and you're fine :-)

Disabling block cache

You want to disable block cache?

You can do it in layout.xml, cms block/page layout update, etc.

<action method="setCacheLifetime"></action>

More about this in Fabrizio Branca's blog

Thanks to sr_aromicon for the link!