SSL Everywhere or HSTS

We have to secure all the data of our users, not only registration, checkout and login. We need to secure the session data too.

SSL Everywhere or HTTP Strict Transport Security

(I hope) Everyone knows, that it is important to secure (read as encrypt) our customer’s data. Because of all the evil hackers in the world and the bad ISPs which intercept all our data.


Use HTTP Strict Transport Security!

The first problem: unencrypted personal data

But the real problems are the all-day security problems:

  • I’m sitting at Starbucks, surfing over their unencrypted wifi, enjoy my coffee and work. Hopefully, all connections are encrypted: email, jabber, VPN to the office, what’s app... But often this is not the case. What happens, if a connection is not encrypted? Everyone in the wifi can listen.

You can encrypt the wifi, but for example WEP doesn't solve the problem, because it doesn't have user isolation. But WPA helps.

If the connection is not encrypted, one can read your emails, your whats app messages or your email login.

First solution: TLS (formerly known as SSL)

Because of the painted scenario all our login and registration pages are SSL secured. We encrypt every transmission, where important data are sent. You can use TLS for nearly everything:

  • POP3 -> POP3S
  • and so on

Second problem: unencrypted session data

Do you only encrypt the login and registration page? Oh, the checkout too. Great! But what is with all the other pages, like »Home«, »Privacy Policy«, »About Us« and so on?
Do you think there are no important data transmitted? You are wrong. With every request there is the session ID sent in a cookie. This means, you convey (a maybe authorized session id) unencrypted personal data in your HTTP header.

The attack vector might be (in Magento):

  • getting personal data: address, order history, wishlist, payment data
  • order with the cusomer’s account to a different address, billing the account owner
  • spending the bonus points or the customer’s credit
  • download already paid virtual content which is found in the account

Second solution: SSL Everywhere

Use SSL everywhere. On every request. If the user comes to your page redirect him directly on HTTPS. Use an official signed certificate, so the user knows, he can trust you.

Third problem: SSL Stripping and ARP Spoofing

There is still at least one problem left. It is called "HTTPS stripping attacks". Moxie Marlinspike implemented a tool called sslstrip and recorded a nice video to demonstrate it.

A hacker can pipe all your traffic (under the correct circumstances) through his own machine and transform all the secured (https) links to insecure links (http).

ARP Spoofing

ARP is a protocol to find the shortest path to another address inside the network, for example between your computer and the router in the network.
Alt text
visualization made by 0x55534C, thanks for that.

ARP Spoofing means, you flood the network with ARP packets and define the way through your computer as the fastest way to the router. This way, all the traffic is piped through your machine.

Now you have the full control over all the traffic. You can read it, you can change it or you can drop some or all packets.

Encryption helps

If your packets are encrypted, you don't have this problem. Because the man-in-the-middle (MITM) can't do anything without your recognition. SSL checks itself for integrity.

The precise problem: Redirection at the beginning

But you remember? The very first connection to your shop is to HTTP:// This means, the connection is unencrypted and an attacker can do his job.

The attacker (let's call him Mallory) ARP spoofs the victims (Alice) laptop, reroutes Alice's traffic through his machine and removes the HTTP Location header. Then Mallory loads the https version of the site Alice wants, changes every https:// to http:// and pipes it to Alice's computer. Now Mallory can do her evil work.

It is important to understand, that most users don't realize or check, wether they are on a https:// site. Or wether their address bar is green or blue. Don't rely on the user.

Third/Final solution: Use HTTP Strict Transport Security (HSTS)

HSTS is a server side HTTP Header, specified in RFC 6796.

It does two things:

  1. It ensures, that the connection is secure. If it is not possible to connect to the server on a secure connection, then an error is shown. This is applied in Chrome, Firefox and Opera so, that a connection via http is no longer possible.
  2. It transforms every link on the page from http to https.

How HSTS works

HSTS is a HTTP Header:

Strict-Transport-Security "max-age=31536000"
Strict-Transport-Security "max-age=31536000; includeSubDomains"

This means: Don't allow insecure connections for the next 365 days to my domain, with or without subdomains.

There is still one problem: The very first request may still be http. This consideration is correct. But the idea is: Hopefully the user are at home or in any secure network, when the user makes this request. If not, he is doomed. ;-)

However, if this first request is made, he is secure for the next 365 (or whatever the timespan is) days (on this device!).

As you can see: This final solution I showed to you doesn't guarantee complete security but they minimize the risk for a security breach.

Advertisement: Magento Module for HSTS

I implemented a module, which does all this for magento: Ikonoshirt_StrictTransportSecurity

More Information

Add Amin User

Magento Cheat Sheet, how to add a magento admin user, directly thorugh SQL

Thanks to Atwix, i found the SQL which is needed to create a magento admin user.

Cheat sheet for me:

SET @fistname = 'Fabian';
SET @lastname = 'Blechschmidt';
SET @email = '';
SET @username = 'fabian';
SET @password = 'password';
SET @salt = 'Fl';

INSERT INTO admin_user
NULL user_id,
@fistname firstname,
@lastname lastname,
@email email,
@username username,
CONCAT(MD5(CONCAT(@salt, @password)), ':', @salt) password,
NOW( ) created,
NULL modified,
NULL logdate,
0 lognum,
0 reload_acl_flag,
1 is_active,
(SELECT MAX(extra) FROM admin_user WHERE extra IS NOT NULL) extra,
NULL rp_token,
NOW() rp_token_created_at;

INSERT INTO admin_role
NULL role_id,
(SELECT role_id FROM admin_role WHERE role_name = 'Administrators') parent_id,
2 tree_level,
0 sort_order,
'U' role_type,
(SELECT user_id FROM admin_user WHERE username = @username) user_id,
@username role_name

You have imported the sample data (which are based on EE)? Use this:

SET @fistname = 'Fabian';
SET @lastname = 'Blechschmidt';
SET @email = '';
SET @username = 'fabian';
SET @password = 'password';
SET @salt = 'Fl';

INSERT INTO admin_user
NULL user_id,
@fistname firstname,
@lastname lastname,
@email email,
@username username,
CONCAT(MD5(CONCAT(@salt, @password)), ':', @salt) password,
NOW( ) created,
NULL modified,
NULL logdate,
0 lognum,
0 reload_acl_flag,
1 is_active,
(SELECT MAX(extra) FROM admin_user WHERE extra IS NOT NULL) extra,
NULL rp_token,
NOW() rp_token_created_at,
NULL failures_num,
NULL first_failure,
NULL lock_expires;

INSERT INTO admin_role
NULL role_id,
(SELECT role_id FROM admin_role WHERE role_name = 'Administrators') parent_id,
2 tree_level,
0 sort_order,
'U' role_type,
(SELECT user_id FROM admin_user WHERE username = @username) user_id,
@username role_name,
1 gws_is_all,
1 gws_websites,
1 gws_store_groups;


Standard customer group with "wrong" id and the consequences

Yesterday I debugged an interessting behaviour:

All categories were shown in the shop, no products. If you request the product directly /catalog/product/view/id/12345 you found the product and everything was shown.

The standard errors were tested:

  • reindex
  • clean cache
  • activate logging
  • use the default template

Nothing helped. It took me about an hour to find the following bug. And to be honest, it was real luck. If you ask me, with this error you can search for days!

How to find the problem?

I checked the database for the indexes, everything looked great, magento said, all indexes are ready and useable.

I had a look into the backend and everything was fine. Products were actived, had a qty > 0 and were in stock. I tried to add a product to the website but it didn't help.

Then I dig into the Product_List of the Category view. I die($_productCollection->getSelect()) the query and played with it. after removing the price_index I got results!

Yea! Results!

The price_index filtered for customer_group = 0. I know this is the id for the NOT LOGGED IN, so I had a look into the customer_group table. Don't ask me why but THIS IS WRONG:

customer_group_id  |  customer_group_code  |  tax_class_id
                4  |        NOT LOGGED IN  |             3

I had a look into the price_index and found prices for the customer groups with the IDs 1,2,3 and 4. No prices for 0.

Ok, I built a long query to change this, I tried it in a big transaction but it didn't work - don't ask me why, it threw after the first query a

Cannot add or update a child row: a foreign key constraint fails

if you can tell me why, please send me an email!

So I skipped the foreign key check:

UPDATE `salesrule_customer_group` SET `customer_group_id` = '0' WHERE `salesrule_customer_group`.`customer_group_id` =4;
UPDATE `salesrule_product_attribute` SET `customer_group_id` = '0' WHERE `salesrule_product_attribute`.`customer_group_id` =4;
UPDATE `customer_group` SET `customer_group_id` = '0' WHERE `customer_group`.`customer_group_id` =4;
UPDATE `catalog_product_bundle_price_index` SET `customer_group_id` = '0' WHERE `customer_group_id` =4;
UPDATE `catalog_product_entity_group_price` SET `customer_group_id` = '0' WHERE `customer_group_id` =4;
UPDATE `catalog_product_entity_tier_price` SET `customer_group_id` = '0' WHERE `customer_group_id` =4;
UPDATE `catalog_product_index_group_price` SET `customer_group_id` = '0' WHERE `customer_group_id` =4;
UPDATE `catalog_product_index_price` SET `customer_group_id` = '0' WHERE `customer_group_id` =4;
UPDATE `catalog_product_index_tier_price` SET `customer_group_id` = '0' WHERE `customer_group_id` =4;
UPDATE `catalogindex_minimal_price` SET `customer_group_id` = '0' WHERE `customer_group_id` =4;
UPDATE `catalogrule_customer_group` SET `customer_group_id` = '0' WHERE `customer_group_id` =4;
UPDATE `catalogrule_group_website` SET `customer_group_id` = '0' WHERE `customer_group_id` =4;
UPDATE `catalogrule_product` SET `customer_group_id` = '0' WHERE `customer_group_id` =4;
UPDATE `catalogrule_product_price` SET `customer_group_id` = '0' WHERE `customer_group_id` =4;
UPDATE `weee_discount` SET `customer_group_id` = '0' WHERE `customer_group_id` =4;

But why is the wrong ID a problem?

Magento expects that the NOT_LOGGED_IN customer group have the id 0:

Mage_Customer_Model_Group::NOT_LOGGED_IN_ID  = 0

And this const is used in:

// app/code/core/Mage/Catalog/Model/Resource/Product/Collection.php:1340
// Mage_Catalog_Model_Resource_Product_Collection::addPriceData()

$customerGroupId = Mage::getSingleton('customer/session')->getCustomerGroupId();

// app/code/core/Mage/Customer/Model/Session.php:174
// Mage_Customer_Model_Session::getCustomerGroupId()

public function getCustomerGroupId()
    if ($this->getData('customer_group_id')) {
        return $this->getData('customer_group_id');
    if ($this->isLoggedIn() && $this->getCustomer()) {
        return $this->getCustomer()->getGroupId();
    return Mage_Customer_Model_Group::NOT_LOGGED_IN_ID;

How does this happen?

I have no idea. Mage_Customer has three install scripts, all of them create the customer group NOT LOGGED IN explicit with the ID 0.


The index is created with a big SQL statement (I think, didn't check this) and therefore uses the customer_group_id from the database and magento's product list uses the constat from Mage_Customer_Model_Group if they don't match, you have a problem.


Damian tweeted that you have to place a


at the beginning of your SQL dump. If you don't do this, your admin store, the customer_group, etc. geting new IDs.

Joining a flat table on EAV

How to use the Mage_Eav_Model_Entity_Collection_Abstract::joinTable() method to join a flat table on a magento EAV collection

Joining a "normal" (aka flat table) to a magento EAV table is easy - if you know how to do.

As always there are a lot resources with tipps I don't like about joining Tables, because they all use getSelect()->join() and fall back to Zend_Db to join the tables. This can (but don't have to) lead to a few problems. In my past I got for example problems with getSize(), because the changes are only made to the Select-Statement not the Count-Statement.

joinTable() TL;DR

But magento collections have a method joinTable() it took me 45min to fizzle out how it is used. To avoid this for you, I share it.

	array('bonus' => 'mycompany/bonus'), 'product_id=entity_id',
	array('bonus_id' => 'bonus_id')

The parameters are:

public function joinTable($table, $bind, $fields = null, $cond = null, $joinType = 'inner')
  1. Table is easy, it is the magento namespace/entity format, which you use in your configuration, resource models and the collection. You can use an array of the format array('alias' => 'namespace/entity')
  2. Bind means, the ON statement in your SQL. This was the hardest part and I will explain it in details later. It is important to have your field of the flat table BEFORE and your attribute of the EAV table AFTER the equal sign. Don't use main_table. before the attribute. Magento will do this for you. More on this later.
  3. Fields is an array. If you use a string instead, you get this: Warning: Invalid argument supplied for foreach() in /var/www/magento- on line 775. You can use an array of the format array('field1', 'field2', '...') or array('alias' => 'field1', '...')
  4. Condition is a ** WHERE ON condition** in the SQL.
  5. joinType. I hope you know what it is. But you have only the choice between LEFT and INNER. app/code/core/Mage/Eav/Model/Entity/Collection/Abstract.php:793

more details

The method can be found here:


check the tablename and alias

First, the array of the tablename is splitted into alias and tablename, the tablename is resolved and the alias is checked for existance.

$tableAlias = null;
if (is_array($table)) {
    list($tableAlias, $tableName) = each($table);
} else {
    $tableName = $table;

// validate table
if (strpos($tableName, '/') !== false) {
    $tableName = Mage::getSingleton('core/resource')->getTableName($tableName);
if (empty($tableAlias)) {
    $tableAlias = $tableName;

check the fields

Then the fields are checked. Are they already defined? If not, add them.

// validate fields and aliases
if (!$fields) {
    throw Mage::exception('Mage_Eav', Mage::helper('eav')->__('Invalid joint fields'));
foreach ($fields as $alias=>$field) {
    if (isset($this->_joinFields[$alias])) {
        throw Mage::exception(
            Mage::helper('eav')->__('A joint field with this alias (%s) is already declared', $alias)
    $this->_joinFields[$alias] = array(
        'table' => $tableAlias,
        'field' => $field,

the hard part, check the bind

The bind is exploded at the =, and the foreign key is expected after the equality sign. It took me 25min to find this. I didn't think about this. I thought it will be took directly into the SQL, but no, magento processes it a lot. If the column of your flat table is the second "argument", magento can't find the attribute and throws an exception: Invalid attribute name: product_id

I started with 'main_table.entity_id=bonus.product_id' and ended with 'product_id=entity_id' and here is the code:

// validate bind
list($pk, $fk) = explode('=', $bind);
$bindCond = $tableAlias . '.' . $pk . '=' . $this->_getAttributeFieldName($fk);

Then the join method is choosed and the conditions are added. Interessting to see here is, that the whole condition is put into the ON statement. To be honest, I don't know the difference between ON and WHERE, but this looks like ON is processed on the table and WHERE is processed on the result?

And the second gem, I found here is str_replace('{{table}}', $tableAlias, $cond)., so you can use {{table}} if your condition is a string instead of the alias.

// process join type
switch ($joinType) {
    case 'left':
        $joinMethod = 'joinLeft';

        $joinMethod = 'join';
$condArr = array($bindCond);

// add where condition if needed
if ($cond !== null) {
    if (is_array($cond)) {
        foreach ($cond as $k => $v) {
            $condArr[] = $this->_getConditionSql($tableAlias.'.'.$k, $v);
    } else {
        $condArr[] = str_replace('{{table}}', $tableAlias, $cond);
$cond = '('.implode(') AND (', $condArr).')';

// join table
$this->getSelect()->$joinMethod(array($tableAlias => $tableName), $cond, $fields);

return $this;

The end

And in the end, magento uses getSelect->join() wonderful :-) BUT magento makes a lot of checks for you and using this method feels better.

Add Address To Registration


Thanks to Steven Henk Don

	<reference name="customer_form_register">
    	<action method="setShowAddressFields">

Longer story

I talked a few days ago with Sascha from about old, not more functional or "wrong" answers to "standard" magento questions. I had to add address fields to the customer registration and thought about a observer to do it, but there is an easier way. This is my contribution to replace the google finds with better answers to always the same questions.

Dig in to the controller

I took a look into the Mage_Customer_AccountController and found:

if ($this->getRequest()->getPost('create_address')) {
[... create address ...]

Deeper into the template

So obviously magento has the option to ask for the address in the registration. A look into the form template brought the finding:

<?php if($this->getShowAddressFields()): ?>
// echo address fields

How to return true?

Ok, how use this method and how to return true?

The "wrong" answers

There is the suggestion to add the XML to your local.xml I prefer not to use the local.xml but this is ok too - if you ask me.

The right answer

Writing an extension is a lot of overkill but does the job. Adding the Setter to the layout or local XML is the way I prefer.

But hacking the core, changing core templates or layout XML files brings a lot of problems while updating!