A student of mine sent me this link in which it is argued that changing the composition of the Dow (or SP 500), causes these indices to underperform a fixed basket of stocks.
The components of both indices change when a stock drops out and is replaced by a new stock and the new addition is invariably some hot growth stock that has seen a rapid run up (i.e. MSFT, GOOG, Yahoo etc) and the stock that is dropped out (particularly in the Dow) is a value stock. So, in effect, the indices buy growth stocks when their values are high.
Indexing is great because it reduces the cost of active management. But as this article shows, there is still the active management of the index creator that can mess things up for you.