Are US small caps overvalued?

The problem with small cap indices

The rise of passive investing in the last decades has sparked much debate about whether stocks are still priced appropriately in the market (see e.g. this video for a discussion). As a matter of fact, the aggregate PE ratio of large cap US stocks in the S&P 500 has been hovering around 20-25 for the past 5 years now, a level which some consider indicative of a bubble — although it is unclear whether this is due to passive investing or other factors such as equity buybacks fuelled by low interest rates, ageing demographics, etc. But what about small cap stocks? Small cap stocks have received much attention recently as they have outperformed large cap stocks in the US over the past 15 years. US small cap stocks may represent a good investment opportunity, but some also argue that they are currently highly overvalued (e.g. this, this or this).

Index funds tracking small caps are becoming more popular with the rise of factor investing. However, it is not always easy to get a clear picture of the quality of the firms included in the underlying index. The issue with aggregate PE ratios for small cap stocks indices is that a significant share of small cap firms might have negative earnings, and those are usually excluded from aggregate PE ratio computations. For example, the aggregate PE ratio (excluding negative earnings) for the Russell 2000 index is 15.71 in April 2020 according to the factsheet provided by FTSE Russell. Not so bad … but according to bloomberg, the PE ratio is actually closer to 45, and the WSJ also publishes a similar figure. So, are US small cap stocks overvalued? How many firms in the Russell 2000 actually have negative earnings? What is the aggregate PE ratio of the Russell 2000 index if these firms are included in the computation? This post attempts to answer these questions by gathering information on company fundamentals for all firms in the Russell 2000 index. The methods shown here can easily be adapted to other index providers such as MSCI or STOXX.

Obtaining index constituents

Index providers are required by ESMA guidelines to disclose a list of constituents of their indices: this means that it is usually possible to get a list of firms included in an index. The list of firms included in the Russell 2000 is available here in pdf format. We can use the python package tabula to read the pdf and extract the list of constituents directly as a dataframe.

import requests as rq
import tabula as tb
import tempfile as tf
import pandas as pd

url = "https://research.ftserussell.com/analytics/factsheets/Home/DownloadConstituentsWeights/?indexdetails=US2000"
with tf.NamedTemporaryFile() as f:
    f.write(rq.get(url).content)
    f.flush()
    t = tb.read_pdf(f.name, pages='all', silent=True)

constituents = pd.concat(t).reset_index(drop=True)
constituents.columns = ['name', 'weight', 'country']
display(constituents)

Getting ticker codes

This provides us with the names of all 1995 companies in the Russell 2000 index, as well as with their weight in the index. However at this stage we lack a standard identifier for the firms, which makes it hard to get financial information. A convenient way to get a firm's ticker or RIC is to use the OpenPermID search API. OpenPermID is a database of entities (organizations, persons, financials instruments, etc.) curated by Refinitiv. It comes with a python package, making it very easy to use. You will need to ask for a free API token, which allows for up to 5,000 requests per day (enough in our case). The search API will return matches in three categories (organizations, instruments, and quotes), with up to 5 matches in each category (by default) for a given query. The only issue is that it is quite slow, as the API limits queries to 1 or 2 per second for a free token.

from OpenPermID import OpenPermID
from collections import defaultdict
from time import sleep
import logging
from tqdm import tqdm_notebook
from IPython.display import Markdown

# Initialize the OpenPermID api
TOKEN = 'your_token'
opid = OpenPermID()
opid.set_access_token(TOKEN)
opid.set_log_level(logging.CRITICAL)

# Search the OpenPermID database
entities = defaultdict(list)
for r in tqdm_notebook(constituents.itertuples(), total=constituents.shape[0]):
    data, error = opid.search(r.name)
    if (error is not None):
        # Maybe we exceeded the rate limit: wait and retry
        sleep(5)
        data, error = opid.search(r.name)
    if type(data)==dict:
        # the API returns a dict with relevant matches for each category
        for k, v in data.items():
            v['id'] = r.Index
            entities[k].append(v)

# Print the first match in each category
entities = {k: pd.concat(v) for k,v in entities.items()}
for k,v in entities.items():
    display(Markdown(f'* **Matches for {k}:**'))
    display(v.loc[0].head())

• Matches for organizations:

• Matches for instruments:

• Matches for quotes:

# Format the information on names and tickers
ent = dict()
ent['org'] = (entities['organizations']
              .rename({'organizationName': 'opid_name',
                       'primaryTicker': 'opid_ticker'}, axis=1))
ent['ins'] = (entities['instruments']
            .rename({'primaryTicker': 'opid_ticker'}, axis=1)
            .assign(opid_name = entities['instruments'].isIssuedByName
                    .fillna(entities['instruments'].hasName)))
ent['quo'] = (entities['quotes']
            .rename({'hasExchangeTicker': 'opid_ticker'}, axis=1)
            .assign(opid_name = entities['quotes'].isQuoteOfInstrumentName
                    .fillna(entities['quotes'].hasName)))

# Pick the ticker
for c in ['org', 'ins', 'quo']:
    d = (ent[c]
         .groupby('id')
         .first()
         .loc[:,['opid_ticker', 'opid_name']]
        )
    if c=='org':
        info = d
    else:
        info.update(d, overwrite=False)

# Add to the data and display for inspection
constituents = pd.concat([info, constituents], axis=1)
with pd.option_context('display.max_rows', None, 'display.max_columns', None):
    display(constituents.sort_values('weight', ascending=False))