The shock of the COVID-19 pandemic created urgent demand for “high frequency” national statistics. Prior to the pandemic, many economic indicators were available only on an annual basis (or even less frequently). One important exception was the unemployment rate, which the Bureau of Labor Statistics estimates every month. The unemployment rate jumped more than threefold—from 4.4 percent in March 2020 to 14.7 percent in April, reflecting the onset of the COVID-inspired shutdown after the March data were collected.

Soon after the shutdown, researchers interested in its other effects began to field new monthly and weekly surveys. One of these efforts inspired by COVID was the Survey of Working Arrangements and Attitudes (SWAA), launched by the newly formed WFH Research. The SWAA launched in May 2020, reporting (among other findings) that 61.5 percent of adults who worked did so from home that month.[1]

Understandably, the new numbers from the SWAA and other pandemic-era surveys created demand for pre-shutdown estimates to put them in context. Such estimates—often from large federal surveys—exist in many cases, and it has proven tempting simply to connect those earlier estimates to the new post-shutdown ones. Following this approach, WFH Research concluded that working from home rose to its May 2020 level from a rate of about 5 percent before the pandemic, suggesting a 12-fold increase.[2] In July 2023, the work-from-home rate (31 percent) was still 6 times higher than before COVID.

However, linking similar-seeming but different surveys can lead to inaccurate conclusions about trends. The best data (unfortunately only available annually) measure the work-from-home rate consistently before and after the COVID shutdown. Using two different data sources that span the early 2020 shutdown, we find that WFH Research’s numbers substantially overstate the increase in working from home; the increase since the start of COVID has been half as large as its estimates suggest.

While WFH Research finds that working from home increased by a factor of 8 between 2019 and 2020, we find that it rose by a factor of 3—more than 60 percent lower. In 2021, the work-from-home rate was about 4 times the 2019 rate, not 7 times higher as the WFH Research numbers indicate. Rather than being 6 times higher today than before the pandemic, we estimate working from home is less than three times as high. We estimate the work-from-home rate for 2023 will be around 19 percent, compared with 7 percent before COVID.

Comparing the Incomparable

The reason the WFH Research estimates overstate the rise in working from home is that the pre-pandemic surveys to which it links its estimates are not comparable to the SWAA. As one of us (Winship) explained in a report on food hardship with Angela Rachidi, linking estimates from two surveys can produce misleading results, even if they seem like they should measure the same thing.[3] That can happen if questions are asked differently in two surveys, if their placement within the surveys differs, if the length of the surveys differs, or if the administration of the surveys generate different sample distributions or responses. In any of those cases, a spike that appears between the two surveys would be due, at least in part, to influences unrelated to the issue under study.

Winship and Rachidi countered claims that food hardship rates had increased by a factor of 3 to 8 in the first month of the pandemic by showing that the pre- and post-shutdown estimates that researchers were linking midway through 2020 were potentially incomparable. Sure enough, when it became possible a year later to look at what happened to food hardship using data from an annual survey conducted by the federal government, food hardship in late 2020 was no worse than in late 2019, even though the incomparable estimates developed post-shutdown still showed a rate four times higher than the late-2019 rate.[4]

The same sort of issue appears to plague the WFH Research claims. One can see the importance of survey inconsistencies even within their own SWAA estimates over time. WFH Research admirably makes these estimates available to the public on its homepage, and it provides methodological details on the site.[5]

The SWAA has asked about days working from home in three different ways. The methodology was the same from May to October of 2020, from November 2020 through October 2021, and from November 2021 to the present. Importantly, the survey also continued to ask the questions underlying the second methodological approach (November 2020 to October 2021) through April 2023, even though they are not incorporated into the published WFH Research trend. WFH Research also provides the estimates using these questions in its publicly available spreadsheet.[6]

The thick solid lines (of varied colors) in Figure 1 show the work-from-home rates using the estimates from these three distinct methodologies. It is clear that the two changes in the way the survey asked about working from home produced significantly different results. The work-from-home rate rose by 8.4 percentage points between the October and November 2020 estimates, though it was falling rapidly before that. Then between October and November 2021, the rate fell 8.6 points, though it was only gradually declining before and after the series break.

To produce the smooth series that WFH Research publicly presents today, it uses a basic statistical technique (regression analysis) to adjust the middle-period estimates (bold green line) downward. After October 2021, estimates using both the middle-series methodology (see the faded green line) and the most recent methodology (bold red line) are available. Therefore, one can use the relationship between those two sets of estimates after October 2021 to impute new estimates for the middle series. WFH Research then connects the adjusted middle-series estimates to the earlier and later ones (as the dashed green line in Figure 1 displays).

Even before we link the SWAA estimates to figures using pre-pandemic data, the survey results are ambiguous. Did working from home fall from 61.5 percent of workers to 30.9 percent, as the smoothed estimates suggest? After the adjustment of the middle SWAA series, we can assume that the trend shown by the dashed green line is roughly consistent with the trend for the most recent series of estimates (in the red line), since it is adjusted based on those recent estimates. But are the middle and recent series consistent with the earlier series shown in the blue line? The smoothed series shows a 4.5-point decline from October to November in 2020, but the true decline may have been smaller or larger. (The rate could even have increased.)

Crossing the Divide

But the real problem comes when WFH Research links its initial May 2020 estimate to estimates from before the pandemic. The blue line in Figure 2 is included in charts published by WFH Research and in its academic paper on trends in working from home.[7] It conveys the long-standing low work-from-home rate before the pandemic and the jump after the shutdown. The estimates from 2000 to 2019 come from the Census Bureau’s annual American Community Survey (ACS).[8] The ACS includes a question about how people got to work during the previous week, and one response offered to participants is that they worked from home. From May 2020 forward, the line displays monthly estimates from the SWAA. The share of days worked from home rises from 4.7 percent in 2019 to 61.5 percent in May 2020. It fell thereafter in the SWAA, but it remained at 30.9 percent as of July of this year.

We replicated the ACS results using data available on the IPUMS USA website and the details WFH Research provides.[9] Then we made small adjustments to make the results more comparable to the most recent SWAA estimates. These are shown in the red line in Figure 2.[10]

Since WFH Research published its pre-pandemic estimates, the 2020 and 2021 ACS data has become available (with the estimates included in the red line). This allows us to compare the ACS and SWAA estimates for those years. To do so, we have to put the monthly SWAA estimates into an annual form by averaging across the twelve months of the year. There are no SWAA estimates for January, February, March, April, or June of 2020, so we impute them conservatively.[11]

These estimates are shown in the purple line in Figure 2 (plotted at January 1, for consistency with the way WFH Research plots its ACS estimates). Between 2019 and 2020, the work-from home rate increases from 4.7 percent to 38.0 percent, rising by a factor of eight.

But this compares a 2019 estimate from the ACS to a 2020 estimate from the SWAA. When we compare the 2019 and 2020 ACS estimates, the work-from-home rate rises from 5.1 percent only to 16.3 percent, 3 times higher than in 2019. This 11.2-point increase compares with a 33.3-point rise in the WFH Research trend. Similarly, while the SWAA indicates the work-from-home rate was 33.6 percent in 2021, the ACS puts it at 18.8 percent—more than 40 percent lower. The 2022 ACS estimate is not yet available, but if it fell by 9.6 percent (not 9.6 percentage points) from 2021 to 2022, as happened in the SWAA, the 2022 rate would be 17 percent rather than the 30.3 percent given by the SWAA—just over half as large.

In conversation over email, WFH Research’s Nicholas Bloom and Steven Davis indicated that they do not believe the ACS provides a good indicator of the trend in working from home. The question wording solicits an all-or-nothing response about working from home or commuting to work, and the interviewer instructions explicitly say to choose commuting to work if someone did both in the survey week. The ACS thus misses shorter stints of working at home, and that only really matters after the shutdown. The ACS trend, therefore, probably tracks changes in working only from home.

As an alternative, Figure 2 also includes estimates from the American Time Use Survey (ATUS) that we computed using data from the IPUMS Time Use website.[12] The ATUS requires participants to keep a “time diary” for an entire day. We define individuals as having worked on a given day if they indicated they worked at all (on a job, doing work-related activities, or doing other income-generating activities) for at least six hours, and we define them as having worked at home if all of that work was done in their home or yard. We again restrict the sample based on age and earnings, and we exclude people whose 6-or-more hours of work is done neither at home nor on premises at their workplace.[13]

Our ATUS estimates (green line) indicate similar work-from-home rates to the ACS before the pandemic. No 2020 rate is available in the ATUS, but from 2019 to 2021, the work-from-home rate rises by 19 points, compared with 14 in the ACS. The linked WFH Research estimates rise 29 points—twice as much as in the ACS and over 50 percent more than in the ATUS.

Another Look Post-COVID

In Figure 3, we replicate a second WFH Research chart—this one focused more narrowly on what happened during and after the COVID pandemic. The blue line shows the trend published by WFH Research, this time linking the SWAA estimates to a “Pre-COVID” estimate.[14] Consistent with the WFH Research results shown in Figure 2, the blue line in Figure 3 indicates a jump in working from home from 4.8 percent to 61.5 percent. The purple line in the chart shows the annual averages of the SWAA data for 2021 and 2022 that we included in Figure 2 (this time plotted at July 1, midway through the year rather than at January 1). These indicate an increase from the “pre-COVID” rate to 33.6 percent in 2020, or 28.8 percentage points.

The increase shown by SWAA is too large however, due to the now-familiar problem of linking incomparable survey estimates.

The green line in Figure 3 shows the work-from-home trend using the ATUS estimates in Figure 2.[15] We choose as our “Pre-COVID” estimate the 2019 ATUS figure of 6.7 (plotting it at March 1, 2020 for consistency with the original WFH Research chart, under the assumption that the work-from-home rate was the same on the eve of the COVID shutdown as in 2019). The increase from “Pre-COVID” to 2021 was 18.7 points—one-third smaller than the increase using the WFH Research estimates.

By 2022, the SWAA work-from-home rate (30.3 percent) was still six times WFH Research’s “Pre-COVID” rate. Our 2022 ATUS rate was significantly lower (21.7 percent), and the rate increased by a factor of just over three rather than six-and-a-half. The increase from “Pre-COVID” to 2022 was 40 percent smaller in the ATUS than in the WFH Research chart.

Levels vs. Trends

While trends are clearly more accurate when using the same survey for the entire period under analysis, perhaps the SWAA levels today are more accurate than the levels shown in the ATUS? WFH Research shows that the SWAA levels are very similar to those from the Census Bureau’s Household Pulse Survey, which has tracked work from home since June 2022.

There is no particular reason that the levels of these two series should be so close to each other. While the SWAA asks about full days (six or more hours) worked for each of seven days in a week, the Household Pulse Survey simply asks whether someone worked from home at any time in the previous seven days (without specifying they be “full days”). In our ATUS estimates, the share working from home was three or four points higher in 2022 without the restriction that workdays be at least six hours. Furthermore, the income threshold for inclusion is $25,000 for the Household Pulse Survey estimates but $10,000 for the SWAA estimates, and that threshold is based on household income in the former but individual earnings in the latter.

Moreover, the Household Pulse Survey, like the SWAA and other pandemic-era surveys, suffers from compromises that were necessary while the coronavirus impeded in-person contact, disrupted the mail, and complicated phone-banking. These surveys, unlike longstanding large-scale and thoroughly conducted federal and academic efforts, often have relied on suboptimal methods to recruit panelists, such as text messaging and self-selecting internet panels, producing relatively low response rates. In Winship’s papers with Rachidi, they showed that the Household Pulse Survey grossly overstates the level of food hardship compared with long-running annual Census Bureau surveys. (They also described specific deficiencies of the Household Pulse Survey.) We should presume that the post-shutdown estimates in the SWAA and the Household Pulse Survey are inferior to the better-established survey estimates.

It is difficult to pin down the “right” level of working from home. In our ATUS estimates, for 2022 we obtained a range of estimates depending on the criteria we used to define the population of interest. Most of these fell within six points of each other (and produced a similar 2019-2022 decline). At the high end, including only workdays of at least six hours but applying no income cutoff, 26.1 percent worked from home. At the low end, including only those working at least eight hours rather than six and only wage and salary workers with $20,000 or more produced an estimate of 20.4 percent working from home. With the exception of a few unrepresentative subsets, all of the estimates were lower than the 30.3 obtained from averaging the 12 SWAA estimates from 2022.[16] 

What work-from-home rate would make ATUS estimates comparable to those we present show today? From 2021 to 2022, the proportional decline in the work-from-home rate was about 6 percent larger in our ATUS data than in the SWAA. According to the SWAA, the fall in the work-from-home rate flattened out in 2023. Given these facts, we assume the 2023 work-from-home rate will be the same as the average over the first seven months of the year and that the 2022-to-2023 decline in the ATUS rate will be about 6 percent larger than the decline in the SWAA. Under these assumptions, the ATUS rate for 2023 will be 19 percent rather than the roughly 28 percent implied by the SWAA.

Conclusion

The COVID-19 pandemic has surely made working from home more common than it was before 2020—we don’t actually need surveys to tell us that. Surveys can help us understand the magnitude of the increase, but not if we make inferences from pre-shutdown and post-shutdown surveys that aren’t comparable. We should even be a little concerned about whether the SWAA trend since May 2020 is affected by changes to the survey since then.

To be sure, high frequency data from new surveys can be of great value. The SWAA data, for instance, allow us to see how working from home varies by industry and region.

But overstating the rise in working from home can lead not only to incorrect inferences but misguided public and private policies. For example, employers’ assessments of office space needs and appropriate remote-work policies may differ depending on their beliefs about how many people are working from home today compared with before COVID. If they believe working from home has risen markedly, they may assume it is more permanent than if they believe it has risen more modestly.

Finally, understanding from month to month how working from home is evolving is much less important today than it was in the earliest months of the COVID pandemic. Multiple well-established annual surveys allow us to track year-to-year trends in working from home, and those estimate are both more reliable and more consistent over time. Private and public policymaking should rely on those surveys to guide their decisions.