Not in my backyard (NIMBY) adherents across the country are beginning to weaponize a recent Urban Institute study that reviewed 180 zoning reforms and concluded these reforms barely affected the housing supply. Given Urban’s wide distribution and the paper’s seemingly comprehensive approach, coupled with the eagerness of NIMBYs to exploit such research, housing supply advocates need to be aware of the serious flaws inherent in this study’s methodology and conclusions, which my coauthors and I highlight in a new paper.
Urban’s Flawed Methodology and Misclassifications
Central to the study’s methodology is the use of machine learning algorithms to classify news articles related to land use changes. According to the study, this approach identifies 180 major land use changes from a set of 240 articles.
We evaluated a 20 percent random sample of the 180 policy changes the authors studied using the same database, publication dates, and news sources as the original study. We found a staggering 86 percent misclassification rate (see table). Many changes were incorrectly marked as major changes when they were minor (33%) or ineffectual (11%). For example, the Skokie Review mentioned a policy change that allowed for the construction of 27 homes—a minor reform given the town’s total housing stock of 25,000 units. Similarly, the Mesquite News discussed the easing of ADU regulations but also revealed several stringent requirements on ADU construction—like a maximum size of 500 sqft. or a half-acre minimum lot size—which make widespread ADU adoption less likely.
Although outside our random sample, an example from Philadelphia is also illustrative. Urban classified a policy change under the “Rezoning” category and as more restrictive. The article in the Philadelphia Inquirer on May 7, 2017 states that the policy “decreases the density of certain areas bounded by North Broad Street and Girard, Ridge, and Cecil B. Moore Avenues in an effort to preserve more single-family housing here.” The area affected by this ordinance constitutes less than 1 percent of the city’s total area, as illustrated in the accompanying map. This case underscores the necessity for a more nuanced approach in the study’s methodology, one that more accurately reflects the true extent and potential impact of the included land use reforms.
Map: Philadelphia City and the area affected by the “Rezoning” policy change (colored in red)
Source: Google Maps and AEI Housing Center.
In many cases of the randomly sampled articles, we could not confirm the policy direction from the available information (22%). In the remaining cases, the algorithm incorrectly classified the nature of the policy changes (8%), tagged unrelated articles (8%), or was based on op-eds (3%). This highlights an overreliance on automated systems and media sources, which often lack detailed, accurate information necessary for such assessments.
Table: Results from a 20% Random Sample
Source: AEI Housing Center.
The Oversimplification Problem
The Urban Institute’s binary classification of land use changes as either “more” or “less” restrictive oversimplifies the complexity of housing supply reforms. This approach fails to capture the nuanced reality of these changes and their varied significance, potentially skewing the perceived impact on housing markets. Minor zoning adjustments, which typically have limited effects on the overall housing supply and affordability, are treated with the same significance as major reforms. This methodological flaw could lead to misleading conclusions about the effectiveness of land use reform.
The Need for Detailed, Context-Aware Analyses
The issues with this paper underscore the need for a nuanced, comprehensive approach to analyzing the effects of land use reform. Moving beyond broad datasets and generalized machine learning interpretations, we must conduct in-depth case studies that rigorously examine local contexts, specific policy details, and their outcomes. This ensures more precise and practical insights that directly inform policymaking.
Based on extensive case studies, we have distilled what we call the “Housing Abundance Success Sequence”:
- Enable by-right zoning: Simplify the approval process to allow developments that meet preset criteria to proceed without additional permissions.
- Promote greater density: Especially in areas that are walkable and rich with amenities, to efficiently use land and resources.
- Streamline regulations: Implement clear and straightforward land use rules, expedite the permitting process, and lower construction costs.
This strategy taps into the creativity and capacity of builders, particularly smaller and mid-sized builders, facilitating the construction of ample market-rate housing. Countless case studies show that adopting these measures can lead to approximately a 2% annual increase in housing supply. Crucially, we have also observed that deviations from the Housing Abundance Success Sequence either failed to stimulate a market response or halted an existing one.
Conclusion: Championing Precision and Local Adaptability in Zoning Reforms
Problems with the Urban Institute’s study on land use reforms accentuates the need for a deeper understanding of land use reform policy changes and their contexts. For supply advocates, the path forward is to rigorously evaluate the methodologies of studies and advocate for reforms grounded in detailed, context-aware research that genuinely reflects the intricate dynamics of urban planning and its impact on market participants.
By adhering to this principle, supply advocates can effectively argue for truly effective zoning and land use reforms, ensuring that development regulations foster, rather than hinder, housing abundance.
This article was originally published by the Metropolitan Abundance Project. You can read the original article here, or by following this link: https://www.metroabundance.org/the-real-impact-of-zoning-and-land-use-reforms-contrary-to-the-urban-institutes-claims/.