[Note from the editor: Below is a sample special business intelligence report, originally published for members of the GEM. Another example of our work is the property management software category product review series for small landlords. — Drew Meyers]
The Rise of Automated Property Valuations
BY ADAM NAAMANI
Originally Published: March 24th, 2020
When Opendoor launched in 2014, it brought about a pivotal question: Can you value a house solely on data? My optimism led me to believe it was an inevitability, given the data infrastructure and technology to support the assumption. Even within a relatively short time-frame, the wave of digital innovation has grown exponentially. The International Data Corporation (IDC) predicts that the global Datasphere will swell to 175 Zettabytes by 2025 (1 ZB = 1 trillion GB for perspective), and global spending on AI and cognitive technologies to exceed $50 billion by 2021. It’s imperative we discover new ways to consolidate disparate processes into single platforms.
As the iBuyer business model reinvents how property is transacted online, it brings to light the many iterations of use cases in predicting home prices — traditional collateral estimation within the lending industry, lead generation for real estate professionals, instant estimates on public-facing portals (Zestimate®), and most recently, informing property transactions at scale to bring certainty and liquidity to an increasingly on-demand economy (Opendoor, Offerpad, RedfinNow, Zillow Offers™).
What then is the distinction between an automated valuation model (AVM), broker price opinion (BPO), and an appraisal? How has one evolved, while others appear stagnant? At what point does an AVM substitute or outperform the accuracy of an appraisal to become a proven single source of truth? How do consumers cut through the noise of various ‘proprietary algorithms’ to not be misled with misinformation?
In answering these questions, it would be useful to look at a brief history as well as the current state of the underlying methods in objectively determining market value. In doing so, we can gain a better understanding of how technology can augment or even replicate human cognition, and bring efficiency to an ageing profession.
An Industry in a State of Flux
The two most recognized professional organizations of real estate appraisers are The Appraisal Institute (17,000+ members) and The Appraisal Institute of Canada (5,400+ members). Appraisers in both the U.S. and Canada follow generally accepted appraisal standards as evidenced by the Uniform Standards of Professional Appraisal Practice (USPAP or CUSPAP). Educational requirements in some instances include possession of a degree, rigorous curriculum, and mentored experience en route to being designated.
According to the Appraisal Institute, the number of active members has been declining 3% per year, with more on the horizon retiring en masse (average age is around 50). With a litany of pain points, the industry at large is ripe for disruption — faced with fragmented, outdated, and unnecessarily burdensome regulations.
Federal regulation led to the proliferation of Appraisal Management Companies (AMCs) — a firewall between the lenders and appraisers. Following a bidding process, files are triaged by a team of administrators — a strenuous function that is becoming modernized by companies like Anow and Reggora.
Appraisers may operate independently, and along with private lenders, don’t necessarily need to engage in business with an AMC. The variety of parties involved led to a market that has reached a level of oversaturation, resulting in varying inconsistencies.
Through a myriad of intermediaries, complicated workflows, lack of standards across web portals, and software that isn’t platform-agnostic, appraisers are at the behest of organizations that suffer from layers of excessively complicated administrative procedures.
Signal for Change
There had been debate in the U.S. over the threshold in which mortgages should require a formal appraisal, with a majority already exempt by virtue of being below $250,000 ( a limit increased from $100,000 in 1994). Effective October 2019, the OCC, Board, and FDIC adopted a final rule for the federal de minimis on residential real estate transactions to be raised from $250,000 to $400,000.
“The final rule defines a residential real estate transaction as a real estate-related financial transaction that is secured by a single 1-to-4 family residential property. For residential real estate transactions exempted from the appraisal requirement as a result of the revised threshold, regulated institutions must obtain an evaluation of the real property collateral that is consistent with safe and sound banking practices.”
Since 2017, Fannie Mae and Freddie Mac have been providing mortgage products with low loan-to-value ratios unbound by traditional reporting requirements, placing greater reliance on their in-house proprietary analytics. According to their policies, they even allow unlicensed or uncertified appraisers or appraiser trainees to complete a property inspection. It’s their prerogative to identify eligible properties and offer waivers at the application stage.
Consequently, brand whitespace has encouraged new incumbents to develop alternative forms of estimating market value through evaluations, bifurcated and hybrid appraisals — products designed to significantly reduce friction, turn around times, and cost for lending institutions as these elements become untenable.
Factors of Value
You Don’t Know What You Don’t Know
Beyond one’s own property, there’s always curiosity surrounding what other homes are worth so that more informed decisions could be made. The challenge then is in justifying property values, which are inherently controversial as there is an emotional aspect stemming from bias or lack of insight. It’s common to see lenders and homeowners push back with their own unsubstantiated opinions of value — for example, comparing property types such as a half-duplex in a quiet suburban neighbourhood to a condominium in the Downtown core, or concrete to wood-frame house. There’s an even greater chance of misinterpretation when valuations are displayed on public-facing websites devoid of any reconciliation, as it’s not a matter of what but why.
External obsolescence (privacy, street noise, crime), topology, flood plains, conformity to neighbouring properties (progression, regression), proximity to schools and amenities, title to land, orientation (exposure), lot dimensions, floor level, view obstructions, physical depreciation, functional obsolescence, government policies, and demographics — representing only a fraction of thorough preliminary research required for a holistic analysis.
Sales comparison is easier with properties that are homogenous. A cookie-cutter new construction condominium or rectangular Vancouver Special will often have a wealth of nearly identical sales to compare it to, while a custom-built house in an affluent community might not. If a home falls within typical parameters, the margin of error will generally be lower.
Three or more recent sales of similar properties are collected within 60 to 90 days of exposure in an open market. An appraiser will first attempt to find 1-2 sales in the subject building (if it’s a condo) in accordance with standard lender requirements, moving out to the same street, then to the neighbourhood — expanding the geographical area as necessary.
Conditions of sale, stigmatization, and special assessments are also of concern in the process of elimination to gather arm’s length transactions. After qualitative and quantitative adjustments are made for differences between the subject and comparables, a final estimate of value is weighted towards sales with the least adjustments.
Back to the question at hand…
Is it possible, given the technology at our disposal, to replicate this cognitive process with Neural Networks and derive a property’s fair market value commensurate with its human equivalent?
With the abundance of property data comes the issue of fragmentation, yet despite its slow adoption curve, is destined for decentralization. Reducing redundancy is akin to the programming principle of DRY (Don’t Repeat Yourself):
“Every piece of knowledge must have a single, unambiguous, authoritative representation within a system.”
– The Pragmatic Programmer
In many an industry, manual filtering, copying and pasting, calculating, and form filling is becoming archaic as more data is made available in a machine-readable format. Performing repetitive, tedious tasks leads to decision fatigue — a most rudimentary use case for offloading to automation. It is not to say the efforts required by human intervention should be undermined, yet its effectiveness is eroded by a high cognitive load and prone to error. Machine Learning correlates large datasets around the clock better than any person ever could, factoring in new data points the moment they are made available with a high degree of granularity.
Open Standards and the API Economy
Open Data portals and RESTful APIs provide necessary access to key datasets like property parcels, tax records, streets, transportation, city services, schools, crime, etc. to facilitate comprehensive research into understanding utility. The City of Vancouver exemplifies how Open Data APIs have evolved, offering a variety of export types according to the JSON Schema vocabulary standard.
As innovation in the vertical flourishes, so too do the need for integration and standardization. Cleaning and labeling big data is no small feat — it’s laborious, expensive, and prohibitively complex. The Real Estate Standards Organization (RESO) created open standards through a Data Dictionary as the industry’s universal language to make access to MLS data easier and more streamlined.
Zillow’s BridgeAPI™ — a combination of their acquisition of Vancouver based Retsly and Bridge Interactive — is platinum-certified by RESO and provides one of the more modern single point of entry solutions. Its API streamlines the transport process for public and transactional records on 148 million properties throughout the U.S.
Turning Point in Canada
Historical property data had been infamously difficult to acquire, compile, and normalize — hidden behind walled gardens of old. Sold prices and cumulative days on market have for too long been a blind spot to consumers in Canada doing their own due diligence.
Between Canada and the U.S., there had been differing levels of access to property data, until the Competition Bureau was successful in their anti-competition case against TREB; a costly 7-year legal battle to end data restrictions.
“It’s time to move forward, embrace policies that align with the law, and open the door to innovation in real estate services.”
– Matthew Boswell, Interim Commissioner of Competition
Ubiquity of Technology
Machine Learning as a Service (MLaaS) platforms from Amazon Web Services and Google Cloud have levelled the playing field, with a pay for what you use business model and entry-level tiers to build and scale any project at a fraction of what it used to cost.
Training ML models has never been more approachable — made simple and accessible by open source libraries like NumPy, SciPy, scikit-learn, and pandas — with d3js and Jupyter notebooks for data-driven visualization. SageMaker, AutoGluon, and Tensorflow are toolkits that provide deep learning solutions with only a few lines of code. The development of Neural Networks opens up many use cases applicable to real estate:
- Image classification (qualitative analysis)
- Object detection (degree of affixation)
- Tabular prediction (comparative analysis)
- Regression models (predictive analytics)
The Future of Housing
Beyond advancements in property technology software, is the emergence of Smart Real Estate — technology-based platforms that facilitate the operation of real estate assets. Buildings are being fitted with Internet of Things (IoT) sensors able to monitor interior activity, energy efficiency, or eventually much like smart cars, indication for when renovations are required (settling, moisture, building code).
Building Information Modeling (BIM) — digital representation of physical and functional characteristics — is considered one of the top 3 technologies likely to cause maximum disruption according to a 2019 Altus survey with real estate development firms. It’s well-suited towards property valuation, such that creating a ‘digital twin’ could provide real-time access to the most recent version of a survey or floorplan for pre-construction analysis. In combination with LiDAR for 3D models, or drones for geospatial imagery, the integration of technology into the physical world opens up possibilities previously unthinkable.
Power to the People
While not an exhaustive list, the following companies are the most interesting to watch develop. They have the capital, network effects, and greatest potential to revolutionize AVMs:
At a 4.5% median error rate, half of all Zestimates are within 4.5% of the selling price. To further their efforts, Zillow held a $1 million global data science competition to improve the home valuation, won by a team that hadn’t even met each other in person — a common occurrence in today’s distributed workforce environment facilitated by tools like Github and Slack.
The winning team developed a system that mimicked the neural circuitry of the brain, leading to the building of accurate predictive models to improve the algorithm that changed the world of real estate.
Every 60 seconds, a homeowner requests an offer from Opendoor. That scale is inconceivable without data science and automation. To improve the accuracy of their valuations, Opendoor uses an ensembling approach — building multiple models and computing a weighted average of their estimates. Through hybrid pricing, they use both automated and human-led valuations to scale rapidly while maintaining accuracy.
With four decades of transaction data on 106.5 million homes throughout the U.S., HouseCanary is pioneering modern end-to-end valuations for investors and real estate professionals alike. Their predictive analytics have a Median Absolute Prediction Error (MdAPE) at 2.8% as of July 2019 on 1,994,203 transactions.
Predicting an Uncertain Future
In an on-demand society, customers are trained to expect speed and efficiency. For a capital intensive industry like real estate, cost and scale are among the biggest barriers in achieving widespread adoption of novel technology. As for the appraisal profession, it’s not a matter of advocating that it become obsolete, but how technology can best be incorporated into the trade at a faster rate. Automated valuations bring the promise of reducing friction from the complexity of property transactions — enabling smarter, faster decision-making with unprecedented levels of access to information. Data transparency is becoming the new norm, and as technology rapidly evolves, businesses must adapt to maintain a competitive edge.
Even since I began writing this article, a certain pandemic occurred. The world changed in an instant. What will happen to professions that require entry into a property? Perhaps the sentiment towards privacy will change, as the data collected digitally is considered less of an infringement than a host of people coming through your home. It has never been more critical to accept innovation as the only way forward — case in point as Fannie and Freddie have already begun to adopt alternative appraisals due to the coronavirus. Nearly every stage of a real estate transaction can now happen virtually. That’s profound to even think about when it comes to the world’s largest asset class. The emotion we have tied to our homes carries a value greater than any number could possibly represent, making it infinitely difficult for intelligence in any form to predict with absolute certainty what tomorrow will bring.
GEEK ESTATE MASTERMIND BRIEFING
A PRIVATE GROUP OF INDEPENDENT THINKERS, FREE FROM SPONSORED MESSAGES, SALES PITCHES AND NOISE
There are four parts to membership:
- Long-form articles covering the spectrum from shipping container co-living spaces to the battle for listing acquisition in the first iBuyer world war (Weekly Transmission).
- Curated real estate, startups, & built world links & analysis blended with out of the box ideas (Weekly Radar).
- Special reports (our first is a category review of Small Landlord Prop Mgmt Software).
- Networking opportunities with 235+ innovators from across the globe through the private forum & in-person gatherings.
Membership is $109 / quarter
OUR MEMBER PROMISE
- We deliver an exclusive, objective lens into the trends, companies, people, and ideas shaping real estate technology with thought-provoking analysis and conversations that keep you inspired every week.
- We help you make better, more well-informed decisions to help grow and support people and companies making a difference in real estate.
- We enable discovery and meeting others with shared interests online and in-person (whether they live near you or are traveling to the same conference).
With a mission to attract the 1,500 most forward-thinking, and diverse, innovators, we’re looking for the best and brightest in all the land...
READY TO JOIN RIGHT NOW?
NOT QUITE READY?