Appraiser Versus Machine?
Loan officer, the bank, homeowner… want a higher value…
Automated Valuation Models: Are they any good?
Appraiser vs Machine? AVMs were run on a few properties that were recently appraised. The results are well, ….. they are what they are.
The questions are: Are they good? Are they good enough? Will the consumer accept them? Will the banks and lenders accept them?
Below are the samples. We have to be careful as we are appraisers and don’t want to be sitting in front of the VREAB explaining why we shared assignment results in violation of the confidentiality requirements of the Ethics Rule of USPAP. We will be brief as possible but provide enough commentary so you will understand a little about the property.
The left column states the website of the AVM. The middle columns are the property particulars stated on the AVM. The next to the last column on the right shows the AVM results and the last column shows the percentage of the appraisers opinion of value.
Property one is an older colonial style home that was recently professionally renovated. This home is larger than surrounding homes, has an unfinished basement, 2 car attached garage and a 1 car detached garage. Here are the results:
The AVM values range from 81.0% to 109.1% of the appraiser’s opinion of value. Only one AVM result was higher than the appraiser’s opinion of value. Let’s think about this. The loan officer, who works on a commission that is a percentage of the loan amount, wants a higher value. The bank wants to make a loan and wants the higher value. The homeowner has an incentive in a lower loan to value for a lower interest rate or an incentive to maximize the cash out, so the homeowner wants the higher value. Only one out of the eight AVM’s would any party to the transaction be happy with. Who’s the better choice, appraiser or machine?
Property two is an attached single story town home with a 1 car garage. This home is dated with original finishes from 33 years ago. Here are the results:
The AVM ranges from 91.1% to 98.5% of the appraiser’s opinion of value. Not one AVM result was higher than the appraiser’s opinion of value. Let’s think about this. The loan officer, who works on a commission that is a percentage of the loan amount, wants a higher value. The bank wants to make a loan and wants the higher value. The homeowner has an incentive in a lower loan to value for a lower interest rate or an incentive to maximize the cash out, so the homeowner wants the higher value. Since no AVM was as high as the appraisers opinion of value, would any party to the transaction be happy with an AVM? Would this loan even happen? Who’s the better choice, appraiser or machine?
Property three is an older brick ranch with an attached 1 car carport with a full finished basement. This home has the original quality materials in the kitchen cabinetry and ceramic tile baths. Updates include newer counter tops and vanities. Here are the results:
The AVM values range from 86.1% to 112.0% of the appraiser’s opinion of value. Only two AVM results were higher than the appraiser’s opinion of value and two were within 1%, but still under. Let’s think about this. The loan officer, who works on a commission that is a percentage of the loan amount, wants a higher value. The bank wants to make a loan and wants the higher value. The homeowner has an incentive in a lower loan to value for a lower interest rate or an incentive to maximize the cash out, so the homeowner wants the higher value. Who’s the better choice, appraiser or machine?
Property four is an older tri-level style home with all the original finishes in fair condition with deferred maintenance. This home has an attached 2 car garage. Here are the results:
The AVM values range from 78.8% to 118.0% of the appraiser’s opinion of value. Five AVM results were higher than the appraiser’s opinion of value and two were within 2%. Let’s think about this. The loan officer, who works on a commission that is a percentage of the loan amount, wants a higher value. The bank wants to make a loan and wants the higher value, however, the bank also wants to ensure they are protected with adequate collateral in the event of a default. The homeowner has an incentive in a lower loan to value for a lower interest rate or an incentive to maximize the cash out, so the homeowner wants the higher value. Five of the eight AVM results were higher than the appraiser’s opinion of value. However, the condition of the property is impacting the market value. Would the lender close this loan with an AVM knowing the condition? Who’s the better choice, appraiser or machine?
Recently the US House of Representatives, Committee of Financial Services met to discuss “Modernizing Appraisals.” There is no doubt AVMs are here to stay and this simple comparison of the above properties is no where close to concrete evidence if an AVM is good, good enough or just as bad. Like it or not, right or wrong, appraisers are being subjected to big data once more. The question to ponder is how will you, the appraiser, move forward.
Happy Holidays!
- The New Con: Hybrids, Waivers & AMCs Threaten Public Trust - December 16, 2024
- VaCAP Supports Shane Lanham’s Legal Fight - September 10, 2024
- It’s Just Responsible Journalism! - February 21, 2024
What a crazy whacky profession, I am sure if the the job market was solid enough that enough appraisers would leave and tell them where they could really shove this profession and there AVM at !
We have to face the fact that AVMs are here to stay. The amount of use and dependence on them is the question. I have to believe that there will be some test programs conducted and some heavy analysis done on the results. It will have to affect the lender’s ability to make mortgages and the overall profitability of the project. From the data above, it doesn’t look like there is a problem with homes being overvalued but, just wait until a cracked slab, water leak, rodent infestation, poor condition, lack of updates, or some other problem arises and the lender has to eat the mortgage. Sounds risky to me! However…Wells Fargo has been using desktop appraisals for some time now and continues to do them. Must be a low risk issue for them.
But it looks like they are being sued AGAIN….
Yes Diana, you’re right. It’ll probably never end either. One lender or another will be in the hot seat for something stupid. WF is still soliciting fee appraisers and on the application form, it asks if you would be interested in performing Desk Appraisals for them. did a few some years ago but told them to take me off their list. I was getting 20-30 of them in a batch, with a due date in 3-4 days. I was working so fast on them that it made me nervous. They would review them just like the normal 1004s and with that many over a short time frame, I wasn’t comfortable that I didn’t miss something. Not good.
They get hit with a putback and then write off the loss. The larger they are, the more subsidy they get. It’s not a low risk issue, it’s more reflective of an increased risk model which is supported by taxpayer write off allowances via volume scale. Never objectify financial risk by volume. If it was your dollar, would that be acceptable process to you?
In most cases the AVM does not even have the GLA, room counts or perhaps major contributory assets even counted correctly! AVM’s may give a snap shot or ball park figure, but the appraiser will always be needed to physically inspect the property and surrounding market to get the true market value. AVMs can not see the cell tower, smell the cat odor, see the dated interior or notice that the back roof is falling in.
Just my 2 cents.
Mee I agree, but it appears they just don’t care.
Such analysis was not opted for in development costs when they ordered the software from the tech jockey. The AVM is not like for the public good, the people presenting them need to make money too. How to extrapolate interpolated data into hard cash….. Better pump up the coefficients and have stronger page placement in google! How do we clickbait this competition properly…. We need higher numbers, stat!
appraisers have argued for decades, that the result of an appraisal should be a value range and not a single value. “no appraiser knows exactly what the value of a house is, and single value appraisals dont make any sense!”
thanks to AVM’s, it looks like appraisers finally got what they wanted.
If only as a borrower I could borrow in such a variable range… The rubber has to hit the road somewhere. Should all refi’s really be heloc’s? LOL?
everyone wanted a value range, now there is one. in this example, there are eight sources, and thus a value range.
here is my question though. where in the range does the loan fall? banks dont give loans based on ranges, they give out loans based on a single number. for example, they cut the seller a check for $100,000, or pay off bills in a refi for $100,000. banks dont cut checks for a range.
if the banks are given a range, how will they determine what the single number is? they have to cut a check for something.
i think it is pretty accurate to say that they would push for the higher end of the range to boost their own profits, and go right back to their old habits of screwing the borrower. wouldnt you say?
oops. all of a sudden, all house values are over-inflated again.
welcome back to 2008.
Trash in Trash out is what I always say. Just sit back for the next mortgage crisis and watch them try to blame us…lol I’ll send them right back to themselves with their AVM’s. You know this is gonna happen.
Re: My Experience With AI (Artificial Ignorance)
I posted a home as a For Sale By Owner on Zillow a little over a year ago for $150,000. They promptly found the need to post their own Zestimate on the ad stating the home was worth $125,000. I then called them and told them I was a retired appraisers with 20 year’s of experience. They said send us comparable sales. I forwarded them 3 recent sales that were nearly identical within 1 block of the home. They were the strongest comparable sales that I recall seeing in a small town in my life. Zillow soon responded by dropping their Zestimate to around $120,000 and over the next couple of months dropped the Zestimate of value to $110,000. The home sold within a year in bidding war for $200,000. Zillow responded by raising their Zestimate to $125,000 again. I receive emails from them every few weeks watching them slowly dropping their Zestimate just like they did before.
Talk about a sad sack business! I could derive more accurate values by drawing random numbers from a jar. Meanwhile, the public eats it up because it’s free and easy.
Retired, entering the wide wide world if internet meme’s. Gotta love it! Strangely enough zillow is under market in CO sometimes. I think the analysis points are tied to population density, but I may be wrong. I’d like to know if zillow is accurate in highrises or if the variance is more or less depending on housing type. That would be a fascinating analysis to know how much data inconsistency these companies cover up, and how they manage base variability in analysis peramiter setting by categorical housing type. If AVM’s replace appraisers in the regulatory structure, a brand new regulatory structure will have to be developed to regulate the AVM’s. Think they’ll require the software programmers to be state licensed?
For AVM’s, modernization of popular building methods is going to alter the consideration points for AVM acceptability.Â
The modern build method incorporates much higher variance in both build style, lot location, and options for buy inclusions. The more ‘modernized’ builders get with stylistic options, the less reliable ‘modernized’ avm’s will become.
The days of silver gold platinum build choice options are over. These days buyers design new homes from the comfort of their computers in complex ‘build gallery’ software applications and literally itemize everything down to the kitchen sink.Â
I mean quite literally, the base variability within a single suburbia neighborhood jumped from what used to be narrow ranges, into something with dramatically more variability for price, character, and material qualities.
There is simply not enough reliable data for AVM’s to operate effectively. Perhaps if FNMA revised the 1004 to a 10 page detail form, MLS followed, and then 10 years down the line. I would then caution that differences in housing locals with such increased variability points will merely result in a brand new array of unreliable data disassociation points and the lender will then still need to….. Drumroll….. Have a live human appraiser there to properly assess the market, comparative qualities, effective ages, apply the legal permissibility question, h&b land, and other analyses necessary to come about ‘credible valuation results’. The argument continues to rest on the basic fundamental principal of; credible valuation results. Not sometimes, not most of the time, not all but once, but every single last time, no exceptions.
Pop quiz: Are tech programmers whom set analysis peramiters for AVM systems also appropriately qualified to assess H&B land, changing zoning, altered assessor approach, apply tests of legal allowability per individual county? And what is the software system in place to allow such fluid application of ever changing data integration on a mass national scale with each individual county and markets and local jurisdictional rules contained therein?
It’s the same old thing in real estate, pay no attention to the man behind the curtain. Who is this mythical god like person whom is so good at analyzing real estate with computer softwares that they can recreate the entire array of personalized human services and legal analysis points in a single program? Better yet, who are the appraisers whom posses such godlike skills compared to me they can do that in 2 hours w/ staff? The AVM and integrated analysis systems proliferate the way they do because truth be told, they’re rudimentary data analysis systems which are both cheap to create and apply simple probability logic to scraped assessor data via rss feed extract, or something similarly basic. Do they hire appraisers and assessors to consult? Legal principals have a limited role in data coefficient variables, often rooted in rudimentary 1-5 range assessment categorization or interpolated data and zoning description analysis. AVM is synonymous with interpolated data. It’s a forecast with infinite variability. You need a human to achieve a logical result with consistency.
AVM’s serve a valid role as a free market alternative. They would not serve a valid role as a private enterprise tool to corner the valuation market and drive human appraisers out of business. In an actual free market system where lenders took their own risk in full, the use of an AVM in securitization of collateralization would be a free market question, not a regulatory one. Think they’d prioritize speed of closing over adequate collaterlization with their own money? The question and answer to the AVM argument continues to rest on the credible valuation results principal, regardless of whom shoulders the risk, the taxpayer or the lender. Of course lenders whom control their own regulatory environment would prioritize speed over security. What is left to be argued about here? The structure of the regulatory environment itself?
Hi Baggins, no changes were made to the format of the site in more than a year. Try a different browser or clear cache by pressing the CONTROL and F5 keys at the same time a few times.
Cntrl Shft R successfully cleared out my previous ready to post comment. LOL. But no, no difference with a page refresh. Last week the ‘more’ column was absent and the content areas occupied right 2/3rds and page was wide and read really clean and better. Now article and read is confined to center 1/3rd only. FYI this is an interesting tech site; learn what your browser is requesting without telling you… Click in, do a few tasks, listen to commands, page refresh, move mouse, click button, and then achievements list will appear to click into. Then you can see all the things your browser asked to do you may not have been aware of. Remarkably interesting for being so simple.
https://clickclickclick.click
Oh yeah, and I too want the higher value! LOL. Think of it like cars and bluebooks. You don’t just sell with a blindfold on because the bluebook said so. What about my awesome punisher themed tire caps and deluxe make you deaf sound system? How about my $5000 bill for total new front steering package, suspension, new chain and tune up? That’s a figure higher than blue book. Is there an AVM for reo vs independent? Should there be? AVM and real property is similar to the blue book approach in this regard. AVM is an estimate, anything unique results in deviance from the median. Cash contribution matters and this whole vehicle is not 15 years old, just parts of it. Until AVM’s run like the blue book and force homeowners to enter like a thousand data points before presenting a number, I’m not buying into the legitimacy of AVM software as a viable alternative for a human appraiser. I mean come on, the blue book set the standard for these automatic value approaches did it not? Is it logical to regulate the AVM for housing this early in the game and act like it’s necessary to enter all this data to get vehicle value information, but no data needs entered by the AVM system to get housing estimates? Do you really believe that homeowners always are good boys and girls and get permits for improvement efforts so AVMs can track that via county data? Do you really think the assessor and builder departments make a point to keep all data ultra current and prioritize data input for third party data users as the main operational responsibility? The more you know about real estate, the longer it takes to furnish credible valuation results.
HAHAHAHA…Just ran a few AVM’s on my house and here’s the results: Zillow: $230K, SmartZip: $237K, Eppraisal: $245K, Redfin $260 and I’ll take the higher one Bob…..HAHAHA Guess what they all include the basement sf as living space. Recently sold homes nearby show 2 that closed almost two years ago and 2 others were REO’s! This is the info the banks are using. Love it!
Welcome to the Matrix mls program, combined gla first, sorted agla, like a distant second. For the variable values, well there has been this big deal about consumers needing ‘options’. You wanted a range, you got one, and two, and three, and four, how many of these out the box AVM programs are out there again? Perhaps when it’s labeled as Realquest with the red corelogic box, it’s more credible? I don’t know, which industry partner corporate brand of avm do you trust the most? LOL.
too funny.
so again, i am curious.
lets say you (or anyone) are refinancing their house, 150k is owed on the house and the borrower wants some cash out. where in that range of $230-260k is the value of the house? that bank has to cut the borrower a check for something. if the bank chooses 260k, the borrower gets a much bigger check than if the bank chooses 230k. will the bank choose the mean of 245k?
how is that going to work? will the high or low value be based on a borrowers credit rating, or will the high or low value be based purely on the greediness of the banks. in the end they will need a hard number to come from somewhere to write the borrower a check.
does anyone know?
if the banks will always be using the higher number just to make more money, (you know they will), then it wont take long for all houses to be upside down again, and say hello to 2008 again.
Yeah Bubba. Sounds like we’ll need a regulatory engine regarding the sound use of avm’s. Sort of nullfies the need for them, considering one already exists for appraisers whom have a long track record of doing a better job. You know, comp shopping was a problem, but calling your appraiser to comp search was a valid approach method for ethical persons. Separation from loan production causes long delays and assignment inconsistency woes. They could have simply fired every broker whom ever ‘comp shopped’ and that would have been the end of it.
I completed both a detached condominium property and an attached home property (non-condominium form of ownership) this week and as always the public records data and MLS data was a mess. How does an AVM account for the age restrictions (55+), when its not disclosed in public records. Garbage in equals garbage out.
This article concentrates on the Value differences between the appraisers and the 8 AVMs and makes a good point however what about the property details. With just the limited amount of details provided there are numerous mistakes in description of the homes.
Comp # 1 All AVMs had incorrect bath counts, 6 did not show the basement.
Comp # 3Â Â Four AVMs has incorrect sq ft of living area, 6 did not show basement, all bath counts were incorrect.
Comp # 4Â All AVMs show incorrect Sq Ft of living area and one shows a basement, incorrect.
If any of these errors were in an appraisers report they would need to be corrected however apparently no one need to correct errors in the AVM report, they only care about the value.
If they want creditable results they need to have creditable information.
My take varies from the VaCap authors. Lets assume the examples above are “representative” of the whole universe of AVMs.
Sample One-likely the loan did not get made at the anticipated loan to value ratio since the majority of indications were well below the appraisal. A single AVM being high would not over ride the risk of lending above the safe or expected investor LTV limit.
Sample Two probably happened at $222,000 OR Underwriter over rode the AVM and called it $225K…maybe even $227K Its well within any reasonable range of variance.
Sample Three– is another “probably could have been made as planned loan EXCEPT that the AVMs from the Banks were ALL low. Very low. Quote loan fees for one LTV but deliver a much lower LTV product to investors to increase net proceeds to the bank? Happens all the time. Consumer fraud perpetuated and facilitated by the AVMS in this example.
Sample Four -Probably made loan at the 105% to 115% of appraised value. AVM failed to consider property condition. Assumed it to be average. Buyer may or may not have funds to raise property from fair to average condition. Chance of default statistically higher. Taxpayers once again take it in the shorts when this loan goes bad. Planned 80% to 90% LTV far less than actual. Mortgage Insurance Companies take loss on this one initially; then taxpayers if trend is wide spread.
In summary: 25% either killed the deal or left consumers upset by far below market value based LTVs.
25% appear to be reasonably safe as a coincidence (provided they are close to area or national medians), 25% facilitate consumer fraud by banks allowing them to charge for risky 90%-95% LTVs while only lending 9% or 10%+- less (non PMI ranges BUT with PMI anyway! ), and 25% leave FDIC or taxpayers holding the bag again when the loans go bad.
So, at best 1 in 4 has a chance of being an acceptable risk. Certainly FNMA will leap at that!…Their investors? Not so much.
THAT is why they have to be lied to with AVM/Hybrids that look like and pretend to be real appraisals. As always, follow the money folks.
Awesome Mr Ford. And an extra tidbit for you;
Scenario 5: Appraiser completes order, and is reliant on automated development and statistical analysis program assistance. They fail to properly recognize and apply substantial adjustments for quality contributions like utility updating, quality materials, decks, porches, outbuildings, etc. This happens because they’re unlikely to be able to easily reconcile automated trending data results with real world cash out of pocket contribution costs to home. Does not compute or attain reasonable distribution of total applied adjustment balance, in a strictly statistical based analysis setting. The unfortunate consequence of presuming real property value is strictly tied to numerical based statistical analysis. The whole enchilada for both avm, and automated developmental assistance for full appraisal orders rests on numerical measurements, rather than artful and meaningful generalized and individual expressions of grid adjustments for real property characteristics that can not be measured in numerical data or statistics.
Scenario 6: Manually oriented appraisers like myself step in with what is quite often a higher value opinion which is a result of more careful, much more time consuming manual analysis. Backed by thorough knowledge of real property, construction methods, participant behavioral patterns in the larger market, and real world current costs of labor based improvements. Guess what, I do not have a cell phone, I don’t use spreadsheets, I am paper use intensive, I don’t use mobile devices, my reports never short cut with ‘see addenda’ or boilerplate, I demand direct assignment, maintain minimum fee expectations, maintain longer turn time expectations 1 wk or more, and I don’t use any outsourced assistance, digital or human.
On a related note, you’d have to be plainly out of your mind to trust an unlicensed person to perform the appraisal inspection. I currently have a letter to CJ at Mercury, demanding my Mercury profile carry an option for me to note I do not use unlicensed inspectors. I think it’s important that lenders be able to know who does and who does not use unlicensed inspectors from this point forward. This critical rule change just nullified the turn time ratings system, in my personal opinion.
And that’s the difference between a human and a machine. The data speaks for itself. I’m like John Henry over here, and I did beat that steam hammer, repeatedly.
PS-Your tax dollars at work ” Protecting the Public Trust” yet again
Baggins-Absolutely right. Particularly pleased to see the reminder about “artful” application of relevant ‘market data’Â other than or rather than statistical analyses and regression that only captures 70% of significant characteristics and then apportions the other 30%Â where it doesn’t belong.
Thanks Mike. Your comment illustrates why you’re a better appraiser than me though. You capture the message in simple terms. Adequately expressing what took me several paragraphs. Ha! I really honestly do take the John Henry approach. We’ve all got to stick in our own competency range, and utilize our own best skills to present credible valuation results. Who programmed that algorithm and are they trustworthy? Probably another in a long line of non appraiser tech guys whom may not have yet repaid their student loans and attained a mortgage on their own first home. It’s just a job to them, and likely not the job they initially went to tech school for. Amc’s have enough tech guys to create video games, but very few appraisers on hand. It’s like a ghost ship, crewed by the damned.
Baggs, you DO realize how funny your last post is, don’t you? Suggesting I EVER wrote anything briefly or concisely! I think you probably had half the regular blog readers peeing themselves with laughter over that one. (The other half are still sitting in stunned silence)
Hey, thanks Mike. If the topic is interesting, I normally read through the entire article and all posts, before trolling through the commentary with one or several posts of my own. For legistic and legal issues, I usually take the time to read through your posts completely. My highest hope is that hopefully someone somewhere at least skims through mine. Got to have fun with it.
I think this blog and ensuing posting calamaties would make a great printed monthly publication. Surely the other guys stop by here now and then in hopes of scooping something or another, scraping a story out of thin air. I’d like to think I give real journalists something to strive for. Publications which stand on merit are not afraid to allow open posting, and don’t need the heavy handed moderation approach. I’m loving the format here because disguss based blogs are really horrendously managed and heavily censored. The blogging will continue until the appraisal management methods improve.
Our city Assessor says they arrive at property values using algorithms, I say garbage in, garbage out. Â I’m sure the algorithm can determine the physical condition of the improvements, or maybe they use a drone. That will be the next step in appraising. Heaven help the next generation of appraisers, if there is one.