Should Appraisers Study Statistics?
- Bias. It’s Easier to Blame the Messenger - April 20, 2022
- Hocus Pocus Magic Focus? - March 24, 2022
- Opinion, Estimate, or Prediction? - September 8, 2021
Yet what was (and still is) presented in appraiser education is the old inferential statistics. Difficult statistics. Convoluted statistics.
Statistics for appraisers started to become a hot topic some 15+ years ago.
I was more involved with the Appraisal Institute then, and often argued that with the changes in data sources — appraisers would have to learn to apply statistics. Around 2003, the AQB (Appraiser Qualifications Board) revised qualifying education to include the requirement for Finance, Statistics, and Valuation Modeling class. Although I was not solicited to write this class, I was put on the development team, which reviewed and suggested edits and improvements. The class had little on valuation modeling, and a lot about non-useful inferential statistics. This disturbed me. I later declined to teach this class as my integrity was challenged.
Although my background includes many graduate-level statistics classes, my greater influence was from the world of micro-economics and econometrics, from both San Diego State, and The University of California in San Diego. In particular, the classes at SDSU were oriented toward applied econometrics. In each case, my appraiser mind-set found the realities of how “statistics” applied to valuation to be quite different from what ended up in appraiser qualifying education.
In particular, I was already used to the fact that for both general and residential work, the data available to me included all, or substantially all, the sales in my area. I benefited from the fact that San Diego public records, assessor information, and even GIS (geographic information) was also ahead of data availability and quality in most other parts of the country. But one thing stood out. There was a conflict between what I was being taught in most of the PhD-level stats classes, and the reality of what I saw as a practicing appraiser.
The conflict was simple, yet convoluted by a belief system. The confusion mostly has to do with the difference between a sample, and a population.
The reality was that for appraisal, the relevant data set is the competitive sales. It’s the set of sold properties. This is the population. It’s not all the houses in a neighborhood. It’s not all the income properties in a city or a state. It is only those that sold, and are actually options for a potential buyer! Houses in a neighborhood are no more part of the market than the apple in my refrigerator is part of the market for apples! It is not.
This I learned in econometrics classes: You have to deal with actual transactions. Real data. Not all the hypothetical sales that could have taken place if they had been on the market. No make-believe.
Yet what was (and still is) presented in appraiser education is the old inferential statistics. Difficult statistics. Convoluted statistics. The statistics which assume you do not have most all of the sales data. The statistics which pretend natural variation was somehow equivalent to the random sample needed to make inferential statics valid.
What started to be taught was wrong on several counts:
- That the population was some imaginary market bigger than actual competing sales.
- That appraiser-judgement selection of comps was kinda-like a random sample.
- That statistical tests would then show how good your model was.
The reality is different:
- The population is the CMS© (Competitive Market Segment).
- Appraiser judgment is the polar opposite of random selection.
- Statistical tests show only how well a random sample represents a population.
In litigation practice, I have seen this misuse of “statistics” repeatedly. This misuse has no place in legal proceedings.
It’s time for our appraisal organizations to recognize the laws surrounding these fallacies, and cease teaching what has been fully debunked in a statement by the American Statistical Association itself.
Now is the time.
While much of this is true, between the paragraphs where the author tries to tell us how advanced and educated he is, he fails to mention some basic, common-sense ideas. While his thoughts on “population” are valid, they only apply, in part, to the valuation section of an appraisal- not the entire analysis. Real estate is not bought and sold by robots. It is bought and sold by human beings and that’s why appraisers still appraise houses and not robots. People buy things and pay certain amounts for irrational reasons that cannot be quantified by statistics but can be grasped by the human mind over a computer. Real estate appraisal is partly a science but it is more a SOCIAL science than a statistical one. Academics who spend too much time in Excel may have forgotten this. If this author’s ideas are so valid, why isn’t he teaching his own appraisal statistics class to compete with others – if his ideas are valid, won’t he produce better appraisers? Or is it just one more “author” who has a bone to pick with a particular appraisal organization (that AppraiserBlogs was of course more than happy to publish in their ongoing war with said appraisal organization).
Thank you for sharing your thoughts. George Dell has earned the MAI, SRA, and ASA designations. He is a graduate of San Diego State University with extensive post-graduate work in Economics, Statistics, Mathematics, Finance, and Information Systems, Certificate level work in Environmental Management and Geographic Information Systems (GIS).
George has served at all levels of the Appraisal Institute, including the national board of directors, Appraisal Journal editorial board, Curriculum Committee, Technology Committee, specialty admissions project teams, peer review, education development, and numerous presentations at national and international conferences.
It is worth noting that we publish articles written by appraisers that may raise controversial issues. Articles on the other side of the controversy might appear subsequently. We do not censor articles on ideological grounds, and do publish articles we disagree with. Your assumption that we are in an ongoing war with an appraisal organization because we publish articles criticizing it, is just flawed.
If interested, you can write and submit an article defending “said appraisal organization” to be published here.
Yes, we’re all aware of Mr. Dell’s “qualifications” – he spent a good part of his article reiterating them to us. After all, it matters that it was “PhD” statistics work. I have many designations and qualification myself- however they pale in comparison to real world, day to day experience on the ground, not sitting behind a computer or in a classroom. And if you think we’ll believe your blog doesn’t have a bias against certain appraisal organizations, one has only to browse the article titles and comments by your regular contributors (or listen to your regular contributors in person at their “conferences”) to see that bias. I have only to look at the right hand border of this website and start reading article titles – “AI Ignored the Residential Appraiser”. Did you ask the AI for rebuttal comment on that article? Being a former officer of an organization (like Mr. Dell is) or even published in their journals doesn’t mean you aren’t biased against it now. I’ve made my point above about Mr. Dell’s ridiculous arguments- no “counter-article” is needed. I’m also not an officer of said appraisal organization nor is my job to “defend” them. Instead, I’ll go back to finishing up some reports using common-sense, real world solutions and not worry about an academic “theory” being disguised as “evidenced-based appraising” that has no basis in the reality on the ground. No one, anywhere will convince me that Mr. Dell’s stated goal of “Objective, science-based appraisals” is a worthy goal. It assumes a ridiculous idea that human home purchases can be analyzed strictly by science and that subjective opinion somehow has no place. If Mr. Dell has his way, the USPAP definition of “appraisal” would become “the precise mathematical conclusion to the infallible algorithm that determines exact market value based solely on the results of mathematical calculation”.
As stated in our previous comment, we do not censor articles on ideological grounds, and do publish articles we disagree with. We don’t chase articles or ask for rebuttals. We simply don’t have the time to do so. We are boots on the ground appraisers volunteering any extra time we have running this blog. It is up to appraisers, appraisal organization and others in the appraisal industry to contact us.
The Real World is VERY MUCH being taken over by market participants who rely on statistical analyses and Algorithms.
Ever heard of Zillow, Redfin or Open Door? Or how bout “Artificial” Intelligence and “Machine” learning? Are they relying on human appraisers to value properties? How many human beings can analyze Big Data in a matter of microseconds?
Mathematical calculations/formulas are being used every day to estimate value and market conditions.
I don’t like the way any of this is going but unfortunately for human appraisers it’s the NEW REALITY. The “good ole – social science days” of appraising are all but over.
It is not the new reality, it is the new thing that is being pushed right now just like it was pushed 20 years ago and then we had a crash and so on and so on. I will not participate in something that is not accurate and minimizes the role of human beings. If that means that I have to open a hot dog stand then so be it. Why are you so willing to let incompetent people tell you that something is reality just because they say it enough times? I thought appraisers had a little more fight left in them but apparently they are all just sheep now. “Nothing we can do. baaaaah!”
MKS, I feel your frustration, but you should know that there are many individual appraisers and appraisal organizations that are making progress towards a better profession. The hard reality is we are mostly independent business people who have families to provide for and our resources are by no means equal to those of the banks and amcs. Take a look at the state coalitions and the Network. Appraisers are there at both state and Federal levels. The Financial Services Committee from a few weeks ago had several boots on the ground appraisers present and opened the doors for future communication. Appraisers are at AARO, Appraisers are getting state legislation introduced and passed, Appraisers are making a difference. Could we do more, sure, but more appraisers need to step up for that to happen.
As someone who knows George Dell and the owner of this blog personally, I can tell you they are two of the most dedicated appraisers to the profession I know. Do I have to agree with everything they believe? Absolutely not, but civil debate and communication help us understand the other side and working together is how we move forward.
Just FYI, this was posted by National Association of Appraisers:
“Today the Di-Minimus threshold as requested by NCUA for when an appraisal is required was raised from $400,000 to $1,000,000. This is a real setback for appraisers as it will likely affect the commercial appraisal workload for commercial appraisers working in smaller or rural markets. I suspect it is only a matter of time before the commercial banks will likewise request an increase.
NAA has opposed the requested increase and will continue to oppose any future increases in the Di-Minimus for both commercial and residential properties.
In place of an appraisal, the lender is required to get an “Evaluation” as defined by FIRREA and the interim agency guideline. An Evaluation looks like an appraisal but does not require some elements required under USPAP.
The sad fact is that estimates show that over 70% of the valuations services being provided in the USA are by non-appraisers (Including Evaluations, AVM’s and related valuations tools).
What is your plan of action to stay relevant as an appraiser?”
The next crash is coming soon because of the junk lenders/amc’ are pushing, history tells us that. So I will be doing 25-30 (pre) foreclosures a month like was being done 2009-2014.
I was previously a Real Estate Agent in the early 2000’s when everyone (almost) was worried that it would no longer be a profession because of the internet. How that premonition work out?! Zillow and such will have their comeuppance when a wave of their customers start suing them after that crash that’s coming.
Thank you for letting me voice my opinion.
Loosely related, long list of mortgage and lender layoffs and shutterings, 07/2019.
https://www.thetruthaboutmortgage.com/a-list-of-recent-mortgage-closures-mergers-and-layoffs/
I find it sort of odd that there is all this math and analytical action floating around my position, but nobody publishes results for other appraisers to rely on. I liked this article. The point of journalism is to have an opinion. If nobody is speaking to power, then those people are biased towards said power base. Mr Dell sent me a class invite CE postcard. He’s been around.
To my embarrassment, I have often disagreed with George of data science. I say embarrassment because I think George is actually more often correct than the opposite.
George is among the first to concede that improperly used or applied statistics do not benefit anyone; nor is it the sole tool in the appraisers’ arsenal. Properly used in conjunction with more traditional methods, statistics do have their place; and MAY answer some questions for which answers are not otherwise readily discernible.
George is not saying in the above article that stats are the answer to all appraisal problems. What he IS saying is that stats-analysis as is usually taught or postulated, are unrealistic or MAY be unrealistic indicators without proper context.
I had the privilege of appearing on the same dais as George in San Antonio last November in an Appraisal Leadership Panel. He didn’t make a complete believer out of me; nor did he reinforce any of my negative statistical perceptions. The truth is as always, I was struck by this man’s integrity & his knowledge of the topic.
George DOES teach statistics courses, and he also has a firm called Data Science (I believe). I took a free course online from Princeton on statistics. I got out of it about what I put into it. [CourseRA dot com for those interested]
Now that I WANT to learn meaningful statistical applications in real estate, I cannot think of anyone I’d rather learn from. Maybe by paying for the course, I will pay more attention, and it will make more sense; though I don’t expect the statistical analysis to ever replace the ART that is also an integral part of RE appraisal. Many thanks to Mr. Dell for taking the time to write the above article.