Human Behavior Modeling Failures

This blog entry entitled Millenium bridge, endogeneity and risk management features two examples of faulty modeling in bridges and value-at-risk models (VaR) that take their roots in their not taking into account human behavior. One but wonders if people were dealing with unknown unknowns when the initial modeling was performed. In a different direction, when one deals with models and human behavior there is always the possibility for subgroups to game the system and make the intended modeling worthless. Here is an example related to ranking academics and researchers.

RMM Example #3: Nuclear Proliferation and Terrorism Risk Assessment.

In the U.S, the nuclear fuel cycle is termed 'one through' as nuclear fuel passes in nuclear power plant only once before being discarded. The debate on whether the country should be reprocessing some of these materials has been an ongoing discussion as early as the 1950's. The problematic is extremely complex and has many stakeholders at the table. A subset of the discussions include the fact that if the U.S. were to ever perform some reprocessing in their civilian fuel cycle, it would provide some grounds for other countries to do the same. For technical reasons, reprocessing is considered to be a good tool for proliferation because as soon as you allow for your nuclear fuel to be reprocessed, you  also open the door to the ability to extract material for "other purposes". Most countries are signatories of the Non Proliferation Treaty (NPT) and as you all know the IAEA is in charge of verifying compliance (all countries that are signatories are subject to yearly visits by IAEA staff) And so, a major technical effort in any type of Research and Development at the U.S. Department of Energy (and other countries that comply with the NPT) revolves around bringing technical solutions to some proliferation issues (as other proliferation issues are political in nature). As part of the recent DOE Nuclear Energy Enabling Technologies Program Workshop, there was an interesting subsection of the meeting dedicated to Proliferation and Terrorism Risk Assessment. I am not a specialist of this particular area, so I will just feature the material presented there for illustration purposes as they all represents some issues commonly found in difficult to solve RMM examples. From the presentation introducing the assessment here were the Goals and Objectives of this sub-meeting:

Solicit views of a broad cross‐section of stakeholders on the following questions:

• Would you favor an expanded R&D effort on proliferation and terrorism risk assessment? Why or why not?
• In what ways have current methodologies been useful, how might R&D make them more effective?
• If an expanded R&D program was initiated, what are promising areas for R&D, areas less worthwhile, and what mix of topics would best balance an expanded R&D portfolio?
• If an expanded R&D program was initiated, what cautions and recommendations should DOE‐NE consider as the program is planned and implemented?

Panel presentations to stimulate the discussion will address:

• Existing state‐of‐the‐art tools and methodologies for proliferation and terrorism risk assessment.
• The potential impact of improved tools and methodologies as well as factors that should be carefully considered in their use and any further development efforts.
• Identification of the challenges, areas for improvement, and gaps associated with broader utilization and acceptance of proliferation and terrorism risk assessment tools and methodologies.
• Identification of promising opportunities for R&D. Broad discussion/input is essential, active participation of all session attendees will: Provide important perspectives on proliferation and terrorism risk assessment R&D and ultimately strengthen capabilities for supporting NE’s development of new reactor and fuel cycle technologies/concepts while minimizing proliferation and terrorism risks.

Why talk about this subject on RMM ? Well I was struck by the type of questions being asked to technical people as they looked like typical questions one would ask in the context of an RMM example.

In Robert Bari's presentation one can read:

"Conveying Results: In particular, what we know about what we do not know" sounded a little too much like the categories discussed in The Modeler's Known Unknowns and Unknown Knowns

In Proliferation Resistance and Proliferation Risk Analysis: Thoughts on a Path Forward by William S. Charlton, one can read:

I wonder about the type of modeling that goes into estimating uncertainties. Finally, in Bill Burchill's slides, one can read on the proliferation pathways the following:

Solution to the "Selling from Novosibirsk" business model riddle

Bernard Beauzamy (the owner of SCM SA) had set up a 500 euros prize for whoever could find a way for the business model riddle featured on Tuesday. Bernard tells me that nobody won the prize, here is the answer:

....The answers we received fall into two categories:

Those who want to send 10 000 left-hand gloves, and then 10 000 right-hand gloves, and declare them of value zero.

This answer does not make sense ! Do you think that the customs will be stupid enough not to observe that they see only left-hand gloves, and only later right-hand gloves ? They would confiscate both, and the sender would go to jail, for attempt to cheat the customs. Many years of jail !

Those who want to create a branch of the company in the destination country, and claim that they would evade the customs this way.

This answer does not make sense either ! It only reduces the profits, since one has to pay all the people in the local structure. And it changes nothing to the fact that customs tax the final selling price. If you have 10 intermediaries, you will have to give a salary to the ten, and the customer pays the same price, so the producer gets less money. In all circumstances, customs or not, the fewer intermediaries you have, the better you feel.

The answer, as we expected, was not found by anyone, because all our readers, by education or taste, want to build mathematical models, and this is a situation where no model is possible. It defies imagination and logics, and contradicts all existing economical models.

First of all, the solution looks impossible. If we sell each pair at its maximum price, that is 200 Rubles, the customs takes 160, we keep 40, and this is exactly equal to the production cost, so we have no benefit at all. It is even worse if we sell at a lower price.

The solution is this : we have to impose fabrication defects to 5 000 pairs (both left and right gloves). After that, we export 5 000 pairs, of which the left one is normal and the right one is defective (for example a big scar across one finger). These gloves are declared as "fabrication rejects", for a very small price, for instance 20 Rubles a pair. Note that selling and exporting "fabrication rejects" is quite ordinary and legal, and is common practice.

Then, next month, we do the converse : we export 5 000 pairs, of which the left one is defective and the right one is normal. We put all gloves together, and we get 5 000 pairs of normal gloves, which we sell at the maximum price. The total cost is 400 000 Rubles (fabrication), plus 160 000 Rubles (customs). The sales bring 1 000 000 Rubles, so we have a benefit of 840 000 Rubles. We can of course sell the defective gloves, just to have some receipts for the customs.

The ideal solution, but this is a remarkable industrial achievement is to program the fabrication machine so that it put defects on one pair out of two.

We observe that the solution is perfectly legal. Fabrication defects exist and are sold worlwide, for a low price. Each pair is perfectly declared at is correct value.

We said earlier that mathematical modeling is impossible. In fact, this example shows that all Nobel prizes given to economists since 1969 (year of creation of the prize) should be withdrawn, because they have no value at all.

Precisely, we see here that the notion of price is not mathematically well-defined. We cannot talk about the price of a glove, even not of a pair of gloves. We see that the price is not a continuous function, nor an increasing function, nor an additive function. The price of two objects together is not the sum of their individual prices. Still, the economists will build nice models, defective by all parts, and no reassembly can bring them any value !

Remember this : this is a true story, and the man who invented the solution did not know what a mathematical model was and did not have any degree at all…

[P.S: added Dec 4 (3:40PM CST): It's a riddle. One could make the point that this type of business model is not robust. All the countries in the world revise their own laws in order to effectively plug holes such as the one presented here. The ever growing sophistication / complexities of the tax systems in most countries reflects this adaptive behavior. If this system were robust it would be common business practice by now. It may have worked in some countries in the past however. The most important take away from this riddle is that the definition of the price of an item is indeed a difficult problem for which modeling is going to be tricky for a long time]

The RMM LinkedIn Group

As some of you may know, the Robust Mathematical Modeling blog has its own group on LinkedIn. Rodrigo Carvalho is our latest member which put our count to 21.

The 500 euros prize for the solution to the business model riddle ends today in about 7 hours. Good luck!

A Prize for Modeling a Business with Constraints: Selling from Novosibirsk

Sometimes, the model that is given to you has to be tweaked radically in order to explain the situation at hand. With this thought in mind, Bernard Beauzamy (the owner of SCM SA) has set up a 500 euros prize for whoever can find a way for the following business model to work (see below). The solutions should be sent before Friday December 3rd, 2010, 5 pm (Paris local time, that's GMT+1) to scm.sa@orange.fr.

Selling from Novosibirsk
A factory in Novosibirsk produces gloves. Each pair costs 40 rubles to produce, including everything : raw material, salary of workers, machines, transportation, and so on.
They produce 10,000 pairs and they want to sell them in a country where customs duties are 4/5 of the selling price. They cannot sell at a price higher than 200 rubles each pair, because of local competition and local buying power (at a higher price, nobody would buy). How do they manage to make a profit, and how much do they gain ?
One cannot cheat with the customs and corruption is forbidden. The price declared for the sale must be the true price. The exchange rate between currencies is assumed to be fixed and is not to be taken into account : everything is stated in the seller's currency (here the ruble). The solution should work repeatedly, any number of times, without violating any law, between any countries with normal customs.
Prize offered : 500 Euros, for the best answer received before Friday, December 3rd, 2010, 5 pm (Paris local time). Send answers to scm.sa@orange.fr. Answers may be written in English, French, Russian.

[Check the solution here]

Optimal according to what ?

Redefining optimal is a blog entry by some of the folks at the Department of Systems Biology at Harvard Medical School. It is very nicely written and includes some nice comments. The entry specifically points to a paper by Fernández Slezak D, Suárez C, Cecchi GA, Marshall G, & Stolovitzky G (2010) entitled When the optimal is not the best: parameter estimation in complex biological models (PloS one, 5 (10) PMID: 21049094). The abstract and conclusions read:

Abstract

BACKGROUND: The vast computational resources that became available during the past decade enabled the development and simulation of increasingly complex mathematical models of cancer growth. These models typically involve many free parameters whose determination is a substantial obstacle to model development. Direct measurement of biochemical parameters in vivo is often difficult and sometimes impracticable, while fitting them under data-poor conditions may result in biologically implausible values.

RESULTS: We discuss different methodological approaches to estimate parameters in complex biological models. We make use of the high computational power of the Blue Gene technology to perform an extensive study of the parameter space in a model of avascular tumor growth. We explicitly show that the landscape of the cost function used to optimize the model to the data has a very rugged surface in parameter space. This cost function has many local minima with unrealistic solutions, including the global minimum corresponding to the best fit.

CONCLUSIONS: The case studied in this paper shows one example in which model parameters that optimally fit the data are not necessarily the best ones from a biological point of view. To avoid force-fitting a model to a dataset, we propose that the best model parameters should be found by choosing, among suboptimal parameters, those that match criteria other than the ones used to fit the model. We also conclude that the model, data and optimization approach form a new complex system and point to the need of a theory that addresses this problem more generally.

Evidently, the post would have some relevance to compressive sensing if the model were to be linear, which it is not in this case.

The Modeler's Known Unknowns and Unknown Knowns

In Satyajit Das's Blog entitled "Fear & Loathing in Financial Products" one can read the following entry entitled WMD or what are derivatives 

....During the Iraqi conflict, Donald Rumsfeld, the US Defense Secretary, inadvertently stated a framework for understanding the modern world (12 February 2002 Department of Defense News Briefing). The framework perfectly fits the derivatives business. There were “known knowns” – these were things that you knew you knew. There were “known unknowns” – these were things that you knew you did not know. Then, there were “unknown knowns” – things that you did not know you knew. Finally, there were “unknown unknowns” – things that you did not know you did not know...

Then Satyajit goes on to clarify this term a little further:

....In most businesses, the nature of the product is a known known. We do not spend a lot of time debating the use of or our need for a pair of shoes. We also understand our choices – lace up or slip-on, black or brown. I speak, of course, of men’s shoes here. Women’s shoes, well, they are closer to derivatives. Derivatives are more complex. You may not know that you need the product until you saw it – an unknown known. You probably haven’t got the faintest idea of what a double knockout currency option with rebate is or does – a known unknown. What should you pay for this particular item? Definitely, unknown unknown. Derivatives are similar to a Manolo Blahnik or Jimmy Choo pair of women’s shoes....

There is also a word for the last one: Unk-Unk 

n. especially in engineering, something, such as a problem, that has not been and could not have been imagined or anticipated; an unknown unknown.

but I am not sure it fits with the previous definition. Looking up wikipedia, we have:

In epistemology and decision theory, the term unknown unknown refers to circumstances or outcomes that were not conceived of by an observer at a given point in time. The meaning of the term becomes more clear when it is contrasted with the known unknown, which refers to circumstances or outcomes that are known to be possible, but it is unknown whether or not they will be realized. The term is used in project planning and decision analysis to explain that any model of the future can only be informed by information that is currently available to the observer and, as such, faces substantial limitations and unknown risk.

How are these notions applicable to Robust Mathematical Modeling ?  John Cook reminded me recently of the Titanic effect presented initially in Jerry Weinberg's  Secrets of Consulting: A Guide to Giving and Getting Advice Successfully 

The thought that disaster is impossible often leads to an unthinkable disaster.

When modeling a complex  reality, we always strive to make the problem simple and then expect to build a more complex and realistic idealization of that problem. But in the end, even the most complex model is still an idealization of sorts. So every time I get to read about a disaster, grounded in some engineering mistake, I always wonder which part was the known unknown, the unknown known and the unknown unknown.

One of these moment that left me thinking happened back in February 2003 during last flight of the Space Shuttle Columbia. As you know, the spacecraft disintegrated over Texas. It did so because a piece of foam had hit one of its wings fifteen days earlier at launch. That explanation relied on a footage of the launch showing a piece of foam slowly falling from the main booster onto the edge of the wing of the Orbiter. Fifteen days later, when the Orbiter came back to land in Florida, it had a large hole that enabled air at a speed of Mach 17 to enter the inside of the wing, damaging it and thereby destroying the spacecraft. The remains of our experiment showed the temperature had reached well over 600C for a long period of time.

I was in the room next to the MCC during STS-107 as we were flying an instrument on top of the orbiter. We watched the take-off, we listened to all the conversations in the com loop between MCC and the astronauts during the fifteen days it flew (we asked some of the astronauts to manipulate our instrument at one point). At no time was there a hint of a possible mishap. We learned afterwards that even engineers, in the room next door and who had doubts, had requested  imagery from spy sats (but management canceled that request). However what was the most revealing after the tragedy was that I specifically recall that nobody around me could conceive the foam could have been making that much damage. None of us thought the speed differential between the foam and the spacecraft at launch could be that large. According to a simple computation based on the video of the launch, a half pound piece of foam hit the leading edge of the Orbiter's wing at an estimated speed of  924 fps or 1060 km/hr. It's always the square of the velocity that kills you.

More photos of the test performed in San Antonio on July 7, 2003 to recreate what happened can be found here.

As one can see from the small movie above, all previous tests had been performed with small pieces of foam. Further, attention had always been focused on the tiles underneath the Orbiter - never on the leading edge of the wings-. The video of the test performed in San Antonio, several months later, had everyone gasping in horror when the piece of foam opened a large hole in the leading edge of the wing. The most aggravating part of the story is that the Columbia flew for fifteen days with very little care about this issue while the Shuttle program had already seen take-offs with several near dangerous hits in the past.

On an unrelated issue, Ed Tufte also pointed out very clearly how the communication style using Powerpoint was a point of failure in the decision making process of deciding whether the foam strike  was a problem or not.. 

Of note, the communication to managment did not clearly delineate that there was absolutely no experience with such a large chunk of foam (experiments had been performed with 3 inch cube "bullets" vs an actual impactor with a volume of more than 1920 inch cube).

What were the known unknowns, the unknown knowns and the unknown unknowns in this instance ? First let me reframe all the categories of knowns/unknowns:for a modeler of reality or the engineers: With the help of wikipedia, let me summarize them as follows: To a modeler

• the known known refers to circumstances or outcomes that are known to be possible, it is known that they will be realized with some probability (based on adequate experiments/data)
• the known unknown refers to circumstances or outcomes that are known to be possible, but it is unknown whether or not they will be realized (no data or very low probability/extreme events).
• the unknown unknown refers to circumstances or outcomes that were not conceived of by a modeler at a given point in time.
• the unknown known refers to circumstances or outcome a modeler intentionally refuse to acknowledge that he/she knows

Looking back at the Columbia mishap, how can we categorize the different mistakes made:

• the foam hitting the RCC was a unknown known to the modelers. People had done their homework and knew that:
• falling pieces were hitting the orbiter at every launch
• some launches had bad tile destruction because of falling pieces
The engineers went through a series of tests that they eventually put in a database. Most past foam hits fell in the known knowns, as pieces of foam were clearly fitting dimensions used in the database. They knew foam could fall on the RCC instead of the tile yet did not do tests or felt the tests were necessary. At issue is really the fact that there was an assumption that the RCC was tougher than tiles. It actually is more brittle but then a lot of things are brittle when hit with a large chunk of something at 1000 km/hr.
•  The speed of the foam was also a known known to the modelers. It could be computed right after the launch and was within the range listed in the database mentioned above.
• A known unknown to the modeler was the impact effect of a very large piece of foam on the leading edge of the wing. This is reflected in the size fragments used in the database. There was simply no data.
• An unknown unknown to the manager and the rest of the world was the eventual demise of the whole spacecraft fifteen days later due to this impact. To the modeler, I believe the demise of the spacecraft was rather a unknown known.

Unknown unknowns are clearly outside of most modeling for a multitude of reasons. Robust mathematical modeling ought to provide some warning about known unknowns and most importantly provide a framework for not allowing unknown knowns to go unnoticed by either the engineers or their management.

[1] Beautiful Evidence, Edward Tufte.