Category:	Science and Technology
Domain:	Mathematics
Keywords:	Information and Knowledge - mathematics, economics
Outlook:	Mathematical tools for discovering patterns in large databases, along with stochastic modeling, could contribute to better decision making in a range of fields.
Summary Analysis:	Technologies are increasingly available for collecting and storing massive amounts of data, whether about the movement of the equity markets, drug treatment outcomes, or changing air quality. Mathematical techniques will increasingly enable mining of this data to provide a basis for making effective decisions. In the financial world, for example, large quantities of high-frequency data need to be processed effectively and efficiently so that opportunities can be quickly identified and acted upon 24 hours a day, 7 days a week. Rapid implementation of new mathematical models and algorithms, along with an expanding computing infrastructure, has the potential to meet this need. Statistical pattern recognition could be used to identify arbitrage opportunities and improve speculative trading. Integration of techniques from mathematical finance and insurance could lead to new products: catastrophe bonds, futures, and options; equity-linked life insurance; credit risk derivatives; and mortgage-backed securities. More financial decision making could be automated and made more effective by superior models, analytical tools, and processing systems. Advances in data mining are also likely to be made in the areas such as national security and crime detection. Advances will possibly come from the development of human-machine-synergistic methods, designed to maximize the different abilities of people and machines. This will require collaborative work between mathematicians and computer scientists and psychologists, in many ways parallel to the existing collaborations between mathematicians and computers scientists and biologists and chemists on projects such as genome mapping and pharmacological research. The highly visible collapses of Enron and LTCM have brought mathematics-based business into disrepute. In the case of Enron, however, the problems seem to have been associated with people and culture rather than with the underlying equations.

	Implications:	More effective decision making based on analysis of large quantities of data Expansion of the array of financial products on the market
	Early Indicators:	Current application of transaction analysis for fraud detection and product recommendation systems over large data sets
	What to Watch:	Data-mining techniques are applied to massive collections of environmental sensor data to find causal relationships between industrial pollutants and global warming. Data mining is applied increasingly to the massive amount of data associated with health care, concerning patients, procedures, and drug treatment outcomes. Mathematicians and physicists are hired at an increasing rate by investment banks and traders.
	Parallels/Precedents:	Enron's and Long Term Capital Management's attempts to combine academic modeling of financial markets with trading expertise, despite the ignominious end of both enterpises.
	Enablers/drivers:	Effective testing of mathematical models in real-world environments Growth of grid computing Development of massive distributed storage devices Continued development of stochastic algorithms, data-mining and knowledge discovery algorithms, and modeling languages
	Leaders:	Institutions: Department of Homeland Security (the proposed Analysis, Dissemination, Visualization, Insight, and Semantic Enhancement system) National Security Agency KDNuggets, a leading site and newsletter on Data Mining and Knowledge Discovery [link] Autonomy, world-leading data mining software spun out from Cambridge University [link] University of Namur, Belgium [link] University of Technology, Sydney, Australia [link] Wessex Institute of Technology [link] European Bioinformatics Institute [link] University of Kent [link] University of Manchester, School of Informatics [link] UK e-Science Data Mining Special Interest Group [link]
	Figures:
	Sources:	"Data Mining and Discovery." American Association for Artificial Intelligence. [link] Kargupta, H., A. Joshi, K. Sivakumar, and Y. Yesha. "Abstract of Data Mining: Next Generation Challenges and Future Directions" UMBC [link] Agosta, Lou. "The Future of Data Mining -- Predictive Analysis." DMReview magazine (August 2004). [link] Basterfield, David. "Directions in Computational Finance." OGI. [link] "Mathematics: Giving Industry the Edge." Faraday Partnership for Industrial Mathematics (April 2002). [link] McLean, B. and P. Elkind. 2003. "Partners in Crime." Fortune.com 27 Oct. [link] "Case Study: LTCM - Long-Term Capital Management." ERisk.com. [link] "NOVA: Trillion Dollar Bet." PBS. Aired 8 February 2000. [link] Data Mining Techniques (statistical explanation) [link] The Data Mine [link]

	At A Glance:	When:	3–10 years
		Where:	Global
		How Fast:	Years
		Likelihood:	High
		Impact:	Medium-High
		Controversy:	Low

Related Outlooks: