From the July 01, 2012 issue of Futures Magazine • Subscribe!

Big Data: Manage it, don’t drown in it

Analysis and execution

For market professionals, collecting, storing and managing Big Data are increasingly necessary challenges, but these are only requisite steps to addressing the heart of the issue. Truly valuable Big Data solutions must allow for robust visualization of data that enables constructive analysis, whether it is in the analysis that powers trading decisions or the tools that provide a clear picture of risk. With this sort of solution in hand, it is possible to execute decisions with bottom-line impact more reliably. This is the greater challenge, requiring the synthesis of detailed analysis with big-picture thinking.

The complexity associated with the scale of market data in the futures industry can mean that signals are buried in noise or visible only when analysis incorporates an adequately large or diverse data set, so a pressing question arises: Do you have the in-house tools and talent to make sense of Big Data? For the individual trader, this means continuing to cultivate intimate market knowledge, but perhaps incorporating new sources of information and new data visualization tools into methodology as well.

To implement productive and reliable analytical systems, firms need savvy personnel who understand data nuances and can formulate the right questions (i.e., it is critical to have a sense of what you are looking for before plowing headlong into data). These may be individuals who are deeply rooted in mathematics and quantitative analysis, but it is important that the end result is a system that provides decision makers and business drivers, who may or may not be so fundamentally versed, with a broad, accessible and informative picture of complex data. Here, decision makers who necessarily are not versed highly in quantitative analysis avoid situations where they are presented solely with the end results of calculations and left to make choices based largely on trust. They are, rather, able to bring their own dynamic analysis and decision-making to bear.

Increasingly, advanced visualization tools allow traders to expand their market view simultaneously while speeding and streamlining workflow. Consider this challenge in the market data world: A traditional decision-making process often takes a single symbol or a small set of symbols, each with very granular real-time and historical detail, and then adds varying degrees of complex analytics. It is increasingly necessary to expand the scope of this process to include many symbols, often in many markets, while maintaining robust analytics and deeper correlations. This becomes computationally expensive very quickly, so traders and firms are confronted with a balancing act in their workflow: Processing the number of markets and conditions that satisfy strategy requirements on one hand against a burden on speed on the other. Solutions to deal with this sort of Big Data hurdle already exist, for example, in the form of programs that condense data that previously may have occupied hundreds of charts and multiple monitors into an accessible, single-screen view. Data that otherwise may have been very difficult or impossible to manage can now be parsed by the human eye.

comments powered by Disqus