Question: What is the difference between execution analysis and transaction cost analysis?
Q: So, what changes based on the use of execution-level data as opposed to just orders?
A: …if the lens through which you see data evolves with the data.
Data are neutral. Consequences of changes in data can be enormous. In that spirit, we will try to sort out what all this means.
Enhanced granularity of trading data delivers transparency. Dashboards are popular for algorithms and routing. Post-trade analysis follows suit. Whereas order-level transaction cost analysis (TCA) contains average order sizes, durations, cost, market conditions and so forth. This information is now available at the level of individual algorithms. New metrics appear, such as price reversion and fill-size by strategy.
There is more: Distributions of fills by market-cap, by start time, by number of placements... Need we go on? Absolutely: Distributions of algo usage by market conditions, fill sizes, duration and number of venues. There is a blizzard of data once granularity has reached the level of individual fills (see “Strategy cost review,” below).
But this is all just TCA, which also is thought of as transaction cost reporting. It is “observational analysis” conducted with different data. In contrast, the trend in order-level TCA has been to navigate a smaller blizzard to isolate actionable information. In execution-level analysis, we almost achieve a count of distributions of outcomes equal to the number of data points in order-level TCA. That exercise has left granular TCA unchanged from its aggregate cousin: the setting of performance results in the context of market conditions and trader activity.
The most popular report requested in order-level TCA is an outlier report. It may be as simple as the five best and worst outcomes. It may be as complicated as the breach of user-defined bounds in particular market conditions.
The most popular report in execution analysis is also an outlier report. We have more possibilities for the definition of “outlier.” Examples include prices at the millisecond level and reversion statistics.
The aggregate and granular concepts are the same. Beyond extra metrics, there is no change in moving from order-level to fill-level analysis. However, there are some pitfalls.
Granularity and noise
Every outlier has a story, which could not be captured in the assessment of the trade. Studies of trader physiological and psychological behavior suggest that these stories be interpreted with caution. After the fact, trader opinions may be disconnected from market conditions and uncertainty. (See, for example, John Coates, The Hour Between Dog and Wolf: How Risk Taking Transforms Us, Body and Mind, Random House, 2012.) There is physiological truth in the axiom, “You are what you trade, not what you say.”
One person’s stories are another’s noise. While granular data may amplify resolution, like pixels on a screen, they increase uncertainty. Peering closely at a pixilated face delivers more about the nose than desired, while missing the shape of the chin completely.
How many outliers should one expect, viewing them as noise? A pragmatic standard is 5% of the data. That’s 50 outliers for 100 orders resulting in 1,000 executions. The trick will be to establish methods that find patterns in that many outliers. Such methods could turn a poor data situation into something more useful.
Best price, best execution
There is no commonly accepted definition of best execution. This uncertainty has driven some market participants to think about best execution as best price. Execution analysis has a tendency to promote this belief, perhaps because prices are the only things truly moving at millisecond intervals.
Best price at the fill level is not best execution. Best execution entails the best outcome under the circumstances surrounding the implementation of an investment decision (that is, surrounding the order). Consider the example in “Execution analysis” (below).
If a passive participation strategy is followed, getting the best price at every fill, this order would cost 75 basis points. The strategy used resulted in a cost of 60 basis points. There are no obvious outliers in the implementation of this investment decision. We can, nevertheless, do better based on execution analysis, albeit with the right lens.
The lens through which execution analysis differentiates itself must focus on trading strategy. Granularity of data then adds value, and strategy is the core contributor to the final result. Strategy analysis is actionable. Conditional on the investment decision, the best strategy choice leads to best execution.
Strategy is the target of a lens that evolves with the data. Continuing the example of “Execution analysis,” any price path can be differentiated through trading strategies (see “Multiple paths,” below).
A more aggressive strategy drives price, incurring three times the trader’s price impact of the realized strategy. Nevertheless, the effect of momentum in the broader market is minimized, and competing orders are circumvented. Understanding the role of strategy and making the right choice in this context drops the cost of the order by more than a third, to 40 basis points. The circumstances behind the choice become the focus.
The spread between a bad strategy (passive participation here) and a good one is 35 basis points, money worth fighting for. Strategies may be decomposed into effects due to market momentum, competing orders and the trader’s own activity. These factors may be analyzed in terms of market conditions and relative liquidity. Guidelines can be set and hypotheses tested.
The question of outliers can be settled in an entirely different fashion and without a blizzard of numbers. The trading strategy itself is the object of analysis. In the case considered here, passive participation is an outlier. In this example, it is obvious as to why. But generally, it is the “why” that elevates observational reporting to analysis.
Execution analysis sometimes goes under the rubric of execution consulting. In that guise, it has been an educational tool for brokers, applied to their own trading algorithms. This discussion is relevant in a broker-neutral context. Are algorithms truly commoditized, and differentiated only by trader implementation? What are the usage patterns? How do strategy types react to market conditions? (“For example, Algorithmic Trading Usage Patterns and Their Costs,” Ian Domowitz and Henry Yegerman, Journal of Trading, Summer, 2011).
Venue analysis is a 25-year-old theme, revived as part of execution analysis. A venue report is part of any execution study, but care must be taken. Consideration of trading strategy is an essential component in assessing venue performance. One cannot contrast two market structures, or assess individual venue quality, without controlling for the strategy used.
Order-level TCA feeds applications like as portfolio optimization, fund capacity analysis, liquidation studies and fund NAV determination. Execution analysis exposes the core issue in day-to-day operations and trading strategy, and paves the way for real-time decision support. In the end, it is the suite of applications that will distinguish execution analysis from order-level TCA, along with questions that evolve with the data themselves.