So much talent... focused on the buying and selling of securities, instead of creating new things that will make the world better in a directly measurable manner.
Virtually all trading volume today consists of buying and selling old securities -- essentially, legal claims on existing assets. The sale of new securities issued to finance the creation of new products and services -- for example, a company selling new shares via an IPO or issuing new bonds for investment in physical infrastructure -- represents only a minuscule portion of total trading volume.
I'm not sure having so many of our best and brightest minds going to Wall Street (and into high-frequency trading in particular) is a good thing, from a societal perspective.
--
PS. Whenever I read anything about high-frequency trading, I'm reminded of the following passage, written by John Maynard Keynes in 1936 -- 77 years ago (!): "Of the maxims of orthodox finance none, surely, is more anti-social than the fetish of liquidity, the doctrine that it is a positive virtue on the part of investment institutions to concentrate their resources upon the holding of 'liquid' securities. It forgets that there is no such thing as liquidity of investment for the community as a whole. The social object of skilled investment should be to defeat the dark forces of time and ignorance which envelop our future. The actual, private object of the most skilled investment to-day is 'to beat the gun,' as the Americans so well express it, to outwit the crowd, and to pass the bad, or depreciating, half-crown to the other fellow."
Someone said something similar about a lot of people in the startup community. That we were sending our best and brightest out to create products that ultimately only served the purpose of pushing ads (Instagram, Facebook, Twitter, etc). I don't know how true this is. There are a lot of very smart people in the world. The luxury of having that many smart people is that we can have a lot dedicated to designing amazing infrastructure that sends satellites into orbit and we can also have Twitter and HFT.
Theoretically, people would pay for the things they value and problems they want solved, and the market would do the rest.
Practically, most people don't actually have that many major problems that need solving, and it is pretty simple to invent something that isn't what someone needs, and convince them that they need it, then to come up with novel ideas.
Industry and free enterprise does not solve problems much anymore. It sells the process of convincing people they have problems.
Jeff Hammerbacher, CTO of Facebook, now CEO of Cloudera.
That said, actual product companies are making a comeback in Silicon Valley, and are using innovative new tech like 3D Printing, and there's some amazing work going on in Virtual/Augmented Reality as well (Occulus Rift, CastAR, Google Glass), - all physical products.
And lets not forget Tesla upending the world's perception of the economics of producing and owning electric vehicles.
While I agree with the sentiment about smart people wasting their potential chasing cash, I think it's only an intermediary phase before the next wave of "insanely great", life-changing products hits critical mass.
Lets not forget Tesla upending the world's perception of the economics of producing and owning electric vehicles.
Citation? The world's perception of Tesla is arguably: expensive, unaffordable, impractcle plaything for rich people. But yeah I would drive it if you paid for it...
Then why we haven't cured cancer yet or malaria or global warming or psychological disorders ? There are a lot of smart people in the world but we really can't afford to work on silly things.
Some of those problems are "Nine women can't make a baby in one month" type problems. Sometimes the solutions are stumbled upon by accident, rather than by concerted effort, especially in medicine.
In fact it wouldn't surprise me if some sophisticated neural net algorithm developed for HFT was repurposed and used successfully for other things - medical research, climate modelling, brain research - basically anything involving non-linear dynamic systems.
Only problem of course is that all those HFT algorithms aren't FOSS, but the incentive to develop them in the first place would be greatly diminished since they'd lose whatever edge they had almost immediately after publishing.
The markets are a pretty decent lab for experimentation and learning about complex systems, which we're still not so great with, as long as systemic risk can be prevented and every player forced to take their losses instead of passing them off to governments. I personally don't think HFT is remotely as much a problem as what caused the financial crisis - a few flash crashes here and there plus an "HFT tax" on asset prices, vs the arguably-fraudulent origination and dissemination of trillions of dollars of bad debt resulting in the near total annihilation of the global economy. I think we'll gain a lot more (or at least lose a lot less) as a society by prioritizing the latter problem.
Price discovery in markets, and economic boom-bust cycles are a societal/tech problem going back hundreds if not thousands of years. HFT is only part of the solution (or should be), and I don't think its silly. And even if it were, sometimes there's just no getting around the silly/thorny issues (makefiles anyone?) before you work on the important stuff directly.
Virtually all trading volume today consists of buying and selling old securities -- essentially, legal claims on existing assets. The sale of new securities issued to finance the creation of new products and services -- for example, a company selling new shares via an IPO -- represents only a minuscule portion of total trading volume.
--
Leaving aside for a moment the question of the value of HFT, this is a skewed view of the world of finance. The equities markets may be the most visible to the public (quoted on the news, associated most directly with companies and products we all know, etc), but it's just one market.
In fact, the size of global bond markets is almost double that of equities [1] and is a huge source of funding for companies (and governments), and has more frequent new issues for the same entity than in equities. Similarly futures markets are hugely active and important as both a source of price discovery for commodities and as a way for businesses small and large to hedge risk.
I mention all this not to pick on you in particular, but because it seems in the wake of the financial crises of the past few years, this "finance is bad" mentality seems to be pervasive. There are absolutely problems, but I think it's important not to lose sight of the fact that much of what financial markets do is hugely important to modern businesses and governments.
Markets are hugely important. The problem is the size of the investment banks who neither supply the capital nor the equities, but due to proximity of the transaction are able to skim more and more cream out of the transactions and devote more and more brainpower to increasing that arbitrage. They have become so instrumental to the lives of the rich and powerful that they can completely destroy economies and be bailed out by tax payers the majority of whom have absolutely no interest in the solvency of said banks other than some extremely urgent, hand-wavy and fundamentally interest-conflicted warning of economic cataclysm.
Obviously regulation doesn't seem to do much good because it just changes the rules which sends the investments games scrambling for new arbitrage opportunities which no regulatory agency has the resources to keep up with until its too late. Given the lack of political fortitude to combat moral hazard by forcing financial institutions to take their own losses on the chin, I think perhaps widespread and rabid investment banker hate could be the best societal medicine we have against the brain-drain to finance.
It looks like the US bond market daily trading volume is about 25 times higher than the US stock market daily trading volume. However, that statistic is highly skewed by trading in US treasury bonds, which arguably should be a separate category. The same would presumably apply to the stats on market cap.
Also, it's not the relative size of the equities markets that's the problem; it's the amount of effort and talent that goes into what is essentially a zero-sum activity. Bonds and futures (and, to be fair, new stock issues) are positive sum. HFT is not.
(More precisely, HFT is only positive sum to the extent that it improves price discovery and therefore market efficiency; but I have a really hard time seeing how shaving a few more microseconds off trade times changes that significantly. It does, however, significantly change who is able to benefit from asymmetric information by inducing others to take the wrong end of zero-sum trades, which is what HFT is designed to do.)
significantly change who is able to benefit from asymmetric information by inducing others to take the wrong end of zero-sum trades, which is what HFT is designed to do
This is simply untrue, nobody is being induced to do anything. Tactics like submitting orders you don't intend to let trade, to make other participants believe there is interest when there is not is illegal and enforcement is more aggressive than you might think.
To the remaining part, if you believe that HFT improves market efficiency to the extent that it exists, but don't think that the increasing competition provides additional value - OK. I'll work with that, but then if we can come around to a view that HFT is at worst neutral, why should it be curtailed? Simply because some think the people who practice it should spend their time elsewhere? I think that is a dangerous standard to enforce anywhere.
"Induce" may not have been the best word; I didn't mean to imply that manipulation of the sort you describe was going on.
What I meant was that the whole point of HFT is to get information about the market state, and act on it, a little bit faster than others; which means that when you make a bona fide offer based on your HFT algorithm's understanding of the market state, that understanding is based on information that the other party to the trade does not have, which means that the other party might accept a trade that, if they had the same information as you do, they would not accept. That is asymmetric information. There doesn't have to be any skulduggery going on; it's a natural consequence of what HFT algorithms are designed to do.
why should it be curtailed?
I'm not saying it should be curtailed; I'm saying that it's a shame that so much talent and effort goes into a zero-sum activity, and that that fact is a big part of why ordinary people distrust the financial system. In other words, HFT is giving other more beneficial financial activities a bad name. The right way to change that is for the financial system to police itself.
I think that is a dangerous standard to enforce anywhere.
I agree, and that's why I think the financial system needs to police itself, before it gets policed by others who don't agree with you and me that such a standard is dangerous to enforce.
Most of their technology is incredibly secret (for obvious reasons). They won't even tell you they're using your open source project (they can give you money though).
They're (on average) worse than your average tech shop. But then again, people like google or facebook will never buy software from anyone, so if we're left with just them, you can never create a business writing software that caters for other software companies, not sure what is worse.
I always wondered whether it would make sense to legislate a random added latency in exchanges in order to create a latency noise floor
to remove the need for HFT. Without such a cap it's an arms race that adds no value.
Why resort to legislation? Create such a market yourself (or with a team) and see if your tweaks naturally drive demand from market participants. No force necessary.
> Without such a cap it's an arms race that adds no value.
I have seen nobody ever challenge the value of liquidity in these discussions of HFT. Assuming this, the argument is that at some amount of latency the additional liquidity turns into a negative.
It seems more like the additional value is simply extremely small, but meaningful over an entire market.
What's the exact cutoff point for the value of additional liquidity turning negative, and will anyone agree on that arbitrary number?
Assuming HFT does turn into a negative at some point, wouldn't the same apply to optimizations in other markets like online advertising?
If it does apply elsewhere, we should also legislate away the practice of making online advertising more efficient to prevent tech giants like Google and Facebook from wasting billions of dollars and untold man-hours on these activities that are clearly worthless to society.
>Virtually all trading volume today consists of buying and selling old securities.
>I'm not sure having so many of our best and brightest minds going to Wall Street (and into high-frequency trading in particular) is a good thing, from a societal perspective.
Your so wrong. Cash Equities account for notionally c. 50% volume at best the rest being made up from exchange traded derivatives. HFT, MM and Prop firms by speculating and posting contra liquidity lower transaction costs for other market participants. Society benefits from having a fluid and well greased market which reflects all known information.
Similar things can be said about military research, law firms, medical clinics (not medical research), advertisement firms and most other types of work.
I disagree with this assumption. The problems with military research, law firms, advertisements are of degree not of its existence. Sure, they have their own problems. But we need military research: which is arisen out of insecurities of nation states, to given protection to their states. Law firms, too have a place so too medical clinics and advertisement firms.
In case of HFT, the argument is not of degree but existence. Military solves security problems of nation states, law firms communal issues of individual citizens, advertisements to help the product reach the intended audience. But what problem does HFT solve other than making a few people very rich.
It reduces the friction in every single financial transaction. There used to be considerably more people making markets, at first yelling at each other across the pit, then manually clicking on the screens, now automated out of existence. Machines not just doing the same job mind you, but doing it much better and faster (and without sleeping, toilet breaks, emotions). So the better consistency/efficiency means these firms can quote tighter and tighter spreads in a price war, which they do. Never has so little money been leeched from the markets by middle men (less money extracted, but concentrated in fewer hands perhaps...). An enormous benefit to society. A benefit that compounds along the supply line.
The reason you hear so much about HFT is because the traditional firms it is displacing want you to.
I am not as much against automated trading, as I am against High Frequency Trading.
Automated trading itself is still dangerous, in that many of these algorithms are black-boxes. They induce complexity and turbulence into the system. For example the minor bubble just before 2008 crash. May be automated trading/HFT was lucky that, the crash wasn't majorly their part. But it is a ticking time bomb.
Now with HFT it is all the more dangerous, because of the speed of interactions. If I make a bet in 50micro seconds, the space of possibilities explodes on you.
And how is it of enormous benefit to society, when it is concentrated in fewer hands. Again, these micro variations hardly say anything about the quality of the goods being produced than what other traders think about the goods. You are optimizing on a parameter, which does not model the problem you are solving. You might find a minima, but one that has nothing to do with making/distribution of these goods.
"And how is it of enormous benefit to society, when it is concentrated in fewer hands". So by your logic, Tesla or SpaceX are not of benefit to society because it's concentraded in very few ( 2 ) hands of Elon Musk ?
Secondly, anything that is profitable and legal is of benefit to society, unless it generates obviously negative externalities ( costs for others ) such as pollution
the reason is, our society is mainly built on the idea that everyone can pursue whatever they want and not what some central authority deems to be "useful"
That is a weak argument :). Not really, there is a certain and rather high possibility that common man can buy Tesla cars. Sure, there is a premium price tag now, since we are still chipping the first blocks in making that technology a n economic reality. And that specific case is all the more important, since as you too argue helps to reduce pollution.
"Anything that is profitable and legal is of benefit to society" is a little misleading. The definition of what is "legal" is an evolving thing. We know it is dangerous, in that can cause huge fluctuation in the financial control system without real benefit for the society as a whole except a very small percentage. I am not speaking of a central authority that decides it, but our common collective consciousness that should decide it. And again there is not one thing, which has no obvious negative externalities. But the question is whether the positives overcome the negatives. In this case, for the society as whole there is little benefit.
Exactly, when the very society is built on that idea that everyone can pursue whatever they want, the access to opportunities should be equal. HFTs are tilting the balance in one way (helping the rich getting richer) , can cause economic crashes, which hit the rich and poor alike. Now the aftereffects of crashes on the rich (except in case of exceptions) are not as tragic, as in case of the poor. For the poorer part of the society, it pulls the rug under them.
"HFTs are tilting the balance in one way (helping the rich getting richer) "
WTF are you talking about?
1. HFT's are full of math and computer science people, which are usually drawn from the middle and lower classes.
2. Before HFT's, the people on the floor were inbred... That's right. There was a lot of passing down the torch since it was the type of job that you can learn by doing it. Oh, yes, and to claim that they were selecting by some "meritocracity", they were selecting for "street smarts" Haha! That only gave them carte blanche to hire their relatives
Company MakeWorldABetterPlace.com creates new thing and makes us all feel better in a directly measurable way.
They decide to create an even better new thing that has heavy capital requirements so they issue stock.
Even better new thing is not as great as we all thought, so stock goes out of favor as market participants channel capital to new and more promising tech companies.
Years down the road, MakeWorldABetterPlace.com, trading on old securities decides to enter the mobile market and creates revolutionary device. Market participants catch on and start channeling capital out from less promising securities and into the old and existing claims on assets of MakeWorldABetterPlace.com. Thousands of businesses and jobs are created by new privately held companies that create new things using MakeWorldABetterPlace.com's mobile operating platform.
So much talent focuses on the buying and selling of securities because it is the most social way to vote directly - by allocating capital. And the product of that capital flow is what is observed as the creation of new products that make the world better in a directly measurable way.
If you don't think liquidity is important, then with it goes the ability for countless tech startups to exit (through their acquisition by large corporations in exchange for claims on existing assets) and use the resulting capital to create subsequent startups, that perpetuate the cycle of creativity.
> Some facilities such as the Mahwah, New Jersey, NYSE (New York Stock Exchange) data center have rolls of fiber so that every cage has exactly the same length of fiber running to the exchange cages.3
I remember the first time someone told me that. I thought they were kidding. Then I actually got to see the data center.
The exchanges, and HFT firms, take this very seriously. The speed at which they execute is just unbelievable.
The amount of technology and networking stack knowledge is just mind blowing.
I remember tuning my linux distribution's networking stack to gain an advantage only to find out that everyone had moved onto using asic's and infiniband hardware.
I continue to be amazed at the sheer talent and brain power that goes into this field:)
Spread Networks invested an estimated $300 million in building a new fiber line from Chicago to New York in order to shave 3 off ms. However, they found out after completing the line that faster speeds were already being achieved by microwave connections built in secret. It's an interesting arms race:
Obviously if they are building fpgas into the switch itself then well of course they'd already tried running customized ethernet driver firmware and all.
It really depends on what asset class you are trading, like the author mentioned. For equities the performance demands are out of control (< 50 micros). For FX, it's still pretty tight but much more manageable.
How do I get into this field? I'm a 20 year old making a shit ton of money by doing mobile work but I find it so boring. Nothing about it is innovative. I feel like a high level computer construction worker. It's brainless for the most part, it just takes time.
I want to get into some really cool stuff. I've been wanting to get into investment banking from a technical standpoint, HFT, algorithmic trading etc, but honestly don't know where to start.
What are the de facto books/websites/reads for this field?
The worlds of HFT and the rest of algorithmic trading are very far apart. For HFT pickup the exchange documentation for each exchange. Memorize how order types. Get a copy of RegNMS. Memorize that as well. To be effective an HFT needs to exploit the market structure. Learn how processors work. How do you avoid a pipeline stall or a cache miss? Become an expert in c++. Maybe pickup some verilog for those pesky FPGAs.
The rest of algorithmic trading offers a lot more flexibility. In either case read 'Trading and Exchanges'. It will give you a good overview of the mechanics of all different venue types. Then its thinking about how to build better algorithms then the next guy, or get better data.
I second the recommendation of 'Trading and Exchanges'. It's a great book to get insight into how the underlying exchanges work and how to exploit certain architecture to produce profitable strategies.
Well turn the question on its head and ask what can you bring to a hedge fund to allow it to be more successful. That's the test we use.
Hedge funds are similar to start ups in that they are small businesses where everyone needs to pull their own weight and be able to handle their own area without supervision as everyone is busy enough handling their own area of concern.
Or put another way, say the firm has 10 people and each share equally in the year end bonus pool.
If the pool is $100 then each person makes $10 in bonus.
Therefor you must be able to increase the bonus pool by atleast $10, to $110 in the next year or two, in order for the firm to break even by hiring you.
So what do you bring to the table that allows the firm to increase its profits? how good are your networking, math, C or C++, optimization skills, what market structure insights do you have?
Thanks for the answer. I meant more of an absolute beginner in finance, in general, nevermind the stock market or trading or writing bots or hedge funds. Thanks for the insight.
It is /almost/ essential to have a PhD in CompSci/EE from a top school to do HFT. Alternatively one should demonstrate extensive hardware/networking and optimisation skills obtained from other low-latency industries.
All of the top work is being done on FPGAs and latency is now on the order of microseconds (probably lower).
As for lower frequency algorithmic trading, that is a game that one can play at the 'retail' level if you're willing to spend (quite a lot of) time learning.
If I said I worked at Google and I continue to be amazed at the tech behind how they served up ads would I still get the same remark from you?
Maybe I could be doing more with my skill? I don't know but I really love the learning curve I'm on working with cutting edge technology and pushing the performance envelope.
Biologically, we respond to amazement as part of the learning process. But all too often, we receive these dopamine rushes simply as a consequence of the amount of resources and effort someone (or a group) puts into something.
e.g. You can go to Las Vegas, see thousands of people running around topless in gorgeous outfits and be amazed, yet what's the actual value to you?
I see this ALL too often in art. People get "amazed" by things that require a great deal of effort/resources (look how many transistors in this CPU!), but this temporary narcosis blinds us to evaluating /whether it's a good idea/. So many art projects are simply the result of an ordinary person commanding extraordinary resources from a corporate entity to construct a large number of instances of a single, mundane object or a single, large-scale version of the object. In these cases, the art is not in the sculpture, but in the conning to get the resources. I see budding artists respond to this amazement and trying to get huge funding.
Internet bloggers (Cory Doctorow comes to mind) abuse the word "amaze" all the time. I just checked boingboing.net right now and saw amazing used to describe wallpaper (no dude, this is /natural history/ wallpaper).
I realized that there's a frequent uproar against HFT, so I assume people here are pretty tired of it and are not willing to discuss it/have their minds made up about it already. That's why I won't go rhetoric.
edit: I guess, I did though.
I can't help to reply to the adwords question - although you can argue that advertising/marketing is soulless, at least it creates real world value. Someone makes a product, someone else buys it. Exchange of goods happens.
Derivatives are uncontrallable in the my humble opinion. Especially if regulated by uber greedy people. And it makes me sad sad when I see the highest skilled people go and put their talent in something that is pretty much useless, apart from making a few, selected people even richer than they are. But I guess, whatever pays the bills.
But that's not why I made my comment. I made my comment, because what I find even more sad, is people going numb to injustice and admiring this situation and the people who are making it possible.
He wasn't admiring the economic situation, he was admiring some truly amazing high-end computer networking.
If HFT produces nothing else (and I generally agree with your economic view of it), at least it'll have financed the next generation of low-latency networking gear.
I think much of the effort is wasted. When you hire embedded designers to program FPGAs and ASICs just to be a few nanoseconds faster in your rent-seeking, it's a bloody shame.
Technology is making the existing system much more efficient, lowering the total amount of "rent-seeking" in the system.
If this was any other industry it would be lauded. I don't understand the moralizing about this industry while we applaud start ups for building ever more complex twitter aggregators.
What economy did low latency trading destroy? Also, is there any more deep capturing of regulators in the finance space post low latency trading than there was pre?
This is an ignorant statement. The cause of the economic crash in 2008 was leveraged bets on lots of bad debt. These bets were made by fratboys in suits, shaking hands and making phone calls and sending emails. The financial instruments they used to do this were bespoke one-off deals, not securities traded on public exchanges.
High frequency trading had nothing to do with any of this. HFT deals in fungible instruments traded in public markets.
I'm going to say this again: Exchanges should be put on a stepped clock. I'd suggest a one minute interval. Orders pile up during that interval, and then they're processed in random order. If things start to go really, State-concerning hinky, then authorities "pull the plug" before the next interval is up.
Anything less, to my mind, serves point-shaving and profiteers, rather than real investment.
Then, all the "gearheads" can go back to doing useful science. And maybe getting paid decent wages and benefits for doing so.
Hoped to find such a comment here, this is basically about mechanism design - how can we design an exchange that removes the additional profits such microscopically low latency generates, when the low latency itself isn't providing value, i.e. the low latency => profit phenomenon is an artifact of the rules of the exchange.
I appreciate your thoughts, but regarding your suggestion: random isn't well-defined. It can be random as in "randomly pick a participant, let it buy/sell as much of the stock as it asked, then pick another participant etc.". This encourages Sybil attacks - participants have an incentive to create other identities for themselves, and hope one of the identities will be picked "randomly". Another definition of "random" may be "randomly pick an order to buy/sell a single stock, execute it, repeat-until". This, I was told on an earlier HN thread, strongly encourages placing more orders than one wants, again in the hope of fooling the "random" part of the exchange. I was told this is actually what's happening currently in exchanges built this way.
I had a suggestion I tried to analyze (even as a possible idea for a mechanism design paper): What happens if "random" is defined as "your chance of getting picked (to be able to sell the stock) is proportional to the amount of this stock you own"? At least, we got rid of the incentive for Sybil attacks (and didn't create the opposite incentive, for conglomerates - separate entities don't have an incentive to present themselves as a single entity, then do the bookkeeping internally). But it's unclear what to do when there are more willing buyers than sellers. Any thoughts, anyone?
I have also wondered whether this whole thing is an artifact of quantization - what would happen if the price granularity was allowed to be as small as one wants?
(In case anyone's wondering, a friend I consulted told me this probably wouldn't fly as a paper - useful economic policy if I can solve that "more buyers" problem, but nothing scientifically novel).
By the way, this is an awful, awful idea that you hear quite a lot. Think about what would happen. What incentive would a participant have for placing an order early in the one minute interval? If you can't cancel it, you're a sitting duck. And if you can't be executed now, why insert the order now? By doing so, you show your hand. You'd end up with a rolling one minute e-bay auction where everyone tries to insert at the last possible microsecond.
Why would it matter when people place the orders if there is no extra information to be gained during that time? If the orders placed are only revealed after they have taken effect, why bother waiting for the last moment?
I take your point, and I didn't downvote you. From my perspective, there is an important difference between taking a position in / with respect to a product (or its organization) and taking a position for the position's sake.
Perhaps it is something of a continuum, not a strict either/or. If so, I find that currently we've gone too far towards the latter.
I'm also concerned that current constructs exhibit chaotic tendencies that put real people and essential systems at risk. I view stepped trading intervals as a potential damper to some of these negative and out-of-control tendencies.
Ultimately, I view investment as backing and furthering production. And I don't see how micro-second based investment appreciatively, substantially furthers this.
P.S. Admittedly, it has been an important factor in pushing computer and network systems development, which may well have had knock-on effects. E.g. is one substantially aiding the other, or the other way around: Trading versus CERN data processing and distribution?
My comment was a bit snippy. Not intended to be rude (but perhaps pointed).
I suppose I get concerned when I see forces applying moral judgment to investment techniques and timescales (as distinct from companies actions, where I think it's easier to judge - e.g. not wanting to invest in tobacco, for example).
The long term investor is not necessarily more moral than the short term investor, though he may be. But part of the long-term investor's confidence in his investment emerges from the fact that he can exit his investment if he judges that long term trends make it less valuable to hold it.
That liquidity is, in large part, created by the churn of short-term speculators operating in the zero-sum game of the short term. Spare no tears for the short term-speculators - they operate in a tough game, and the ones who actually thrive and stay around for the long haul do provide a valuable service - reduced trading costs, narrowed bid-ask spreads, and faster order execution as a commodity.
Similarly, products such as ETFs (held by millions of small investors and 401(k) accounts) owe their existence to the ability to rapidly rebalance portfolios in response to changing prices of the underlying products that the ETFs are trying to replicate. Prohibiting rapid trading eliminates the possibility of the various affordable products that track indexes, commodities, sectors, and so forth.
Dark pools exist because these "real investors" you speak of are muppets. If you're a market maker out in the big bad world you are exposed to adverse selection risk - i.e. you're up against smarter, better informed or faster traders who hurt your profits. If you set up a dark pool, you can pick and choose your customers, and keep these people out. You can quote tighter in the dark pool than the wider market, so your customers think they're getting a great deal, but in reality they're only getting this deal because they are (collectively) great big juicy marks who you can make a fortune from.
This is why you see the big investment banks setting up dark pools, they can't compete in the technology race or with smaller more agile firms, so they retreat to these pools.
The sub-penny rule (decimalization) was also mentioned by Chris Stucchio (aka yummyfajitas).[1] And it is extensively discussed in the book "Dark Pools".[2] Because order queues only exist at whole-penny prices, the bots are forced to compete on speed. If it weren't for that rule, there would also be competition on price because you could place offers and get price-priority in the sub-penny interval.
The playing field is basically a DDoS arms race, where the main technique is full-throttle quote-stuffing to push anyone without equivalent hardware and connection away from the front of the queue, and out of the game. (Dark Pools also mentions a conspiracy of special undocumented order types on certain exchanges). The ACM article even admits that the algos aren't even particularly intelligent, just very speedy: "In my experience, most high-frequency algorithms are fairly straightforward in concept—but their success is based largely on how quickly they can interact."
A distributed order book like the one in the Ripple protocol[3] is a much fairer system. The network charges a transaction fee in order to prevent quote spam/DoS attacks, price fractions can be as small as 6 decimal places, and offers are processed in an effectively random order (within each discrete ledger).[4]
Mind you, if they lower tick sizes, that means a lot more order cancellations. A market-making algo is simply a control system, taking in a set of inputs and spitting out a fair price that it is willing buy/sell something for. The number spat out is a rational one, that is then rounded to the nearest discrete tick. Smaller ticks = more jumps in the discrete output price = more cancellations/modifications. Which is fine, except the anti-HFT brigade seem to think a trading system that cancels 10000 orders and trades 100 times is the devil. It's not, it's about as remarkable as any other control system. You may as well express outrage that the cruise control system in your car adjusts the throttle ten thousand times a second. The only downside is that the exchange needs to cope with that level of traffic.
I'm wondering if someone here can explain to me the value in HFT for anyone other than the people doing the trading. I went to Wharton and some old friends are in HFT and the one thing they all say, and I hear repeated everywhere, is that they provide liquidity to the market. Something about it just doesn't ring true to me. If it is in fact true I'd love to hear an explanation.
You are assuming that HFT profits come at the expense of investors. In fact, they were taken from insiders.
I'm old enough to remember trading on US stock exchanges in the mid 1990s when prices where quoted in 1/8s and 1/16s and NYSE specialists were the only ones with visibility into order book. Think about it: you as an investor had no idea of the depth of the order book, but the specialist who took the other side of the trade from you had it in front of him. The specialists were minting money. They would lose money maybe one day per quarter, and their ROEs were ludicrous.
HFT and ECN trading killed them. Labranche, Van der Moolen, Susquehanna, Spear Leeds, all gone. Goldman Sachs bought Spear Leeds for $6 billion in 2000 (it is now no more); Labranche specialist business fetched only $25m when it was sold to Barclays in 2010. Van der Moolen went bankrupt in 2010.
Investment bank trading desks, true champions of customer front-running, are shrinking fast. Cash equity trading has become so tough for the banks that they are starting to think of it as a cost center, a loss leader to promote their equity underwriting business.
These were multi-billion dollar businesses, with tens of thousands of middlemen living high on the hog from the spreads and front-running. They're all (mostly) gone. Good riddance.
If you want to relive the old days of trading before HFT, go execute a large trade on Karachi Stock Exchange. Put the order in and watch in amazement.
I would like to see the data showing total revenues of specialists, market makers, bank trading desks, brokerages, and HFT traders, over time. I would bet they have been going down for two decades. This is undeniably a good thing for investors.
Also, it is hard to prove a causal relationship, but with the rise of electronic trading and HFT, price spreads and fees have come down.
I can't prove it, but I believe that insider trading is also much less rampant in modern electronic systems than it was in the older pit based markets.
Re: spreads & fees: Price spreads and fees have come down as a result of electronic trading being open to everyone, and is probably independent of HFT (at least, modern HFT): E.g., The decreased spreads and fees were happening in europe in the early 2000s, when fastest updates were at 1/4 second, and slower updates were at 2 seconds. The nanosecond scale was not the reason for this.
Re liquidity: it depends how you define liquidity. If you define it by "the average size at the bid or offer and next few price levels" or "volume traded", then yes - HFT has helped liquidity tremendously. If you define it as "the probability that a large order can complete", then liquidity has NOT gone up. With HFT, it's the same 100 shares/futures changing hands thousands of times per day, and they disappear often in times of uncertainty. If you need to execute a large order, you observe that you do not have the amazing liquidity everyone in HFT is talking about.
Re insider trading: I wouldn't bet on it. With electronic trading, it's now easier to do insider trading slowly without attracting much attention. Furthermore, HFT has created new forms of fraud, so far unregulated: see e.g. http://www.nanex.net/aqck2/4329.html
> If you define it as "the probability that a large order can complete", then liquidity has NOT gone up.
People seem to have this idea that back in the days of floor-based trading you could just call up the NYSE and say, "sell 1 million shares of Citigroup!" and the market makers would just kindly oblige you, without widening their spreads or trying to eke out a bigger gain from a transaction which inherently carries significant risk for them.
It's possible that the situation for executing giant orders in one fell swoop hasn't gotten any better under HFT, but it hardly seems to have made it worse. And as the large flow traders become more sophisticated and start to implement algos of their own, they'll get better at moving volume at a fairer average price.
> With HFT, it's the same 100 shares/futures changing hands thousands of times per day, and they disappear often in times of uncertainty.
I don't know about "often," unless you're talking about trading individual issues where news events introduce a high degree of uncertainty. There's really only been one case of a market-wide drying up of liquidity, which lasted for about 15 minutes during the flash crash.
There's also nothing new about market-maker liquidity drying up in times of uncertainty or severe volatility. Plenty of stocks went "no bid" during the 1929 crash. Floor-based market makers are no more interested in standing in front of freight trains than algos are. If you're looking for someone/something to blame when markets crash and there are no bids to be found, focus on the Fed and its attempts to manipulate the credit cycle, which periodically fail in spectacular fashion.
> It's possible that the situation for executing giant orders in one fell swoop hasn't gotten any better under HFT, but it hardly seems to have made it worse. Plenty of stocks went "no bid" during the 1929 crash.
1929 is an obsolete example. Heck, the SEC didn't even exist in 1929. A more relevant example is 1987 -- how many stocks went "no bid" on Black Monday?
Here's how HFTs operate today. Bid furiously while the going is good. If you find yourself on the wrong side of a trade, you can always hit up the actual market-makers for a penny loss. And if the markets collapse, you pull out. What could be better? Supply liquidity when it's not needed, and stay out precisely when liquidity is most needed.
Meanwhile, the registered market-makers are pulling back, or pulling out. And why not? They used to count on having the good times to balance out the bad. But now, the HFTs are siphoning off their profits during good times. Thus, the cross-subsidy has gone away. In periods of market stress, the HFTs go hide under a rock and don't participate in absorbing losses.
The effect is: There are now fewer market-makers in periods of market stress. That's how HFTs have made things worse.
> People seem to have this idea that back in the days
I don't know who these people are. But many people today see the screens and volumes, and say "see, HFT gives liquidity!", and that's wrong. I'm not saying HFT took liquidity away (it may have, I don't know, but for sure I didn't claim that). But it does not improve the liquidity that matters to most market participants.
> executing giant orders in one fell swoop hasn't gotten any better under HFT, but it hardly seems to have made it worse.
I agree. I'm just countering the oft repeated meme that "HFT provides liquidity". Essentially, it doesn't.
> I don't know about "often," unless you're talking about trading individual issues where news events introduce a high degree of uncertainty. There's really only been one case of a market-wide drying up of liquidity, which lasted for about 15 minutes during the flash crash.
There's been tens of "flash crash" and "flash smash" events in the last couple of years. Not of the magnitude of the one blamed on Waddell and Reed, but it's still happening.
> There's also nothing new about market-maker liquidity drying up in times of uncertainty or severe volatility.
I agree. I just disagree that HFT are providing any kind of service or benefit to the market. They only benefit themselves, at a cost to the market that goes way beyond the profit from arbitrage/scalping action that they do.
And it is easy to put a stop to: Adopt the rule Eurex and LIFFE had in 2001 that requires 10:1 order:execution ratio.
"I don't know about "often," unless you're talking about trading individual issues where news events introduce a high degree of uncertainty. There's really only been one case of a market-wide drying up of liquidity, which lasted for about 15 minutes during the flash crash."
Actually not true. There was plenty of liquidity during the flash crash. It's just that the people who were selling ended up not liking the price they got. During the flash crash, the market was quite in order, except for a few stocks where all the bids were got (too much sell pressure) and the remaining bids were at $0.01... For most stocks, the market was just fine, just that the spread widened up considerably to take into account the temporary uncertainty in the market. Also, the flash crash took 5 minutes, not 15.
Flash crashes happen in human-traded markets took. Recently there was a flash crash in Indian stocks, where humans mark the trades. In fact, human-traded markets are worse: in 1987, the brokers just stopped picking up the phones, even if technically they had an obligation to do so.
The liquidity it provides is often very low quality. When liquidity in the old days was added, the players involved had to operate under stricter regulations -- they had to post a quote that was part of the National Best Bid and Offer a minimum percentage of time. With HFT it's a wild west scenario. There's a lot of uncertainty and risk of failure in many of its steps.
I seem to recall that the research on this says that HFT is actually a net taker of liquidity and that their profits are fully funded (and then some) by other investors not really by making the market more efficient (they actually reduce efficiency). I've also heard stated many times that an exchange that only matches trades every X seconds would retain efficiency while eliminating HFT.
This isn't the article I remember but goes into detail on most of these issues:
Just to be clear, HFT these days mean microsecond round trip times between trading algorithms and exchanges.
The situation you describe is thanks to electronic trading, and is also available in financial markets that are not dominated (not to say "infested") by HFT, such as currencies and CFDs.
There is no industry standard clear definition of HFT. That is part of the problem surrounding any discussion of it. People use the term to mean some mix of the following:
- Trading is primarily algorithmic, computers making buy/sell decisions with minimal human intervention.
- Trading is high volume with many low quantity orders with short (millisecond, second, minute) hold times.
- Trading decisions happen at low latency currently in the sub-microsecond time frame.
It is better to think of these as dials than as hard requirements. Some algorithmic decisions take a very long time to make decisions, other electronic trading systems hold positions for weeks or even months. I've seen very low latency systems that only trade a few times per day.
Finally, your currency trading example is perfect. The reason HFT's are less involved in those markets is that they are known to be dominated by insider deals and old boy network cronyism. It is not a "fair" place to trade. If HFT were to come in that may or may not change. I suspect it would get better.
I think HFT is now universally regarded as "faster than human reaction time", which means the slowest HFT has a <200ms "time constant".
HFT stands for "High Frequency". That has not meant minutes since 2008 at least.
> It is not a "fair" place to trade. If HFT were to come in that may or may not change. I suspect it would get better.
"HFT" and "fair" in the same sentence, with positive connotation. Now, that's funny: HFT in American exchanges these days is an all out, unregulated war of bits, and fairness is not an attribute you can associate with it - see e.g. http://www.nanex.net/aqck2/4329.html ; The reason HFT stays away from currency is not because of being "fair" or "unfair" (no markets are fair, every market has privileged players).
It's just that in currencies, the people you take money from own the system and will kick you out. They make their own rules. Whereas on ARCA and INET, the HFTs are the landlords and make their own rules (by quote stuffing and stuff).
Neither market is regulated against the privileged players.
Even with your hard constant of <200ms you haven't really specified enough to really talk about HFT with out being more specific. Is your constant the amount of time an algorithm holds inventory? If so tons of low latency systems can and do hold inventory for much longer than that.
Is it the amount of time it takes a computer to make a trading decision? Nearly any modern computerized system in under that time constant, and many non-low latency trades happen in that time frame.
Any electronic trading system (which I think most people agree are good things) will allow for algorithmic trading (how could they not?). The question is should we try to prevent low latency trading? If so, why? And how can we? Will any system we put in place cause more problems than it is worth?
Finally, if you don't think currency markets are fair, why use them as an example where you are getting better prices than you would be otherwise? Currency markets do not provide better pricing to all of their participants than electronic markets with HFT do.
> Is it the amount of time it takes a computer to make a trading decision?
It is time from when information becomes available until order goes out. This is a well defined measurement, unlike things about "decisions", which are not well defined.
> The question is should we try to prevent low latency trading? If so, why? And how can we? Will any system we put in place cause more problems than it is worth?
"Low Latency" and "High Frequency" are not equivalent, even though you seem to think so. "Low Latency" relates to one event. "High Frequency" indicates a rapid succession of events (Frequency is a measure repeating phenomena). High frequency does not theoretically imply low latency (or the other way around), but practically the correlation between low latency and high frequency is 100%.
If you follow the nanex links, you'll see the "low latency" players are bullying the "higher latency" players, and should be prevented from doing so.
In fact, many exchanges do that - e.g. Some european future exchanges will fine you if your order:execution ratio is >10. Meaning, you have, on average, to execute 10% of the orders you give. In the US, some players have an 10000:1 order:execution order. We should disallow that - as everyone else pays exchange fees for a faster system to suit these players, and then pays more to have a system that can keep up with the spurious orders these guys send.
Really, Eurex has this trivially solved. Since 2001.
> Finally, if you don't think currency markets are fair,
I do not think they are fair. But they are as good as the equity markets (liquidity and spread wise) without the HFT players - ergo, HFT does not provide the benefits HFT proponents claim it does.
CME, ICE & CBOE (the major US futures exchanges) all enforce fill ratios as well and have for some time.
Fill ratios are not designed to prevent either high frequency or low latency trading, it is there to prevent a particular form of exchange gaming (quote stuffing).
You are not being clear (and you are being condescending) on what you mean by HFT players & what you find objectionable. Is a market maker that trades 40K contracts a day but doesn't quote stuff objectionable? What about a cross exchange latency arbitrage trade that is not spamming exchanges? If so why?
Also, I've traded Eurex. They have very specific interfaces for low latency/high frequency traders. You can order different classes of connectivity from them that are specifically designed for these use cases. They also have predatory algorithms, so I'm not sure what they have solved.
> Fill ratios are not designed to prevent either high frequency or low latency trading, it is there to prevent a particular form of exchange gaming (quote stuffing).
They are very effective against quote stuffing, but they also work well against player that leave their quote active in the market for 1ms with high frequency. That's false price signaling, and is independent of quote stuffing (as I'm familiar with it: the practice of throwing so many orders at the exchange that slower players got a lag in their data).
> You are not being clear (and you are being condescending) on what you mean by HFT players & what you find objectionable.
I am specifically talking about HFT players in the US Equity markets - these are the subject of all the HFT discussions on reddit and HN. These do a lot of false signaling, quote stuffing, frontrunning in between exchanges.
> Is a market maker that trades 40K contracts a day but doesn't quote stuff objectionable?
That's fine, as long he is not false signaling either (that is, putting in orders he has no intention of executing)
> What about a cross exchange latency arbitrage trade that is not spamming exchanges? If so why?
This subverts the NBBO system. I believe this should either be illegal, or it should be legal and the NBBO system be canceled. But traders are under the illusion that the NBBO system is protecting them from wasting money to these arbitrageurs, when is isn't.
> Also, I've traded Eurex. They have very specific interfaces for low latency/high frequency traders. You can order different classes of connectivity from them that are specifically designed for these use cases. They also have predatory algorithms, so I'm not sure what they have solved.
The same things: a) quote stuffing, b) false signaling. Being faster costs money, and should provide an advantage - but it should be "neutral".
Is there any way (e.g. quote stuffing) in which you are aware that the faster players on Eurex can causally disadvantage the slower players, like they can in the US Equity markets?
Sure, a very common game right now is quote spoofing (that is already illegal in the US) but still happens and exists on Eurex.
Very fast player X puts a few (<10) big orders on one side of the order book at a level that has some but not a lot of quantity in front of him (using random quantity to make it hard to recognize) making it look like there is more demand than there is.
This will cause other participants to quote at this same level. Once enough orders have been entered behind him, he will yank all of his orders and then cross through that level (or even through 2). He has flipped a level.
His speed allows for 2 properties that make this much easier:
1. He is taking much less risk with his spoofed orders turning into real orders because he can cancel them fast when the market conditions indicate they might get filled.
2. His targets can't catch his cancels/fill through order fast enough to get out of the way.
He can keep his fill ratios within the correct boundaries with no problem.
Also as a contrast to your latency arbitrage point, as a market participant myself, I have no problem paying the latency arbitrage fees. It means that I can shop for trading venues that offer features/fee structures that work best for me, without worrying much about price imbalances.
This is much better for me as a participant than the single exchange monopoly is or having to build up an exchange presence at every exchange.
The competition in that space is so fierce that the cut they are taking from me is much less than the alternatives. For me at least, it is a small price to pay.
Ask any HFT person, and they'll tell you less than 1% of their profit is from arbitrage. Unless, of course, you include the practice of quote stuffing (Which causally generates a latency arbitrage against slower players) as part of "arbitrage" - but that's not much different than saying a robber exploits the arbitrage between the fear for your life and willingness to part with the money on your person.
Well, that would be the first I've heard of, a lot of time. Except, I can't really add this to my statistics, because I don't know who you are or what your stats are.
(And no one should trust my assertion either - I'm just as anonymous as the rest of you, and lying for all you know).
What HFT provides is what other market users ask for; a limit order is essentially "I will pay 0.5 cents/share commission (thanks to the sub-penny rule) to the first person who can sell me shares in x at or below $y)". Since you asked for the first one, you must want it fast, right?
Perhaps other order types could allow market participants to ask for something closer to what they actually want - but new order types are often criticized as giving HFT players an advantage, since they can understand and exploit them faster than other market participants.
Alternately, eliminating the sub-penny rule would allow HFTs to compete on price rather than just latency, which would mean better prices for long-term investors and remove a lot of the profit from the HFT industry.
> I'm wondering if someone here can explain to me the value in HFT for anyone other than the people doing the trading.
Well I guess I'm very biased but here's my stab at it.
High Frequency trading is at the for front of alot of technology such as ASIC's, Infiniband networking gear,and low latency OS and networking stacks.
You could argue that they help push these technologies forward by providing the first customer for these areas.
Liquidity has gone up with the advent of High Frequency trading but like someone else has said its hard to disentangle all market factors to say this is due to HF Trading.
> Something about it just doesn't ring true to me. If it is in fact true I'd love to hear an explanation.
To be fair, this statement is the equivalent of "I've heard evolution is a well followed theory but something about it doesn't ring true to me."
What sort of explanation would you like that hasn't already been said 100's of times by people more qualified than me?
You haven't said what you don't like or disagree with about the many existing explanations:)
High Frequency trading is at the for front of alot of technology such as ASIC's, Infiniband networking gear,and low latency OS and networking stacks.
That's not an argument for HFT. If HFT is useless, then it has divested a great amount of research into technologies that no one really wanted otherwise, i. e., that the social cost of HFT is greater. Standard microeconomics.
The question is, what's HFT good for? Do its benefits outweigh its costs? I believe they don't. Low-quality liquidity under the 1s frame is not only useless for traders, it has a great social cost, in the form of research diverted to this inane game. The only plausible arguments I've found for HFT claim that HF traders wouldn't profit if it wasn't useful -- which is really just begging the question; a circular argument.
All intermediaries -- be it floor traders, or HFT's -- earn money from the same source: the difference between prices that actual investors buy and sell stocks at (lower difference is also referred to as higher liquidity).
Now, the resources that were going to traditional specialists and a subset of the professional traders, are going to HFT's, with some of that returning to investors in form of lower trading costs (if we buy the argument that liquidity has indeed improved, of course. Most of what I've seen indicates that it has.)
As a side effect, we have technology improvement, useful for other things.
Now, there can be an argument made against HFT's, and in favor of more traditional traders, related to a potential decline in market depth. So far, I have not seen any convincing proof of this being the case, but I am open to it.
However, I think a lot of people arguing against HFT's do not quite understand that their position is literally that we need to enact legislation in order to protect Wall Street from competition :)
What good, I ask you, has Reuter's telegram company done for the world? Aside from getting some information to a few London banks and trading houses, we've seen no other applications made of this infrastructure, and should therefore assume none will ever exist.
> some old friends are in HFT and the one thing they all say, and I hear repeated everywhere, is that they provide liquidity to the market.
More specifically, HFT acts like a market maker in that it will take the other side of trades that other players (HFT or not) want to make. Let's say I own 1000 shares of MSFT, but I need to liquidate them for some reason. A margin call, a new car, or some immediate, unexpected cost has arisen. The more potential counterparties I have -- the more players that will potentially buy my shares -- the more liquidity there is.
With more liquidity, I can hold out for a higher price (with a cost in time), or I can sell more quickly (at a slightly lower price -- with a cost in money) than I could otherwise.
Whether or not my HFT counterparty is executing profitable trades (which it is, presumably), the very fact of its existence as such is a net benefit to me.
There is so much liquidity in MSFT that it'd trade at a penny spread with or without HFTs. Thus, the only advantage is speed.
So you got your order filled in 5 milliseconds instead of 6 milliseconds. So what? You're a person.
And stocks without enough liquidity to get to a penny spread? The HFTs don't like those stocks, because the "F" is not "H" enough for them to participate. The very nature of HFTs is that they prefer to participate when there is already plenty of liquidity.
> There is so much liquidity in MSFT that it'd trade at a penny spread with or without HFTs. Thus, the only advantage is speed.
You're talking to an unabashed proponent of subpenny increments. Smaller spreads, more counterparties, faster execution -- these things cannot possibly hurt.
Note that I am talking about the concept of HFT. It's true that there are shady operators who abuse rules and undertake things like quote stuffing. I'm not here to defend any of that.
> So you got your order filled in 5 milliseconds instead of 6 milliseconds. So what? You're a person.
Still a net benefit even for just market orders. Limit orders benefit much more from HFT, particularly the stop-hunting variety. I think of the randomized price action as similar to quantum noise. No one really knows to the nth decimal what the price should be, but it wanders around where the money thinks it should be.
So that noise can help trigger your limit trade at exactly the price you want, whereas the false placidity of more granular enforcement is less likely to hit your limit.
Now let's look beyond just me and consider that some firms get more relative benefit from HFT than I do, and that these slivers of benefit sum across market participants.
> And stocks without enough liquidity to get to a penny spread? The HFTs don't like those stocks, because the "F" is not "H" enough for them to participate. The very nature of HFTs is that they prefer to participate when there is already plenty of liquidity.
Of course -- this should not be surprising in the least to anyone paying attention. The reason HFT practitioners participate is to attempt a profitable strategy. Liquidity is the side effect, not the goal. So while it's fair to say they provide liquidity to the market, or even that providing liquidity is the role they play, it's neither their duty nor their motivation.
Similar to how commodities speculators provide counterparties for nervous farmers, they are guided by the invisible hand and not by duty.
The purpose is to make money, as with _all_ businesses.
You think Apple want to make products and improve people's lives without making money? If so, why don't they reduce the prices? You think Google is all about search? No, it's about making money -- you get encouraged to go after... and I quote Sergey and Larry here, "markets of at least 1 billion" and make products that could potentially be worth 1 billion per year. Search my ass... it's all about making money as with most companies, including most of the the non-profit ones (non-profit hospitals that charge more than the for-profit, universities, etc)
""It's legalized front-running. I think it is basically evil and I don't think it should have ever been allowed to reach the size that it did," he said. "Why should all of us pay a little group of people to engage in legalized front-running of our orders?""
What's the difference between HFT and traditional Human Market Makers?
May I suggest that Electronic trading and HFT have made it possible for a few people to do the work of thousands. This has brought down the cost of trading.
This is a well informed and put together article. Up to date technically and the Author knows what he is talking about. As a former HF trader I endorse it, great piece.
I walked away and moved onto more exciting things like entrepreneurship. As the author states, HFT is like rebuilding gravity models every week, so your product is not sustainable and doesn't grow, cash generation from HFT also is usually not scalable on a per strategy basis.
My company builds memory ICs that sometimes sit beside the ASIC or FPGA on an HFT card. The bandwidth & latency numbers didn't surprise me, but the cost of power certainly did. No wonder customers are always crawling up our butt on power consumption.
A great overview of the space. The numbers and techniques are realistic and the article refrains from the moral debate (which you can find elsewhere...)
Why? What's the response time on an airbag release in an accident? On the ABS in your car? Your graphics card manages to rotate, scale, texture and shade a hundred million polygons per frame in around 1000/60 = 16ms. We're talking here about processing one UDP frame, usually under 256 bytes.
Why is this the only topic discussed by HN where technology suddenly becomes this frightening bogeyman?
You'd best look up the definition of "front-running". Reacting to an event the fastest makes you the quickest behind-runner, but you're still behind. Not front-running at all. Where did you even learn this term? Are you just regurgitating something you've heard elsewhere?
Wtf? An example of latency arbitrage is say, two identical fungible securities listed on different exchanges. An order arrives causing the price on one exchange (A) to spike up (the event), you react after that event and buy the security on the other exchange (B). You reacted fastest to public knowledge.
Front running would be if you were say sitting on a broker desk and one of your clients rang up and said to buy lots of A. Then you put in your own buy order before the clients one hits the market. That is front-running. You got in before that knowledge become public. Do you see the difference?
And you've merely tried to change the subject. Something can't be front-running if it happens after a publicly disseminated market event. Understood? Flash orders = front running. Trading ahead of client flow = front running. Latency arbitrage != front running. HFT market making != front running.
Can someone please tell us what is the return or for an hft fund vs. the market in the long run say 5 years. Also , are there hft funds available to general public for investing.
There are no funds (that I know of) that except outside investments. Most are proprietary trading groups meaning they don't solicit investment. As far as returns they can be big. RenTech Medallion has thrown up impressive numbers for years, many 50% or more. Most of the firms that I know of are north of 50% in the small to medium sized space. If you want to peek inside a larger one look at GETCOs S1.
This doesn't solve the problem in anyway. The issue is what if you have a different amount of buy to sell qty. Who gets their order filled? This is called matching and each market matches in different ways.
The typical way (and one of the leading drivers of latency races) is first in/first match. If you continue to FIFO match on your intervals there is still a tremendous advantage to being first into the interval.
If you aren't going to FIFO match that is fine, you just need to determine (and publish) your matching routine.
Virtually all trading volume today consists of buying and selling old securities -- essentially, legal claims on existing assets. The sale of new securities issued to finance the creation of new products and services -- for example, a company selling new shares via an IPO or issuing new bonds for investment in physical infrastructure -- represents only a minuscule portion of total trading volume.
I'm not sure having so many of our best and brightest minds going to Wall Street (and into high-frequency trading in particular) is a good thing, from a societal perspective.
--
PS. Whenever I read anything about high-frequency trading, I'm reminded of the following passage, written by John Maynard Keynes in 1936 -- 77 years ago (!): "Of the maxims of orthodox finance none, surely, is more anti-social than the fetish of liquidity, the doctrine that it is a positive virtue on the part of investment institutions to concentrate their resources upon the holding of 'liquid' securities. It forgets that there is no such thing as liquidity of investment for the community as a whole. The social object of skilled investment should be to defeat the dark forces of time and ignorance which envelop our future. The actual, private object of the most skilled investment to-day is 'to beat the gun,' as the Americans so well express it, to outwit the crowd, and to pass the bad, or depreciating, half-crown to the other fellow."
Source: http://ebooks.adelaide.edu.au/k/keynes/john_maynard/k44g/cha...
The more things change, the more they stay the same!