Mark Best
@MarkBestForex
Followers
609
Following
201
Media
31
Statuses
829
I model professionally. HFT, MM, rust.
Joined January 2015
Don't wanna be here?
Send us removal request.
Explore trending content on Musk Viewer
#RejectFinanceBill2024
• 2439694 Tweets
Julian Assange
• 601832 Tweets
Australia
• 332082 Tweets
WikiLeaks
• 286334 Tweets
Afghanistan
• 259190 Tweets
#FreeOguda
• 232087 Tweets
#FreeOsama
• 231555 Tweets
#ENGFAxHONOR200Series
• 161382 Tweets
Stanley Cup
• 156425 Tweets
Costa Rica
• 144172 Tweets
Oilers
• 136160 Tweets
Neymar
• 130103 Tweets
Panthers
• 120125 Tweets
Nairobi CBD
• 107578 Tweets
#OpeningYSLthXCharlotte
• 102733 Tweets
Brazil
• 98174 Tweets
Bangladesh
• 73222 Tweets
INI THE FRAME
• 69509 Tweets
#AFGvsBAN
• 64564 Tweets
Safaricom
• 60589 Tweets
Vini
• 56872 Tweets
McDavid
• 47466 Tweets
Gulbadin Naib
• 43374 Tweets
INI 6th THE FRAME
• 39236 Tweets
इंदिरा गांधी
• 39056 Tweets
Ronaldinho
• 30567 Tweets
CHARLOTTE AT YSL EVENT
• 22842 Tweets
Conn Smythe
• 15589 Tweets
Tommy Robinson
• 15132 Tweets
ふるさと納税
• 13208 Tweets
@airkatakana
I heard a story once that McKinsey gave a lecture to the new grads about work life balance at 7pm on a Friday.
1
17
626
Recent discussions about tick to trade made me do a more in depth analysis of the practical problems. Data is really bursty and the problem is actually worse than i thought. Thanks
@BruHFT_quant
for the inspiration.
8
15
98
I don't say enough nice things about python.
Being able to write stuff like
beta = np.linalg.pinv(X.T, @ X) @ (X.T @ y)
is absolutely beautiful.
5
0
66
OK, linear algebra in rust is awful.
I take back everything bad I've ever said about python.
5
2
35
Just finished this. A lot of useful ideas for parameter optimisation of hard to simulate systems (see mm). Author is ex getco. Its a pretty easy read and the stats are not too spicy.
1
1
30
If you have a trading strategy with a 20% chance of being profitable. If you keep running it for 6 months there's a 120% chance you'll make money. It really is that simple.
3
0
21
@TimMeggs
I heard one of the bond quants at DB did this. Not in the middle of the day but just disappeared and never came to work again. No one knew what happened to him and it was the days before social media.
1
0
16
Pro tip: if you want people to use your open source project. Don't commit changes that break the build and then tag them in a release.
2
0
6
Rust peeps, does anyone know a good tools for measuring code complexity and hopefully suggesting targets for refactoring?
2
0
7
@BeatzXBT
I'd actually make a counter point to this as if you get a burst of messages on the websocket, if your processing time is about 1ms, you can get quiet behind. Cpp/rust ipc is between 100 and 500 ns to put it into comparison. At least then the qs don't get backed up.
2
0
6
@BeatzXBT
@AirM4rx
@Alice_comfy
@levbeta
@DegenQuant
@sergeesolovev
@Dub0x3A
@OBS_orderflow
@cryptoquantHQ
@max_holloway
@yenwod_
Have you considered conflation. It'd basically just pull everything off the queue if you still have quotes on there or get a trade msg/other non quote. Then process the book and trades.
It saves expensive feature calculations when you get a lot of irrelevant updates.
1
0
6
@phoenixstealthy
@BeatzXBT
The only tricky thing with go is the gc. The production of garbage is correlated to the amount of work you're doing. That in turn is correlated to if the gc will run. I'm generally not a fan of gc based languages for hft.
0
0
7
ChatGtp et Al is really going to change education. I've been using it to learn German. The ability to ask specific questions is super helpful in working out the grammatical nuances. I hope this helps a lot of kids that don't like a subject because their teacher sucks.
3
0
7
Q: Are the CME crypto futures 24/7
A: yes. Sunday 6pm to Friday 5pm with one hour maintenence per day.
0
0
7
@tsoding
A guy in marketing using chatgpt built and fit some survival models without any deep knowledge of stats and econometrics. It was using Cox models and others. He managed to find a few really good insights from the data . This changed my opinion of what possible with LLMs.
2
0
6
Refactoring rust really is a joy compared with other languages. Even renaming variables in python is a nightmare.
1
0
5
1
1
5
@BeatzXBT
not all LOC are equal. When its green field you can get a lot done, if you are fixing someone else code or even your own it might take 2 days to make a 2 line fix.
LOC ~= Productivity.
1
0
6
@deedydas
If you don't embrace AI as a SWE then you will fall behind. I don't get the reticence about it. It makes much of the boiler plate stuff fast and simple and leaves more time for actual work. The skill is probably working out where to draw the line with regards to its use.
0
1
5
@DegenQuant
@levbeta
I was doing a bit more research into this and i think criterion benches give much better numbers than its possible to get live.
I'm assuming it's just because the cache is super hot with using the same message. In reality I get super different numbers than the bench gives.
4
0
3
@likely_a_DFS
@tekbog
This is why i think its better to be explicit with what is happening in memory.
Python has the old nested_list = [[] for _ in range(5)]
Which often causes issues.
1
0
5
@Dub0x3A
Well yeah but its a bit more complex than just higher is better. Its also down to instruction sets and cpu pipelining, membus speed and which cache you're using to share data. If they two process are bound to be on the same physical core and share L3 you also get a boost.
1
0
3
@BeatzXBT
Rather than starting with design and tech choices. Start with requirements. How fast do you want to be. How scalable?
1
0
4
@DegenQuant
@levbeta
Its also cope since i dont want to rework my deserialisers at the moment and im upset you guys are getting these numbers! Its interesting to see that you can get a lot more out of serde with a little tuning.
1
0
3
@ieszczyk
@BeatzXBT
@Dub0x3A
@soundsonacid
@BruHFT_quant
@quant_arb
It can and should be done with zero locking.
0
0
4
@BeatzXBT
Once you start using signals across instruments/exchanges it becomes a lot of data very quickly.
I know your feeling about python. I've written hft systems in python so know the limitations. I'm not sure how much you can offset with numba/ cython.
1
0
4
@Dub0x3A
Its better to count spins rather than use a clock as even calling the clock via the kernal libc functions costs about 50ns. This means if you get a reem of fills you will not pause after the first and you should run any acceptable cpu load. Dm me if you have questions.
0
0
4
@BeatzXBT
@volatilitynoob
@Dub0x3A
You can also just rewrite everything in rust and not have to fight this dog shit language all day.
1
1
4
i know there is a typo and no i am not paying to be able to edit tweets
0
0
5
@RootOfParadox
@BruHFT_quant
yeah, i was thinking about that. You end up quickly with a path integral but as you said most of the paths are short as they don't lead to trades. I was just surprised that this is a random day and you're still talking about peaks of 650 messaged per millisecond.
1
0
2
@Dub0x3A
@BeatzXBT
@max_holloway
@experquisite
@nayshins
Internal latency is nothing on network and network stack latency. You're always going to get hosed by someone with kernel bypass and a direct lines. Tbf you're learning all this stuff but becareful thinking speed in crypto == alpha.
1
0
4
@BeatzXBT
One ways to save time and avoid hashmaps is to use arrays. So on tick you convert the keys to an array index (instrument, base ccy, ex) then at least each thing after that would be a hashmap is an array index.
1
0
4
@BeatzXBT
@volatilitynoob
@Dub0x3A
If you're trying to push primitives via ring buffers then it's going to have a bad time. You can write a share memory ipc impl in cython and then push objects to it. Trying to push arrays is just going to suck.
1
1
3
@Dub0x3A
Yeah its normal. You can look at something like parking lot. The only alternative is to spin for x ms and then park. Waking up from parked state os always going to cost you. Last time I check it was on the other of 40-50 mics.
1
0
4
@Vertox_DF
Wrote my masters thesis on it. Can DM you a link. The exchanges do strange things. Also market making gets around a lot of the fees so they just said. We'll charge what we thing is fair. If there are obvious arbs they'll just internally match them. They can also just kick you off
1
0
4
@BeatzXBT
@0xstdsp
@volatilitynoob
@Dub0x3A
I might try this as I have been meaning to use pyo3. It seems like a perfect use case.
0
0
3
@macrocephalopod
Time is one of those programming problems that's way more complicated than you'd think at first glance.
0
0
4
@Nexuist
I've had this happen in Paris. But they then booked us in somewhere close by of a similar standard. Did they just say, it sucks to be you?
0
0
3
@PtrPomorski
I did a study with riskfolio looking at how various portfolios would perform out of sample. I looked at a few libraries. Riskfolio has HRP and a set of alternative risk measures. Anyway the most stable OOS was HRP. i don't remember what the best risk metric was. It's a good lib.
0
0
4
@BeatzXBT
Yeah, the computation of the hash normally requires summing all the chars. So its slower and dependent on string length.
Plus if you use a map the items are likely not contiguous in memory. That was the other reason I suggested arrays.
1
0
4
@experquisite
@Dub0x3A
@max_holloway
@BeatzXBT
@nayshins
There's some stuff like intrusive data structures and object pools that are a bit tricky in rust. Also it does kind of suck having to use get unchecked on vectors in an unsafe block if you want to avoid out of bnds checks. That said I hope I never use cpp again.
1
0
3
@cryptoklotz
@AlgodTrading
show me a country where the tax rate is 20-30% on a 7 figure income?
1
0
3
The UNI pump was at least good for testing the robustness of infrastructure.
1
0
4
@BruHFT_quant
@Dub0x3A
I know what the reason for my extra latency is. I'm not going to say it here. I think the library I'm using is significantly slower than the way you're doing it and I can only assume you've completely hand rolled it. Bravo. Its a good effort.
1
0
2
@simplex_fx
Unironically for trading market data its the only way. You can also read it insanely fast by memory mapping.
1
0
4
@GuilleAngeris
Have you tried ubunutu? The nice thing is, since it's used a lot there's actually information online about trouble shooting.
2
0
4
@degentheory
@Dub0x3A
Its mainly they don't like some of the stuff you have to do to get around the borrow checker to make efficient data structures. Hft tends to abuse unsafe code.
0
0
4
@Dub0x3A
Its also worth reading more about why println! Is bad. It's why you don't want the os involved. Why interrupts are bad and it's about the cost of mutexes etc.
When looking at latency tails the os is the killer and can cost milliseconds. It also gets worse with higher contention.
1
0
2
@gdelujo
At a big investment bank I heard someone say in an elevator that java is basically the same as html but with loops. So I suppose you're right.
0
0
3
@BeatzXBT
@laurentzeimes
This is worryingly similar to one of the blog example i have. There is a pattern for doing this it turns out that is better as you basically want to know everything that is wrong.
0
0
2
@Option_Bull
@xoreax_eaxeax
@BruHFT_quant
@RootOfParadox
@serzhiio
@Dub0x3A
ok, my bad. Its a bit similar.
some cpus have instructions that take two arrays and return an array. Its the same as doing the same operation (+-/*) etc multiple times in one clock cycle. It can be done by the compiler or you can hand code it.
1
2
2
@BeatzXBT
@frothybeverage
I agree it's better trading than not. If you do switch languages, make sure you use rust. Not just for performance, the stability and time to market is alpha.
1
0
3
@Dub0x3A
Deleveraging is an emergent property. I'd say mean reversion is more of an optimal stopping problem.
0
0
3
@BeatzXBT
Well there's also data oriented design. I have a data structure which is called an indexed sum. Its like a hashmap but with a sum attached. When you update and element, you subtract the old value and add the new one. Then updates are O(1) rather than O(n). Its useful for this.
1
0
3
@AlgodTrading
Having a complex system prevents it from being fair. The loopholes are a feature and not a bug.
1
0
1
@BeatzXBT
@quant_arb
Can you just over write the bba and set a dirty flag rather than using a q? You only really need a q for trades. Then at least you won't get backed up.
2
0
3
@Dub0x3A
Like fpga have lower clocks but much more complex instructions so do more work per clock tick.
0
0
2
@RootOfParadox
@DegenQuant
The best way to do it is in your code output the market data sequence id that triggered the order and the order id as a tuple. Then get the 2 timestamps from wireshark stamped by the nic. That's the "true" warts and all latency.
The issue is u can't get splits on what's slow
1
0
3
@predict_addict
What is the big O complexity of training as a function of the number of parameters. Surely this could make a massive improvement to the work required to train them?
0
0
2
@BruHFT_quant
@Dub0x3A
This is impressive. What chip is it on? On an amd 3950x I get about 350ns. So I'm wondering now where the fat is...
2
0
3
@quant_arb
Its good that you highlight this. Paths that lead to exchange operations are often much longer. If you have bad conditional latency there can be a few reasons. The conditional latency distribution is also hard to measure if you're throughput is bad.
1
0
3
@DegenQuant
OK cool. It's often worth splitting it out since there's a lot of faff with sending orders. Do you look at the tail latencies also? I've found rust has insanely consistent latency. It's like 1.5-2x from median to 99th percentile.
1
0
3
@PtrPomorski
Its maybe worth breaking hft into groups. Not all of it is latency arbitrage and I'd say that likely a small part due to capacity limitations. There's are a lot of hft strategies which require speed for various reasons.
The main thing people use hft for is to reduce cost of MFSs.
0
0
1
@Markos_mom
Well, they're probably technically not wrong. I bet its a long US stock strat and the 12 years seems like it is the significant part of this. For everyone dumping on this, it will probably continue since i cant see how they're going to change monetary policy.
0
0
3
@quantymacro
At DB there was grad party at a bar when id been there a couple of years. One of them smashed a toilet, another was outside throwing up so much they made a river across the pavement. I also got told to go fuck myself by one of them. That class was totally feral.
0
0
2