MFW i log into twitter

RT @AdilSlm: Many many many thanks to the lecturers of the first week for their dedication (could only get some of them on this photo. ) #….

in particular I'd love to see more events like this being organized in africa (and more generally the global south). great learning opportunity for the locals and visitors alike.

besides the event, i was fortunate to spend some additional time in the beautiful country of Senegal and enjoy the hospitality of the locals. i have never met people of such generosity and selflessness before. i hope to be back very soon!.

see more pics and info about the programme in this thread by Eugène:.

i had the incredible honor of teaching a mini-course on RL at #MLSS2025 in Senegal. very inspiring to see the dedication & excitement of all the young people that showed up to learn, not to speak of the heroic work of the organizing team led by @AdilSlm & @eugene_ndiaye !!!

RT @eugene_ndiaye: @dohmatobelvis @neu_rips teaching RL

RT @eugene_ndiaye: Some 📸🤳from the ongoing #MlssSenegal2025 🙌🏿

RT @umutsimsekli: Anyone from #iran looking for a phd/postdoc/research internship in statistical learning theory, deep learning theory etc,….

RL theory seminars going out with a bang for the summer. make sure to check out @NnekaOkolo4's talk on the best offline RL method for linear MDPs 🔥🔥🔥.

RT @LucaViano4: Our new preprint is online ! Structural assumptions on the MDP helps in imitation learning, even if offline :).Joint work w….

wanna know how to do inverse Q-learning right? read this paper then!!.joint work with the best team of students ever ♥️.

i generally think this work uncovers a space of ideas with huge potential impact, and many beautiful intellectual challenges to address. check out the paper for more details if you also find it interesting: 12/12.

i learned a lot while working on this project with this amazing team, and am very happy with the results! we have tons of open questions and ideas for future research, mostly about introducing function approximation and making the algorithm scalable. 11/n.

we also have some cool model-selection experiments that demonstrate the ability of our approach for computing similarities between Markov chains, which could be potentially very useful for representation learning in RL.10/n

interestingly, variables λ_X and λ_Y can be interpreted as "encoder-decoder" pairs giving the conditional distributions of Y|X and X|Y. we ran some experiments to illustrate this, e.g., here we discover the latent structure of a block Markov chain without prior knowledge.9/n

the algorithm is easy to implement and comes with nice sample-complexity guarantees. notably, no "concentrability" / "coverage" assumptions are required for the state distributions, the guarantees only depend on the sizes of the two chains.8/n

. which we can solve by a stochastic primal-dual method that we call SOMCOT (stoch. opt. for Markov-chain OT). the idea is to observe that all gradients can be computed as *marginal* expectations over transition data and use this to build unbiased gradient estimators.7/n

the main idea is rewriting the distance as a linear program whose constraints do not explicitly feature the transition kernels, only the *marginal distributions* of x->x' transitions. this allows us to formulate the problem as a stochastic min-max optimization game. 6/n

we answer this question in our new work by developing a new stochastic optimization method for computing OT distances between Markov chains. the method is fully incremental and works without directly estimating the transition kernels. (so "model-free" if you like such terms).5/n.

in our neurips'24 paper, we developed some neat analytic tools an some computational methods to do this, but left a major open question behind: how to estimate distances for *unknown* Markov chains based on *samples only*?.4/n