My contribution to the #Wordle discourse, with help from @VincentTjeng and @evanrsparks. A definitive look at the game: insights, heuristics, optimal strategies, and the limits of what is possible. https://t.co/jSRXV1BYr5

"Deadlines are final and no extensions will be given" You got me @IEEECDC2025 --- I totally fell for it! Happy April Fools everybody!

Can your AI keep up with dynamic attackers? In a paper to appear at #AISTATS2025 with @avibose22 @LaurentLessard and Maryam Fazel, we study robustness to learning algorithms to dynamic data poisoning attacks that can adapt attacks while observing the progress of learning

This week's #Fiddler puzzle is a random walk: From (k), you hop to (k-1) with probability 1/k and hop to (k+1) otherwise. If you start at (2), what is the probability you eventually hop to (1)? I wrote a short tutorial here: https://t.co/V0QhIIEiLw

This week's #fiddler: Assign distinct prime numbers to the vertices of a dodecahedron so the sum of the vertices of each face sum to 2025. I modeled this as an integer linear program. Here is one possible solution. @xaqwg My write-up: https://t.co/vFbpDurcNP

Update: after 2 hours and 35 minutes on the tarmac, we are finally deplaning.

The pilot keeps saying there is "nothing he can do". You mean to tell me there are no other gates available at the airport? We can't deplane on the tarmac? This is a full flight with 8 seats across and we've been on this plane for almost 10 hours now.

Hey @AerLingus I'm on a flight from Dublin to Boston and we've been stuck on the tarmac at @loganairports after landing for TWO HOURS (and counting). Apparently other planes are being de-iced which is preventing us from deplaning...

This week's #Fiddler: what is the minimum-energy way to pack 9 particles in a square box, where energy is measured as the sum of 1/dist over all pairs of particles? Turns out it's NOT a regular 3x3 lattice! @xaqwg write-up: https://t.co/tBfK0Jy2yq

Halloween #fiddler puzzle: A bag contains N Reese's cups and an unknown number of candy corn pieces. You reach into the bag k ≤ n times at random and pull out a Reese's each time! How many candy corn do you expect to be in the bag? @xaqwg My write-up: https://t.co/z2DxoGC9xr

Great pleasure of hosting @LaurentLessard at USC who spoke about "An automatic system to detect equivalence between iterative algorithms" 🙂

Thanks for reading! Let us know if you have any comments/questions!

One such example (we discuss others in the paper) is quasi-Newton methods. For these methods, line search serves the purpose of guaranteeing global convergence rather than finding a "best" step size. Once we enter a local convergence regime, adjustments are no longer beneficial.

So, at least in the experiments we performed, having a "faster car" generally pays off. Adaptive BT can generally be used as a substitute for regular BT. That being said, there are certain cases where there is little or no benefit to making BT adaptive...

This is a sample of the many numerical experiments we report on in the paper. We also looked at linear inverse problems, the Rosenbrock function, more datasets... In all cases, we observed the same trend: adaptive BT requires fewer function and gradient evals than regular BT.

Experiments (nonconvex): We solved matrix factorization problems, again using GD and AGD. Once again, adaptive BT is faster across the board, sometimes by a lot! We have many more experiments in the paper; all show the same trend. Adaptive BT is a win-win.

One more thing: Our adaptive scheme has a tuning parameter ρ, but we consider it parameter-free since in all experiments we performed, we used the same value (ρ=0.3). This is a nice feature of adaptive BT; it can be implemented on top of regular BT without any additional tuning.

Experiments (convex): We solved logistic regression problems using GD, AGD, and Adagrad on several real-world datasets. In all cases, adaptive BT leads to fewer function & gradient evals, and faster wall-clock time. Here are the results using the Armijo criterion

we prove our car is faster than the competition, but we take a different road to get to the finish. That road may be longer or shorter. On average we arrive sooner, but if our road happens to be a lot longer, it might take us more time to arrive. More on this later...

Important point: our theoretical results do not prove that adaptive BT is always faster than regular BT in terms of total time; only that at any iteration, using adaptive BT requires fewer stepsize adjustments than regular BT to get to the next iteration. In other words,

Theory: For convex problems, we prove adaptive BT requires fewer adjustments to produce a feasible step size vs regular BT for two popular line searches: Armijo and descent lemma. For nonconvex smooth problems, adaptive BT enjoys the same guarantees as regular BT.