Kavi Gupta
@akavidemic
Followers
485
Following
163
Media
27
Statuses
198
PhD student in PL/ML at MIT. Working towards interpretable ML using symbolic-program-inspired structures. Formerly at UC Berkeley. they/them
Cambridge, MA
Joined April 2020
Hi! I made this account to separate out my academic posting from my non academic posting!.
4
2
32
"Duality" is one of those terms that just kinda means a completely different thing in every context it's used in in math. Very annoying.
(2) it’s important that all ways of “dualizing” are not the same! and all do not amount to a sameness of content. my understanding is that people don’t take homology and cohomology to be the same in content, even tho you can describe their relationship by “just” reversing arrows.
0
1
7
Some CEO in 2027 is going to be talking about how 2022-2024 era RLHF had a systematic issue with reducing stylistic variance in the output distribution and be treated by a genius despite this having been obvious for years.
0
1
20
Legitimately unsure what the point of giving reviewers 2 weeks to bid and 4 weeks to do reviews is, everyone knows it takes not nearly that long.
0
1
3
RT @swarat: Day 2 of the @neurosym summer school started with a continuation of the tutorial by @akavidemic, @atharva_sehgal and me on neur….
0
3
0
RT @swarat: The @neurosym summer school is off to a great start!. In the morning, @atharva_sehgal, @akavidemic, and I gave the first part o….
0
17
0
The main issue is that mathematical proofs on paper are allowed to be a little informal in ways that people accept and we just aren't there yet with computer proof systems. Hoping llm assisted proof search fixes this to some degree.
0
0
4
Like you will not convince me that a single person who has a similar amount of experience using sigma notation and for loops actually thinks the for loop is easier. That's just blatant familiarity bias. But formally specified theorems using Lean or Coq >>>>.
3
0
6
My extremely unpopular hot take is that mathematical language works really well for the kind of thing that people complain about like sigma notation but increasingly poorly for more complicated stuff like in this image.
Unbelievable how mathematicians use all these incomprehensible symbols and jargon. Can someone translate this into pseudocode?
2
1
10
This algorithm computes factorial. We can conceptualize it as steps 1 to x. At step k, it computes k * factorial(k-1). Factorial(k-1) has Theta(k log x) digits, and k has Theta(log(k)) digits, so the computation time is Theta(k log² k).
1
0
1
My hot take here is that this isn't a failure of NTP it's just a failure of the model at being a reasonable world model. Being able to answer variant riddles correctly would increase not decrease its score on NTP
@akavidemic I'm agnostic on whether or not it is inherently incompatible. But some of the recent examples where ChatGPT fails when asked common riddles slightly modified to become trivial suggest that there are pitfalls to NTP. How would you suggest fixing NTP to answer those correctly?.
1
0
6
A significant failure of scientific communication in machine learning is the fact that a lot of people seem to think that next token prediction is inherently incompatible with true understanding of a problem . P(x) = P(x[0]) P (x[1:] | x[0]) means this definitively isn't the case.
3
2
14
What is the complexity of this function? (only allowed to be called on non-negative ints). def f(x):. if x == 0: return 1. return x * f(x - 1).
6
1
5
RT @raunakdoesdev: @akavidemic My guess is they are feeding a truncated version of the site in as input and the truncated version is cut of….
0
1
0
Wait is Google doing character level tokenization??? Why is it getting "New York Bay" wrong as "New York Bar"????
3
1
8
Is there a named concept describing a weaker version of independence? Like. For all y1, y2; P[X | Y = y1 ] ≥ k * P[X | Y = y2]. Obvs for k=1 you recover independence but for k =0.5 you'd have like a softer version.
1
1
6
I personally would suggest that CS is not about P=NP, as it is mostly about true statements.
CS is not about computers. It's about computing. The name is wrong. The physical device doesn't matter. It's all about algorithms, data structures, complexity, information theory, computation theory, cryptography, etc. CS is P = NP instead of study of a physical device.
1
1
16