ccmdi
@ccmdi0
Followers
44
Following
2K
Media
40
Statuses
227
documenting knowledge space | @geobench_ & @osintbench
United States
Joined December 2022
Introducing @osintbench, a benchmark testing open-source intelligence (OSINT) capabilities of language models. It encompasses common open-source intelligence tasks like geolocation, identification, temporal analysis, and general analysis.
2
6
18
"Penrose effect": prominent people in the AI field wrote books two or more decades ago. It looks now more and more like some of their fundamental ideas were wrong, but that would detract from some of their life's biggest accomplishments, so they "dig in.".
1
0
3
1 is a technical limitation: the models are too dumb to be useful at long context, devs know their codebases at an intuitive level and retrieval patterns are just much more accurate. 2 is a strategic limitation: if you want productivity gains using AI, it isn't your replacement.
0
0
2
1. these repositories are 100k+ LOC on codebases the devs are familiar with.2. the developers were not using AI to supplement work and instead detracted from their own time (idle time). any other case forces the intuition that with AI, you would at least break even.
We ran a randomized controlled trial to see how much AI coding tools speed up experienced open-source developers. The results surprised us: Developers thought they were 20% faster with AI tools, but they were actually 19% slower when they had access to AI than when they didn't.
2
0
2
non-objective fields like art scale slower with more resources because there's no predetermined goal. it's research but for human preference.
0
0
1
the world is interesting but also impossible to comprehend in full, which is dissatisfying.
1
0
3
podcast hosts need to be just a little bit stupid to relate to the audience.
1
0
1
RT @thegeneralist01: @ludwigABAP we need a new American Psycho -- the Silicon Valley Programmer.
0
3
0
it is actually INSANE there is no good calendar app. it cannot be that hard. event tags/templates. box selection. responsive drag and drop. universal sync. its like 4 things and nobody has it, why.
0
0
1