Accidentally said "hard" instead of "non-trivial" and they kicked me out of SF

made it open-source:

Built PR comments for Claude Code

full article:

Guide from @AnthropicAI on when to actually build agents 👇 The golden rule: "Find the simplest solution possible, and only increase complexity when needed." The complexity ladder (only climb when you must): Single prompt → Prompt chaining → Routing → Agents However, most

Full article:

Prompt engineering = write a good query Context engineering = curate what the agent sees, iteratively Key insight from @AnthropicAI's guide: More context ≠ better performance "As tokens in the context window increase, the model's ability to accurately recall information

suggested infra: (read my last post about why to use refutations and support facts)

Why RAG works better for fact-checking: - Answers are grounded in retrieved context. - Citations = verifiability. - Hallucination risk is reduced. - Updating knowledge = add documents (no retraining).

What about RAG? - Retriever: pulls highly relevant, trustworthy papers or datasets (with metadata) - Generator: uses those documents to craft the answer. - Grounding: you can cite the exact evidence.

Websearch as a solution: - Links are ranked by popularity, not necessarily relevance. - LLMs then have to parse noisy pages - No guarantee the right evidence is surfaced.

Problem: When you ask a plain LLM a scientific question, it’s like a student relying only on memory. Sometimes they recall correctly, but often they hallucinate or cherry-pick.

If you want to zero-shot scientific claim verification with LLMs, should you use websearch or RAG? 👇

"Agentic coding is a skill that scales with your technical knowledge."

Whats even more surprising (at least to me) that this approach outperforms websearch -> more on that tomorrow. Link to the paper covered in this post:

That’s it. With just prompting + a retriever, GPT-4 (!) was competitive with fully supervised models in scientific claim verification....would be super interesting to see how current frontier models would perform

Step 4: Scale it up. Use a RAG to fetch top-k abstracts and run the verifier on each claim–evidence pair you want to check. Aggregate results → most likely verdict + supporting citation.

Step 3: Give you model of choice a citance + abstract of the cited paper, and ask: Does this paper SUPPORT, REFUTE, or provide NOT ENOUGH INFO for the claim? Return verdict + short rationale.

Step 2: Generate refutations. Use any AI to flip the meaning: -> “LiFePO4 does not have superior cycle life.” Now you have SUPPORT vs REFUTE examples.

Step 1: Collect “citances” (citation sentences). Example: “Prior work shows LiFePO4 has superior cycle life [Smith 2020].” This sentence is a claim, and the cited paper is its evidence.