Over the past 36 hours, @jackminong @benxneo and I built DeepTrust.eth at @ETHGlobal. tl;dr SoTA on-chain proofs for LLM Model Executions. We make sure that open-source LLM model providers are honest about the model they are serving. Here's how it works (🧵).

Check out the repo here:.

10/ We won a few prizes! 🏆. - Best Use of Compute over Private Data.- Innovative Applications of ZK

9/ Tokenomics. We ask LLM providers to stake some tokens on the network and incentivize/slash based on their behavior.

8/ The Tech (part 2). @0xPolygon to store the commits (or PoW) onto the chain so it’s publicly verifiable. p.s thanks to @DablClub for pointing us in the right direction in learning this.

7/ The Tech (part 1). @LitProtocol provides us with TEEs which ensure that we have a tamper-proof environment. No collusion can happen. @NethermindEth for providing us with research feedback and direction to go into.

6/ Is it costly in time?. Short answer: No! We can even verify up to 200x faster than inference. If your prompt takes 5 seconds to output, we can verify the model in under 0.02 second

5/ How? (part 2). But how are we outputting the hashes? We made our POC based on adding a plugin that computes the hash. We envision adding this to the base transformer architecture so all LLM models are now publicly verifiable. Tough but hopeful.

4/ The last hidden state. We decided to use the last hidden state since it involved all other states before. Hash this state and compare it with hashes other nodes computed

3/ But isn’t LLM deterministic?. We figured out that the RNG or temperature only comes at a certain part of the LLM (after the LM head). That means the hidden states in the transformer's architecture will always be deterministic. Huge credits to @jackminong for the intuition

2/ How?. DeepTrust.eth is a network comprising multiple LLM providers (different entities). Each LLM provider provides the network with a computed hash and the network works together to verify the hash is coming from the model it claims from.

1/ Why?. - Centralized inference requires trust in the inference provider not to alter the model or data ie How do you know you are getting the LLM you paid for?.- Centralized entities might manipulate or censor outputs, impacting model fairness.

proud of @linathelam and the @helicone_ai team for making this!.

foggy morning in the city

facts.

full demo by @AlexReibman on what we built last week at @mintlify's hackathon ✅.

rice cooker is the real-est MVP. from on waverly in chinatown

RT @arpitingle: i love interesting and ambitious projects such as . @patrickc's arc institute.@natfriedman's vesuvius challenge .@stewartbr….

it’s the socks for me

at airflow summit 2024 in SF today . interesting talks + good food + nice swags