1/ ACCESS TO GPUs matter -Most underestimate GPUs needed to power AI apps -Running LLM requires $$ compute contract (monopolized by large clouds GCP, Azure) -Cheap compute as competitive advantage (subsidizing OpenAI API costs w/ Chat-GPT profits) -No compute = no LLMs at scale

2/ PROPRIETARY DATA = moat -Don't have enough "good clean" data on web to train new LLMs -Data source tracing impt for mission critical use cases -Specialized data sets for specialized workflows w/ data product flywheel is crucial moat for apps e.g. @runwayml -effect of GPT-5/6?

3/ INFRA MARGIN COMPRESSION -Large clouds (AWS, AzureML, GCP) offer cheap AI infra stack -Low margin infra biz w/ cost-cutting giants makes competition for price sensitive customer hard -New players appeal beyond cost (self-host, decentralization, 10x dev UX) @togethercompute

4/ EVALUATION IS AN UNSOLVED PROBLEM -Difficult to evaluate whether or not an LLM responded accurately to prompt/answered question -Evaluation today has been qualitative/hand-wavy -Need a more quantitative, data-driven solution to evaluate LLMs

5/ OPEN vs. CLOSED TENSION -Open vs. Closed models ( @OpenAI vs. @Meta LLaMA) @huggingface hub of OSS community -Self vs. cloud hosted AI infra ( @MosaicML vs. @Azure ) -How will cos self-host AI apps, maintain sovereignty when models/tooling hosted by closed player w/ data loop?

@OpenAI @Meta @huggingface @MosaicML @Azure 6/ in summary: ✅access to GPUs matter 💾proprietary data = moat ⌨️infra margins compressing 🕵️evaluation is unsolved prob 🤗open vs. closed tension what did I miss? re hardware, fine tuning & beyond.. READ MORE:

@graceisford @huggingface @runwayml @MosaicML @Stanford @StanfordCRFM Great thread! I am learning a lot about @huggingface and want to develop a lot of tools for my own personal uses. They take a lot of the "programming" lift out, making it easy to understand/implement

@graceisford @huggingface @runwayml @MosaicML @Stanford @StanfordCRFM @Memdotai mem it