New role: Applied AI engineer at @zapier 🎉 We've been moving fast on AI since well before the ChatGPT era and we just formalized an AI Engineering job family to go farther, faster. I'm grateful for being here through the AI wave and the support I've had working towards this 🙏

🤖 I’m prepping several classes on LLM evaluation for AI products right now. If you’re building in this space, these are the 3 best resources to get started, and I recommend reading them in this order: 1. “Task-Specific LLM Evals that Do & Don’t Work” by @eugeneyan

Compiling my top recommended resources for AI engineer and all paths lead to @HamelHusain 1. Applied LLM the one pager 2. Eval 3. Look at your data 4. Look at your data 5. Jargon What else belongs on this list?

Apply here if the job description resonates with you:

My team at Zapier is hiring one more AI engineer. If you are working closely with LLM and have hands on experience with eval and iterating on LLM systems in production get in touch! Zapier is 100% remote distributed globally. This role is for North America timezones.

Very cool to see @wadefoster on @CogRev_Podcast talking Zapier's AI Revolution! I was surprised to see as I was queuing up my listening for my morning walk. https://t.co/5o5e4ADfVO

Anthropic's Contextual Retrieval technique is a game-changer for RAG systems, addressing the pesky problem of lost context when chunking documents. This instructor implementation takes it a step further by adding async processing for improved efficiency. As someone who's spent

I'm getting to the point with Cursor that it opens doors for exploring (and shipping) a lot that wouldn't make sense otherwise (would take too long the old fashioned way) You have to have patience to learn how to wield it and embrace the rough edges..

interesting timing

pro level career advice

AI isn’t going to replace you, a generalist with no ego spending $1000 on AI tools is going to replace 10 of you.

✅ Dishwasher, Busboy, Waiter At Denny's

For example if a company happens to have **15,000+ emojis** in their Slack @zapier 👀👀 and you need to pick the right one for a reaction trigger

Challenging LLM problem I've been working on: Fill complex form from user query, pick best item(s) for dropdown(s) Query may contain relevant details Many thousands of items per field Processing items ahead of time is not an option I can imagine many over engineered solutions..

Anyone else banged their head against openai structured outputs caching? Seems to cache heavily on schema name (makes sense) so that any changes to schema are ignored Ended up suffixing with _v{n} and bumping for every small change. A pain in early development stage @OpenAIDevs

I gave ChatGPT a fresh shot. It had been a while. 4o was great in chat with a few files uploaded. Then I tried voice mode... Put a lot into a 7 minute voice chat info dump... came back with "sorry i seem to be struggling right now" lost it all, not even left with a transcript 🙃

The demo bar at Zapier is so high now @ckor broke it

Has anyone decided whether sonnet or o1-mini is better for refactoring and nontrivial Composer tasks? I've had good / mixed results with both sometimes and choosing at random at this point. Mostly sonnet. Have been surprised that gpt4 lets me down so much.

detour driven development is my superpower I might not have finished the one thing but I cranked out 10 things I dug up along the way (that help with the one thing once I get to it)

New Cursor update ✅✅✅ my top wishlist items for Composer Impressive how fast Cursor is evolving. I'm finding more magical moments with it as I learn how to use it better.

Cursor shines for smaller focused changes. Highlight block of code, trigger inline chat, mention a couple files / docs, give feedback, commit often to avoid Cursor edits making a mess. Use chat sidebar as ChatGPT / Claude replacement. Occasionally reach for Composer esp for tests