Keywords AI (YC W24)
@keywordsai
Followers
1K
Following
638
Media
126
Statuses
388
The best LLM engineering platform for AI teams 📍Community: https://t.co/JAZ9auu0hr
Joined March 2024
We worked with @OpenAI to build a native integration for the OpenAI Agents SDK. Today, we are very excited to launch as a tracing processor. With just a few lines of code, you can trace all your agent workflows and debug them much faster. Check out this quick demo to get
16
78
225
We're thrilled to announce native support for @vercel ai SDK integration!. Get started with our integration guidelines here:.
1
2
39
Excited to announce our integration with @inference_net - a new LLM provider hosting faster and more cost-effective open-source models!. Inference[dot]net can reduce your LLM costs by up to 90%. Models like Llama 3.3 70B, DeepSeek V3, and DeepSeek R1 are significantly faster than
2
8
29
🔬 Exciting news: We've launched Lab & Testsets!. 📡 Lab: A spreadsheet-style editor for running prompts and models across multiple test cases. Import testsets to easily test, evaluate, and optimize your LLM outputs. 📊 Testsets: Easily manage and organize test cases. Import a
0
4
19
IMPROVEMENT: more observability features in the side panel . We've redesigned the side panel and table view in Logs to improve observability. Now you can easily view details or take action on an LLM log with just one click. Check it out and send your first log to @keywordsai
0
2
8
There are only 5 days until the deadline to apply to @ycombinator S24. Can't recommend Y Combinator enough for startups!. We first applied to YC S23 with our one-stop job search platform but weren’t accepted. Pivoted in September & made it into W24—life-changing! 🎉. Like this
0
1
11
NEW: New graphs for monitoring LLM usage. We’ve just released new graphs to help you track your LLM usage more easily:. - Average prompt tokens .- Average completion tokens .- Average tokens .- Max tokens per minute . These new insights make it easier to spot trends, optimize
1
3
11
🚀 Want to export your LLM data in just 20 seconds? . Let @keywordsai show you how it's done! Check out our demo video below.
2
2
10
Thanks for the shoutout @tryfondo!.
🚀 @keywordsai launched! A unified DevOps platform to build software with AI. "Dead simple way to deploy & monitor AI apps with 2 lines of code." 💯. 🌐 💡 LLM observability for AI startups: Every feature you need to build, deploy, and scale your
1
1
10
Over 350M LLM requests were processed in Keywords AI last month. More LLM engineering teams are choosing Keywords AI to build AI products faster and more reliably. Don’t get left behind — set up your free LLM dashboard today!
0
2
8
NEW: More evaluation metrics are now available on our platform!. We've added more @ragas_io metrics to our Evaluations page, making it easier for you to assess your LLM outputs. ❤️ Big shoutout to our intern Alex for adding these metrics
0
3
8
🤘 @keywordsai was featured in the top 100 API startups backed by Y Combinator!. Our customers love us for using a unified API endpoint to call 150+ LLM models/providers. Shoutout to our friends at @retellai and @resend for also making the list (link in comments).
1
2
9
Happy March :). Excited to introduce the new layout of Keywords AI 💥
0
3
8
Introducing our new Calendar component. As more users use this component to filter LLM logs within specific time intervals, we decided to redesign the calendar to provide a better experience. A huge shoutout to our "Chief Designer".@Andydy42 ❤️
0
4
7
GPT-4.1 is now live on the Keywords AI platform!🚀. GPT-4.1 vs. GPT-4.5 👇. Try it yourself and see how the new model performs with your prompts!
0
2
7
New: Integration. We now support @inference_net as an LLM provider. They offer open-source models like Llama 3.1 at 50-90% lower costs compared to other providers. Try it out in our model playgroun
0
3
8
👩👩👦👦 New Organization Feature Update. - Create teams for your projects under 1 organization for easy switching. - Invite teammates to specific teams for better access control.
0
2
8
Managing prompts back and forth with the team is frustrating. You can now share the prompt with your team via a URL and invite them to collaborate on it in a shared workspace.
0
2
7
NEW: View details of historic prompt versions. You can now view the details of each previous prompt version. This includes configurations and variables for every version. With this update, you can review changes and roll back to an earlier version with confidence.
0
2
7
IMPROVEMENT: Full-text search in Logs. We have improved the full-text search in Logs to let you search any LLM logs faster. Watch the video below to see how it works on the platform.
0
2
6
Over 100k+ LLM requests are cached daily on @keywordsai. Our Caches feature helps developers save costs during development and increases production speed. Try it out and share your experience!. Check out docs here:
1
2
7
should we do a $1 hackathon?.
should we do a $10m hackathon?.
0
1
7
🎉🎉 Say goodbye to using spreadsheets to manage prompts - we just launched a prompt management feature where you can easily collaborate and iterate on prompts. Features you might find helpful:.- dynamic variables.- open prompt in playground.- version control.- code view/export
1
3
7
NEW: Custom Properties. You can now index customer properties for faster filter and search performance in Dashboard and Logs.
0
0
7
👩👩👦👦 Free users can now add one team member to their organization!. Invite a teammate to collaborate on LLM monitoring, prompt engineering, and output quality improvement!
0
1
7
🚀For developers who are using OpenAI SDK🚀. Call 200+ LLMs with OpenAI SDK!. Thousands of developers are using @keywordsai's LLM proxy to call 200+ LLMs with 2 lines of code change. Get started 👇
1
3
6
New integration: GPT-4.5 preview 🔥 . We just integrated GPT-4.5 into our platform! Now you can build, test, and evaluate your AI applications with this powerful new model.
1
3
6
🚀 Introducing Caches🚀.Supercharge Your LLM Calls! . Our new Caches feature allows you to store and reuse LLM responses, eliminating redundant API calls. - Deliver instant responses .- Reduce LLM costs .- Ensure consistent, high-quality outputs .
0
2
5
NEW: Prompt variable logging support. You can now view LLM logs with variables directly in the side panel. This makes it easier to separate the original prompt template and the populated version, streamlining debugging.
0
4
5
🦙 The new Llama 3.3 model is now available on the platform. Enjoy Haiku-level pricing and fast speed, while having sonnet-grade performance. Give it a try on our platform today.
0
2
5
🌟Unlock the potential of your custom LLM with a golden dataset! . Our latest Dataset feature lets you curate the perfect dataset for fine-tuning. 🚀. Check out the video to see how it works!
1
2
6
NEW: Word & Character count for LLM results. Now, you can easily track your LLM log's word and character count. This feature helps you evaluate outputs when you need to limit text based on actual words or characters instead of tokens.
0
2
6
NEW: Create your own Human Evaluators. We've just released new features for Human Evals. You can now create custom human evaluators and annotate LLM logs using your own metrics.
0
2
6
NEW: Draggable dashboard charts. You can now easily drag and rearrange dashboard charts to customize their layout just the way you want. Try it out and make your dashboard truly yours!
0
2
6
💪Thanks for the shout out!.
After running many tests, @AnthropicAI's Claude 3 Haiku is better at code than GPT-4-turbo AND its 40x cheaper. You should try building with it. p.s. If you want an easy way to switch between @OpenAI and Anthropic you can use @keywordsai from the newest @ycombinator batch.
0
0
6
❗LLM monitoring ➡ Multimodality monitoring❗. Launching new features on Friday is thrilling!. We're officially multimodal! Our unified Model API now supports 200+ LLMs, embedding models, and audio models — all monitored on a single platform!. Explore our new integrations:. -
0
2
6
Got a large dataset for testing prompts?. Try @keywordsai to run hundreds of experiments simultaneously. Upload your datasets -> Import prompts -> Test them all in Labs. Check it out here 👇
0
2
6
UPDATE: We've just launched new vision models. Here's what's new:. - qwen-2-vl-72b-instruct.- groq/llama-3.2-90b-vision-preview.- groq/llama-3.2-11b-vision-preview. Plus, we're improving image support on our platform. Stay tuned for more updates :).
0
0
5
NEW: Prompt version monitoring. You can now track prompt versions by linking a prompt ID in your code and deploying new versions in the UI. This helps you track the performance of each version, ensuring you're always using the best one. We are making it easier to test, deploy,
0
2
6
🚀 Exciting news! . We've just shipped the UI for our load balancing feature!. 🎉 Now, you can add credentials and set weights to balance LLM requests between deployments effortlessly.
0
1
6
Thanks for the shoutout @coffeewithone! We all love @octolane_ai ❤️ the incredible AI CRM!!.
We are pushing insane number of AI features on @octolane_ai in the next 3 months and fell in love with @keywordsai! As if they thought of everything we needed already. ❤️
0
0
6
0
2
5
NEW: Human, LLM, and Code Evaluations. We're excited to launch the Evaluators!. With Evaluators, you can:.✅ Build custom LLM-as-judge evaluators to quickly assess your LLM outputs. ✅ Easily involve Human Evaluators when complex cases or outliers happen. ✅ (Coming soon) Use
1
3
6
IMPROVEMENT: Redesigned prompt list UI. We’ve redesigned the prompt list UI to make managing prompts easier and more efficient. Here’s what’s new: .- 🔍 Search for prompts seamlessly. - 📝 View additional details, like version and model, without needing to click. - 🎨 More
0
0
6
🚀 We teamed up with @posthog so you can bring LLM metrics from @keywordsai to your PostHog dashboard easily. 🌟 We’re now inviting a few AI startups to be the first to try out this powerful integration before the official launch. We’ll help you set up our dashboard template in
1
3
6
NEW: Upload CSV in Testsets and add production logs to a testset. We’ve improved Testsets to: . - Easily upload your testset for prompt experiments. - Use production logs to directly test your prompt with real-world test cases.
0
2
6
NEW: Our 3rd integration this week - Azure DeepSeek. We're excited to announce that DeepSeek models are now available through our Azure integration. Give it a try today!
0
2
6
The highlight of our Friday?. Heard that users spent their evenings exploring @keywordsai, diving into model comparisons in our model playground. We love seeing Keywords AI becoming a Friday night go-to for developers 😁
0
2
6
UPGRADE: Set spending limits for LLMs and customers. We revamped the Limits page so you can easily control your LLM spending and ensure your users won't exceed the budgets you set for them.
0
1
6
IMPROVEMENT: Revamped Testsets table . We’ve added Tab and Enter key support to the Testsets table for smoother navigation and improved the overall UX. Now, you can edit your test cases just like in Google Sheets! . Try it out today! 📝
0
2
5
We're #1 on @ProductHunt now!!. Thanks to everyone who supported us!.Let's make it stay at the TOP! Every vote counts.
0
1
4
🎄 NEW: Messages auto-saving in the playground. Your messages now stick around after refreshing. One-click to save :)
0
1
5
🎉 Your dashboard just got a little better!. - You can now compare data with the previous period (i.e. this week vs last week).- Added 24h time filtering (thanks @sulaimanghori for sending the feature request)
0
1
5
IMPROVEMENT: Better prompt experiment experience in Labs. We've made Labs more intuitive, allowing you to seamlessly compare outputs from different models and prompts side-by-side. Doc link in the comments 👇
1
2
5
Excited to announce @deepseek_ai integration on @keywordsai!. You can now call DeepSeek models from providers like DeepSeek, Fireworks, Together AI, and OpenRouter. Switch between models seamlessly with just 1 line of code change using Keywords AI’s LLM proxy 🥳
2
2
5
NEW: Group traces across sessions!. You can now group traces by assigning a Trace group ID to each trace. This makes monitoring your AI workflows as a complete chain easy, even when they run in different sessions.
0
0
5
Fine-tuning now available for GPT-4o!. @OpenAI just announced that developers can now fine-tune GPT-4o to get better performance at lower costs for their specific use cases. Even better, OpenAI is offering a generous 1M training tokens per day, completely free for every
0
1
4
NEW: Prompt filters. We wanted to make it easier for developers and product managers to organize and find prompts. You can now filter by prompt title, commit time, update time, and favorites. More filters are coming soon!
0
3
4
IMPROVEMENT: In-Prompt LLM Run. We have redesigned the in-prompt LLM run feature. Now, you can test and validate your prompts directly within the Prompt Editor. Simply configure your prompt and click the Run button to immediately see its performance. This improvement eliminates
0
2
5
At Keywords AI, we're keeping our Thanksgiving team bonding simple and fun -. with a classic game of Connect 4! 🎮
0
2
5
👨💻 Your LLM playground updates:. - Added support for multiple images in playground messages. - You can now define functions in the playground. - Added support for file upload in playground messages. Check it out here:
2
0
4
Due to the lack of LLM observability, spotting and improving poor LLM output can be challenging 🙂 . One of the most popular features on @keywordsai is opening LLM logs in our model playground with a single click. This helps our users easily improve bad LLM output by
0
1
5
🦙Meta just launched Llama 3.2 models!. @keywordsai added support in just 30 minutes!. Check out here:
1
2
5
🎉🎉We built a new models page that has much better browsing functionalities. We now support ~200 models across different providers, Check them out and let us know your what favorite model is!. @AnthropicAI 's claude-3 family is so popular lol.Check out this new page here:
2
1
5
NEW: We just added two new models: .✨ GPT-4o Search Preview .✨ GPT-4o Mini Search Preview . Say goodbye to the hassle of integrating extra tools for real-time data sync or search functionality. These models make it seamless. You can explore both in our Playground, Prompts,
0
2
5
Actions you can take with an LLM Log on @keywordsai . ✅ Share the log with your team for collaboration.✅ Open the log in the LLM playground and refine the prompt.✅ Add the log to your golden dataset for further analysis.✅ Run evaluations on the log to assess performance
0
2
4
🤯 New @OpenAI model gpt-4o-mini is integrated into @keywordsai ! . - GPT-4o mini is more than 60% cheaper than GPT-3.5 Turbo, priced at $0.15 per 1M input tokens and $0.60 per 1M output tokens.- GPT-4o mini outperforms GPT-3.5 Turbo in textual intelligence and multimodal
5
2
5
NEW: Human annotations. You can now review LLM logs and provide feedback or add notes for your team. Human review is essential for evaluating LLM outputs. This new feature makes it easier to collaborate, improve, and ensure better outcomes.
0
0
5
Keywords AI has become developers’ first choice for API key management. With our system, you can:. - Store your API keys safely.- Collaborate with your team.- Customize usage by setting a load balancing weight for each key. Try it now and keep your workflow secure and
0
2
5
NEW: Variable support in Playground. You can now bring your prompt with dynamic variables to the model playground. This makes it easier to test prompts and save new versions directly from the playground.
0
2
5
NEW: Try o3-mini on Keywords AI. We’ve integrated o3-mini. You can test it in our Model Playground or access it via our LLM proxy. Key features of o3-mini: .- Optimized for STEM reasoning .- 200k context window .- Affordable pricing: $1.1/1M input tokens, $4.4/1M output
0
2
5
NEW: Added Prompt name column to the Logs table. This helps you track which prompts are running in production. Now you can monitor prompt performance and find specific logs faster.
0
3
5
The Responses API is OpenAI’s most advanced interface for generating model responses, unifying the best of Chat Completions and Assistants API. Today, we're launching the support for the Responses API - making it easy for developers to set up LLM observability and evaluations
1
2
6
🔄 Customize Retries on @keywordsai!. Retries feature: When an LLM call fails, our system detects the error and retries the request to prevent failovers. Customize your Retries settings!. - Set number of retries.- Define initial retry time in seconds
0
2
5
🔔 New Alert & Fallback page! . You can now specify when to receive a warning email when an LLM outage is detected.
0
0
3
🔥Evaluations are now available for production logs on Keywords AI!. You can turn on evaluation metrics in API > Organization > Evaluations, and view them run on logs in Requests > Sidebar > Eval. More eval features coming this week:.custom metrics.random sampling.topic/intent
1
0
4
⚡️⚡️New models integrated.- Claude 3 Haiku.- Groq llama2 70b.- Groq mixtral 8x7b.- Groq gemma 7b. 💪Integrate these models into your AI applications with 2 lines of change in your code.
1
1
4
NEW: Prompt labels in Prompts. You can create various prompt labels in Prompts which makes managing prompts easier.
0
0
4
🍓 🍓o1-preview and o1-mini are available on Keywords AI (YC W24)!. O1 family is designed to spend more time thinking before they respond. They can reason through complex tasks and solve harder problems than previous models in science, coding, and math. Check it out here:
0
2
4
🔥Llama 3.1 family is available on @keywordsai!. Check it out in our model playground: Powered by @FireworksAI_HQ
0
1
4
NEW: Custom slug for Prompt ID. You can now add a custom slug to Prompt ID to improve code readability.
0
2
4
NEW: You can now add ideal outputs to testsets using the ideal_output column. This allows you to compare ideal outputs with generated outputs in prompt experiments, making it easier to check if the results meet your requirements.
0
0
4
let your coworkers know which prompt is your favorite❤️. You can now star prompts that are important to you! . It really improves visibility when you collaborate with others on prompt management.
0
2
4
Introducing the new pricing plans for Keywords AI. Yes. We revamped our plans to provide more flexibility and value for our users. We updated our Team plan to offer a lower entry cost. For $49 per seat per month, you can debug and ship reliable AI features so much faster with
0
2
4
Grok 2 vs. GPT-4o: Which is Better?. We tested both LLMs by asking them to create a simple website. Grok 2 provided a more detailed framework with additional functionalities compared to GPT-4o. Try them out yourself in the model playground!
1
2
4
🥳Cool feature for conversational AI!. you can now group chatlogs by thread ID to have better observability
0
2
4
NEW: Duplicate a prompt. You can now easily duplicate a prompt in Prompts. This lets you reuse an existing structure to create a new prompt without starting from scratch.
0
2
4
Fun fact: This entire post was written using DeepSeek V3! 🤖.
0
0
4
NEW: Jinja support in Prompts. We’ve added Jinja support to the prompt editor - you can now define conditionals, write comments, and apply filters directly in your prompts, allowing for more dynamic and customizable content. Simply write plain text in the editor, and the Jinja
1
2
4
It's fantastic to see more new models being released!.
Introducing Meta Llama 3: the most capable openly available LLM to date. Today we’re releasing 8B & 70B models that deliver on new capabilities such as improved reasoning and set a new state-of-the-art for models of their sizes. Today's release includes the first two Llama 3
0
0
4