Comet
@Cometml
Followers
15K
Following
2K
Media
784
Statuses
3K
Comet provides an end-to-end model evaluation platform for AI developers, with best in class LLM evaluations, experiment tracking, and production monitoring
Joined October 2017
How well do you understand perplexity, really? Could you implement it from scratch? . Well, if you need a refresher, @anmorgan2414 has written an 🔥incredible🔥 deep-dive on perplexity, covering the theory, history, and contemporary uses of the classic metric. Check it out 👇.
7
11
411
Here is one of the best books to improve your data skills. But don't stop there! . Combine your feature engineering skills with automatic experiment tracking, and you'll 10x your ability to build machine learning models that work. Start for free today:
1
49
341
⭐Opik has officially hit 2,000 GitHub Stars! ⭐. Since releasing our open source LLM eval framework in September, we’ve been blown away by your support. Thank you all so much 😍 To celebrate, we want to shoutout some of people who’ve contributed to Opik 🧵
9
15
249
Here is one of the best books to improve your data skills. But don't stop there! . Combine your feature engineering skills with automatic experiment tracking, and you'll 10x your ability to build machine learning models that work. Start for free today:
1
41
224
Distance metrics are critical for machine learning. Many algorithms use distance metrics to calculate the similarity between two data points. A few examples: . 1. Euclidean distance .2. Manhattan distance .3. Minkowski distance .4. Hamming distance.
4
32
188
A critical skill for every Python developer: Learn how to deal with data. Here is a step-by-step example for you: . 1. Grab data from multiple locations .2. Automatically load it into a spreadsheet .3. And do it all using Python . Here is the link:.
7
21
114
Building Machine Learning Systems is a team effort. If you want to build a team, here are the roles you need:. • AI Architect.• Data Scientist.• Data Engineer.• Machine Learning Engineer. Here is the responsibility of each of these roles:.
3
14
102
Experiment Tracking is one of the simplest ways to improve your machine learning setup:. • Version your datasets.• Debug and reproduce your models.• Visualize performance across runs.• Collaborate with teammates. And here is the best part:. 1 of 3
2
13
99
We talked to dozens of Computer Vision practitioners, and their feedback was consistent:. Visualizing data is painful during exploratory data analysis. To solve this problem, we built and open-sourced Kangas. And the community loves it!
1
14
80
How do you improve your #DeepLearning setup in two minutes?. Integrate your code with "Automatic Logging." A few lines of code, and you'll never go back. Here is how it works:. 1 of 5.
1
11
78
We’ve worked with 100s of researchers building LLM-as-a-Judge evaluations for hallucination detection, content moderation, and more. The one thing almost all of them have in common?. They use structured generation. 🧵 (1/7).
2
1
81
It’s here! Our first open-source project is now live! Introducing Kangas, a smart data exploration and analysis tool for machine learning. Dive into your data and analyze it like never before.
4
23
74
As a team of programmers aiming to help #machinelearning engineers, this book in progress on ML from @jeremyphoward (@fastdotai) + @the_antlr_guy (USF) was a great refresher for us! . Both Jeremy and Terence are looking for comments + suggestions! .
0
26
77
If you're looking for a summary of mathematics for #MachineLearning, this paper is a goldmine!. It's 100% free, courtesy of the University of Berkeley, and covers the following topics:. • Linear Algebra.• Calculus and Optimization.• Probabilities.
1
14
71
There are 3 fundamental roles in a team dedicated to building a Machine Learning system. But we usually only talk about Machine Learning Engineers. Here are the roles you are missing and their responsibility:. 1/4.
1
6
67
Sebastian Ruder (@seb_ruder), a research scientist @_aylien, reviews the history of NLP — from sequence-to-sequence learning to adversarial learning. See the full history here: .
0
10
57
From this week's Comet Newsletter: Our friends over at @huggingface just released a new, completely free video course on transformers and other key #NLP topics.
0
7
47
Most data scientists focus on training models. That's great, but it's only 5% of the work! Building end-to-end systems is hard. Here are 30 requirements for an MLOps environment. Link:
0
6
41
Today, we’re thrilled to announce a $50M series B funding round, led by @OpenViewVenture with participation from @scalevp, @TrilogyEquity, and @TwoSigmaVC! We also welcome Ofer Bengal & Dr. Oren Etzioni as strategic advisors. Read more in @TechCrunch:
5
11
39
Multimodal datasets are becoming more common in deep learning. Inspired by a seminar at LMU Munich, a team of students and researchers set out to collate existing knowledge and burgeoning approaches. Read more:
1
11
40
Recommender systems are one of the most impactful ML models in production. They influence our everyday decisions, from what news to read to what shoes to buy. Unfortunately, evaluating the performance of these models isn't easy. This is where RecList comes in. 1/4.
1
6
40
Introducing a brand new suite of Prompt Engineering tools from Comet! With advanced capabilities like Prompt Playground, Prompt History, Prompt Usage Tracker & seamless integration with OpenAI & LangChain, streamlining LLM workflow was never this easy.
1
8
34
Data scientists struggle to select the best Machine Learning tool for their project. With so many options, deciding becomes overwhelming, and the wrong choice can kill your project. Here are 9 of the most popular Machine Learning tools in the industry right now:.
2
6
36
What Is the Machine Learning Lifecycle?. There’s no one formula for developing models, but most projects follow standard steps. Here are the top 10 tools to optimize the ML lifecycle.
4
7
34
With the new #StyleGAN3 released, we thought it'd be helpful to have a way to visualize training run outputs—images & videos, but also metrics, hyperparameters, etc. @DN06_90 & Michael Cullan put this #Colab notebook together that does just that . 👉
0
10
35
RSVP for our meetup tomorrow with @GuggerSylvain, who will cover an infinitely customizable training loop using @fastdotai — food, beverages, and networking w/ data scientist included!. Co-hosted with NYC @PyTorch 🤓.
1
7
35
Comet + @huggingface = Debugging transformer models has never been easier. Check out the details of our new auto-logging integration:
0
6
33
If you haven't had the chance to start building #NLP Models, we suggest getting started using this awesome Sentiment Analysis tutorial. You'll learn how to create BERT models using the @huggingface package.
0
8
30
This #BlackHistoryMonth we'll be highlighting some great Black ML & AI contributors. 👇. Deb Raji is a @Mozilla Fellow working on AI accountability & model auditing. Raji has worked on many crucial ML papers like Model Cards. @rajiinio. Read her work:
0
5
25
We're big fans of @huggingface.🤗 Check out this blog on how to auto-log model metrics and parameters to Comet from the Hugging Face transformers library. #AI #ML.
0
4
28
🔥We’re very excited to announce that Opik is now fully integrated with @llama_index!. LlamaIndex offers tools to ingest and structure your data, as well as an advanced retrieval/query interface for your RAG applications. Check out the docs for more:
0
5
27
The recording of @SanhEstPasMoi's talk on productionizing NLP models is up! . Check out the awesome work that the @huggingface team is doing with #NLP and #NLG models within conversational AI 🗣. Watch the recording here:.
0
10
26
With transfer learning, should you use pre-trained models for feature extraction or fine-tuning?. @bhutanisanyam1 reviews a great paper from @seb_ruder and @mattthemathman et al to compare how these approaches perform on NLP tasks. See the full overview –.
0
7
23
The 4 stages of a machine learning project lifecycle: . 1. Project scoping .2. Data definition and preparation .3. Model training and error analysis .4. Deployment, monitoring, and maintenance . We built Comet to streamline the process. Get started today:
1
6
22
Comet is Growing! . Welcome aboard Comet's new Data Science Advocate/Evangelist Ayodele @DataSciBae! We're excited to have you here creating helpful content for our users. Are you open to roles in #MLOps & helping Data Scientists track their experiments?
0
0
16
Calling all Machine Learning enthusiasts! Mark your calendars for June 22nd, 8 AM PT for a free webinar on LLMOps + Model CI/CD with Comet . Register today for free and unlock the next level in your ML journey: #webinar #LLMOps #PromptEngineering
1
2
20
🧵 (1/8) ICYMI: Our public #AIArtwork gallery, powered by @kvfrans' #CLIPDraw, @Gradio, and Comet is live and ready for submissions!. 👇TL;DR .Submit your prompt: .View the project: Full blog post:
1
7
21
🔥 Big News! 🔥. Comet is now natively available in AWS SageMaker partner AI apps as a fully managed solution!. ✅ Automated LLM evaluation with Opik (incl. Bedrock support).✅ Seamless experiment tracking in SageMaker. Fully managed by AWS 🚀
3
4
21
📢Heads up NYC #AI & #ML developers! . Join us & @LightningAI on Oct 9th for a technical meetup 🌆 Hear from @JayeetaPJ on the challenges of evaluating LLMs and enjoy rooftop networking and refreshments. 👉RSVP:
2
0
19
We’ve recently welcomed several new faces to the @cometml team and they’ve already hit the ground running! #LifeAtComet
2
2
18
In just 2 days, Opik reached 1,000 ⭐ on GitHub! Huge thanks to everyone who’s contributed and shared — it's amazing to see hundreds of users already excited about what we’re building together!
1
0
20
is excited to announce our $2.3 million seed round! Huge thanks to @TrilogyEquity @TwoSigmaVC @founderscoop @FathomCapital and Techstars Ventures. We're looking forward to continuing to build the team and the product in 2018 and beyond!.
2
6
18
Today we announced a $4.5MM investment to double-down on Comet Enterprise and Meta Learning capabilities. Read the announcement: #ai #ml #metalearning
0
6
17
Every bank deals with the same problem:. Credit card fraud. Here is a tutorial showing you how to use autoencoders to detect fraud. (Includes the full source code.).
2
1
15
Voted #3 up-and-coming NYC enterprise startup!.
Up-and-coming enterprise startups in NYC by @DannyCrichton.
1
6
17
🛠️ Prompt Engineering is the most cost-effective strategy to leverage LLMs for your applications. But current prompt engineering workflows are incredibly tedious and cumbersome. 🔥 CometLLM is a new tool that documents all your experimentation with LLMs. 🧵👇.
2
5
17
Excited to partner with @LangChainAI and launch our new LLMops product!.
LangChain 🤝 @Cometml . Excited to announce an integration with CometML: they recently added a prompt playground, prompt history (can visualize chains), and prompt usage tracker. All needed to help take your LangChain apps from prototype to production.
0
1
18
To deal with the subjective rules of grammar, Google uses machine translation models to suggest corrections. These models work by treating text with incorrect grammar as the “source” language and correct grammar as the “target.”. Read on here — .
0
4
15
Kangas is still in its early days, but super excited to see this growth. We've gotten code contributions and stars from MLEs at companies such as Dell EMC, IBM, Oracle, LEGO, Wix, Google, Autodesk, Docusign, and more!. Try it out:
0
1
11
This project from Victor Sanh (@SanhEstPasMoi) and the @huggingface face blew us away🔥. The Hierarchical Multi-Task Learning (HTML) model can learn embeddings that can be shared between different semantic tasks. See the demo here —.
0
5
15
Proud to be part of #CodeCarbon, an amazing initiative with @Mila_Quebec, @BCG, and @haverfordedu, that will help organizations measure the environmental impact of #AI. Learn more about this incredible work:
0
3
12
How do you improve your #DeepLearning setup in two minutes?. Integrate your code with "Automatic Logging." A few lines of code, and you'll never go back. Here is how it works:. 1 of 5.
2
3
13
Today in Heartbeat: Omale Happiness uses Python to develop a supervised learning text classification model that can predict the sentiment category of a sentence, and builds an interactive web application that displays the result to the users.
3
4
11
🚀 We're excited to announce another excellent integration with Comet --@raydistributed!. Leverage Comet's experiment tracking and visualization tools with Ray's incredible library for scaling compute intensive Machine Learning workloads.
0
3
14
Today in Heartbeat: With @LangChainAI's LCEL, users can adopt a declarative approach to chain composition, facilitating operations like streaming, batch processing, and asynchronous tasks. Learn more in this guide from @DataScienceHarp:.
0
6
13
🚨 Upcoming MeetUp!. We're restarting our industry Q&A series next Monday, July 26th, at 11am ET. We’ll be chatting about collaborative #ML tools with @jakubjurovych of @deepnotehq & @abidlabs of @Gradio. 👉 Register for free, & bring your questions!
0
5
14
🚀 We’re hiring a Technical Account Manager to join our rockstar customer success team!. Know anyone who might be interested? Share this with them. #Hiring #Careers #MachineLearning
0
1
13
Love these @feedly NLP virtual breakfasts! . The latest edition with @SanhEstPasMoi of @huggingface on Hierarchical Multi-task Approach for Learning Embeddings from Semantic Tasks is excellent 🚀.
0
2
13
Today in Heartbeat: With the rise of #LLMs like #ChatGPT and #Bard, a new, lucrative field has emerged. #PromptEngineering is the art of guiding language models with clear, detailed, and well-structured prompts. @BasakBuluz explains more here:.
0
3
14
Object detection is the most popular application of computer vision. But how can you systematically find the best object detection model for a particular task?.Check the thread for the full-code tutorial using TorchVision.
1
1
5
Very cool app (and end-to-end machine learning pipeline) idea for automatically tagging Github issues! . Hooks together the Github API, BigQuery, Flask + more and tested with the @kubeflow repo 🏷. See the full run down from @HamelHusain and team here:
1
2
10
LLM-as-a-judge evaluators may seem simple on the surface, but implementing them in real-world applications is challenging. Evaluating multiple metrics often means separate pipelines that must be combined. G-Eval simplifies this process 🧵 (1/4).
4
3
13
Mirror lets teens:.📝Express themselves through writing, voice, images, or video .☀️ Track their moods .💡Get guided prompts to process feelings. Learn more🔗
3
1
12
Congratulations to @FrancescoLocat8 and team for their Best Paper Award at ICML 2019. Check out the blog post explaining their paper "Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations".
0
3
13
🚀 Tomorrow — we're diving into building scalable GenAI platforms with our friends from the AWS SageMaker team. It's not too late to register! 👉
1
1
9
Can AutoML match hand-crafted machine learning models?. Adrian Rosebrock (@PyImageSearch ) puts Auto-Keras to the test in this great classification tutorial . See the full post here —.
0
2
13
🧵When building mental health apps powered by GenAI, evaluation is essential. We’re proud to see Opik supporting Mirror, a free, research-backed app from the @childmindinst that helps teens manage stress & anxiety.
5
3
12
⚡️Lightning Studios is a cloud-based AI development platform that requires no environment setup, has unlimited storage, & supports distributed training on both CPUs & GPUs. Get started using @Cometml with @LightningAI in this quickstart Lightning Studio:.
0
2
13
👋 Come see us in Seattle! We’ll be at #CVPR2024, along with an impressive lineup of workshops, papers, & more on #ComputerVision & #PatternRecognition. Stop by our booth to talk all things #MLOps! @cvpr @ComputerSociety #IEEECS
0
3
13
Today in Heartbeat: Jeremiah Oluseye uses Logistic Regression to detect credit card fraud with an imbalanced dataset.
0
4
9
RSVP for our next Meetup to hear speakers from @XaxisTweets, @huggingface , and @runwayml discuss productionizing machine learning models! . See the full details for March 7th here:.
1
4
12
🔥In a continued effort to close the gap between Data and Experiment Management, we are very excited to announce a new partnership with #Snowflake! ❄️. Comet now offers support for tracking and versioning SQL queries, as well as logging sample DataFrames!.
0
4
11
Welcome to Friday! . To celebrate, we like looking at machine learning memes — thanks to @ericjang11 for curating 🤓 (and the rest of his blog is pretty spectacular) .
0
1
11
And we’re just getting started – new features are on the way! Stay tuned!. #OpenSource #LLMevaluations.
0
0
11
Unlock the true potential of your ML workflows like Shopify did with Comet's Model Registry and Experiment tracking tool. Learn how Shopify served machine learning models across multiple teams with a robust solution for real-time predictions.
0
0
12
Want to see a neat trick on how to log an interactive plot in Comet? . Let's say you have a ploty figure called py_fig. Use the following command in your code!. experiment.log_html(py_fig.to_html()). Then go to the HTML section to see your interactive plot 💪
0
2
9
We are incredibly excited to announce our new short course with @DeepLearningAI: . 🚀 Prompt Engineering for Vision Models 🚀.
In Prompt Engineering for Vision Models, taught by @anmorgan2414 @JacquesVerre and @KaiserFrose of @Cometml , you’ll learn how to prompt and fine-tune vision models for personalized image generation, image editing, object detection and segmentation. The prompts you'll use for
1
1
12
💡Check out Comet’s latest integration with DeepSpeed, a deep learning optimization library!. 🤝With the @MSFTDeepSpeed + @Cometml integration automatically start logging training metrics generated by DeepSpeed. Try the quick-start Colab to get started:.
0
2
12
Introducing a brand new suite of Prompt Engineering tools from Comet! With advanced capabilities like Prompt Playground, Prompt History, Prompt Usage Tracker & seamless integration with OpenAI & LangChain, streamlining LLM workflow was never this easy. :
0
1
10
Today in Heartbeat: a deep dive into the challenges of deep learning in software development. Learn more in Daniel Onugha's new article:
0
2
10
🧬Hey San Francisco #machinelearning community! Connect with fellow ML practitioners and expand your knowledge on the intersection between #AI and #genealogy next Thursday, 7/18, at our new technical happy hour featuring @geletavc. Sign up here:
0
1
6
Our second big announcement this week! We are excited to announce Comet has been recognized as a 2022 @Gartner_inc Cool Vendor.
0
4
7
Interviewing teams with a mission to reproduce the latest in machine learning research. Hear from @robert_aduviri and @alfo_512 who share an interest in generative models and machine learning competitions. #machinelearning #datascience #reproduce.
0
4
11
🎙️ #ConvergenceConference speaker announcement!. @ShivikaKBisen, Lead Data Scientist at @PAXAFE1, who will present "Testing ML models for production" at Convergence on March 2. Register now to catch this and all the other great talks!
3
4
11