When I tell people I am about to publish an O'Reilly book on prompt engineering and AI, their first question is always "won't it be out of date the moment it's printed?". It's something
@jamesaphoenix12
and I had to think about, a lot...
So we took the things that were working
What bizarre times we live in
Musk and the SpaceX team launch Starship’s first orbital flight, and all these morons are tweeting trying to dunk on them because it wasn’t perfect on the first try
They’re doing something that’s changing the world. What the hell are you doing?
@PayPerMitch
Yup it's a common problem in recommending systems. Because almost everyone has watched Harry potter, it almost always ranks as the highest recommendation unless you have some sort of strategy for dealing with it.
@besttrousers
Also living paycheck to paycheck is a terrible measure of fairness. I have friends in finance with coke habits who live paycheck to paycheck on far more than I earn.
3/ Click-based attribution is a joke: they found *no correlation* (R2 = 0.0001) between engagement metrics and sales (this is something traditional marketers have been harking on about for years, good to see it verified by FB).
@Altimor
We implemented a simple excel test (literally just vlookup and pivot table, open book) when I ran a marketing agency and about half of all applicants simply gave up. They didn't even try to cheat (the whole test was published on our blog and they'd find it if they googled)
@fchollet
From what in understand it's not that simple. The countries that have these things have low birthrates as well. The only major correlations with high birth rates are being poor and being religious.
Everyone is sleeping on Google Data Studio - world class analysis and reporting functionality... for free. I literally couldn't do half of the things I did today with GDS for any amount of money a couple years ago.
@cblatts
The assumptive close: whenever there are a few options to decide between, rather than waiting for input and making your manager a bottleneck, instead state which option you'll move forward with by X deadline unless you get any objections.
1960s: "COBOL will let non-programmers make the software!"
1980s: "4GLs will let non-programmers make the software!"
2000s: "UML will let non-programmers make the software!"
2020s: "AI will let non-programmers make the software!"
@ArchRose90
Does the UK get money off for banning slavery before any other major European power and deploying the Royal Navy to West Africa for 60 years to enforce the ban?
An agency is a collection of creative people that would be fired or quit if they worked directly for the client, loosely held together by a commercially savvy but talent friendly translation layer we call 'account management'.
If you showed a single Facebook ad impression to everybody in the country today, they would claim attribution on 100% of your sales for the next 24 hours (even though we all know you would have had sales today anyway). Digital isn't as measurable as you think.
#FBAds
#FacebookAds
Happy to announce I'm writing a book on Prompt Engineering for
@OReillyMedia
with
@jamesaphoenix12
The first few chapters are available for early-release here:
No I didn't get to choose the animal on the cover (always wondered if authors did...).
The shoggoth strategy for freelancing / job applications / company building for ambitious people.
Ambitious people are usually generalists, but clients and customers want to hire specialists. How do you solve this?
The shoggoth strategy: hide your messy generalist experience
It'd be the funniest thing ever if the UK shooting itself in the foot with brexit ends up establishing it as the central AI hub in Europe thanks to being able to avoid over-regulation.
OpenAI and Anthropic also have London offices. And a big chunk of Google DeepMind is there.
On the AI Safety side, there's also UK AISI, the Alignment team at Google DeepMind, Apollo Research and LISA.
@dougludlow
You don't want someone that worked at FAANG whose impact was a rounding error, you want someone who gave it everything and still didn't get there. They still need that win and all you have to do is give them better odds this time.
After Apple killed user-level tracking in iOS14, Facebook went all in on Marketing Mix Modeling with Robyn, an open source library. Now Google is getting into the game with an *unofficial* library called LightweightMMM, and it couldn’t be more different.
Lend me your case studies: anywhere someone has talked publicly about using Marketing Mix Modeling, Multi-Touch Attribution, or Conversion Lift Studies. I'm building a database.
5/ ...so they recommend including it with a 'score card' approach. You score creatives against a set of quantitively measurable best practices. Then include that as a proxy in your model. i.e. share_of_impressions_following_best_practice = [0.23, 0.24, 0.3, etc]
In traditional marketing we're taught to work out our strategy first, then choose our tactics based on that strategy. In practice I've found that the most successful startups change their strategy based on the economics of the available tactics.
Working with a Marketing Mix Modeling (MMM) client, and a Conversion Lift Study (CLS) client, now all I need is a Multi-Touch Attribution (MTA) client and I will have collected all the incrementality stones.
You could literally build an "import from Meta" feature like Bing had from Google and I guarantee you hundreds of agencies will jump at the opportunity to add "turned on Twitter ads" to pad out their quarterly business review.
I bought myself a Nintendo Switch after Christmas and now my wife and I play Mario Kart instead of watching Netflix. This is the life 8 year old me always wanted.
@levelsio
Basically every scientific paper that says "AI can't do X" only tested it on GPT-3.5 or below and when you try it on GPT-4 it works. Even professional researchers are too cheap to pay for plus.
4/ The biggest mistake analysts make with MMM is not incorporating creative: it has the largest impact on performance but is almost never included (for practical reasons, you just can't have more than 1 variable per 10 observations, so there's no room to spare)...
1/ Facebook is bullish on MMM to solve the iOS14 problem. They are investing a lot in open source code (Robyn) and have created a special MMM feed for measurement partners.
Writing publicly about your work.
What you think will happen:
- they'll steal all my secrets and laugh my mistakes
What I thought would happen:
- people will point out where I'm wrong so I learn and improve
What actually happens:
- "hey can we pay you as a consultant..."
Yesterday I went viral on Twitter live tweeting a Facebook event on the normally dry topic of Marketing Mix Modeling, an attribution technique from the 1960s. Over 80,000 views in 24 hours! Good news – Facebook has a (more technical) talk on right now.
@dougludlow
I did this for our first head of marketing at my agency. First salary he had taken in years. He took a month to push the first campaign but then doubled our business every year for the next three years.
@apocryphalcoder
@jothwip
Yeah I think everyone is getting hung up on semantics of the task as defined, and he just figured out a better definition of the task. Lossless should mean that the original signal is not lost: if the recording of that signal introduced error, removing that error is good.
Tabasco is three simple ingredients: distilled vinegar, red pepper, salt. 100% of it is made on Avery Island, by a family-run company over five generations. Most indie businesses should aim to be Tabasco companies, not the next Uber, Google, or OpenAI.
I'm finally going for it.
- 18th Dec: Last day of consulting 🙌
- 19th Dec: My 35th birthday 🎉🎂🍻
- 20th Dec to 4th Jan: family time + some coding 👨👩👧💻
- 5th Jan: full time on 📈
@levelsio
@Molson_Hart
In that analogy the only other place to buy caffeine is a boba tea shop (China) and a sit down restaurant with rude waiters (Europe), so people keep buying Starbucks.
I love marketing, but couldn't do a job like this where I have to pretend that people care at all about the made up character traits of sugar-coated chocolate balls.
The VP of M&Ms explains that they’re making the M&Ms characters more inclusive in order to “start a movement.” The girl M&Ms will be focused on “empowerment” while the orange M&M will “embrace his true self.”
This is not a parody
Indie hackers just spend 1 day a week on marketing, Fortune 500 just spend 20% of revenue. It's only the startups in the middle that care about marketing attribution.
Whenever I see someone post like this it's an instant unfollow. If you have to rely on gimics it's immediately obvious you have nothing valuable to offer.
This was a really interesting read, great to see these topics being discussed with no holds barred. Personally I recommend:
- MTA for daily campaign optimization
- Surveys for weekly calibration
- MMM for monthly budget allocation
- Lift Testing for proving assumptions
The world that digital marketers have known for 20 years is rapidly disappearing with the death of digital attribution. Is econometrics a good replacement?
In my new
@TheDrum
column today, Les Binet tells me "yes" and Byron Sharp says "no." What say you?
A performance marketer who makes changes daily and sees performance in real-time is just going to have a much more honest view of reality than a brand marketer that does 1-2 campaigns per year decided on by committee and subject to client approval.
Men in tech
if you’ve got 50k in savings and you aren’t willing to give your girlfriend kids THIS YEAR if she wants them...
grow the f**k up, stop wasting her time.
@GergelyOrosz
Wasn't the issue that advertisers were (in his words) blackmailing him? As in withholding their ad spend until he implemented certain standards that other ad platforms aren't held to (again his interpretation). He's basically signalling he can't be pressured.
I broke into marketing mix modeling by reading four textbooks and working for free for three months while writing blog posts and live tweeting webinars. Even for quite complicated topics the bar is lower than you think to be able to make a positive contribution.
imo: if you’re deliberate about “doing the reading”, whatever that means for any domain, within a year it’s possible to be in the top ~5% in the scene. This is because 95% of people don’t do the reading
As you know Marketing Attribution became my full time obsession when I saw the iOS14 bloodbath coming. Excited to get published on the topic by
@reforge
. Awesome to work with
@Mike_Kaminsky
and
@tvladeck
from
@getrecast
on this absolute beast of a post. 💪
90% of Meta accounts only need a simple Champion-Challenger testing structure: put two ads live, keep on the winner (highest spend), keep trying to beat the Champion ad with new Challengers 1-2 times a week.
Advertising has to be remembered to be effective. So it's no coincidence that the adstock curve (the effect of ads over time) and the forgetting curve (the retention of memories over time) look the same.
Man I worked so hard for 1.5 years on making marketing mix modeling courses for Vexpower, and all our signups are coming from a handful of AI courses I made in the past few weeks. 🤷♀️
@emollick
Prompt engineering is about passing the right context to an LLM in the right format and measuring eval metrics to prove it's working. This doesn't add context it just rewrites the existing context, and it doesn't measure success, so the upside of a feature like this is limited.
Just had a really wild call with Google — they said that ~any change~ to a campaign resets learning to ZERO (like a new campaign) for 3 to 7 days. This is big — every serious advertiser makes daily (hourly?) changes and this will be killing performance.