"This is what experimentation is all about. It’s about how you can solve your customers’ problems, not about how fast you can roll out new features."

"Unplanned interactions are incredibly rare. However, despite the rarity of such interactions, simply the fear of interactions often prevents many teams from fully leveraging the potential of running overlapping tests at scale." 👏👏👏.

Join millions who have switched to Grok.

RT @ronnyk: What should be the primary metric for an #abtesting experimentation platform (internal or built by vendor)?.@lukasvermeer and I….

Stop. Look. Listen. The third post in the series about detecting interaction effects in online experimentation is finally out! 👀

RT @AnalyticsHour: Experimentation, done well, can lead to an org consistently making better decisions. But getting a company to the point….

Very significant if true. Could be a false positive. Error rates seem especially high today.

"There is no such thing as the perfect experiment. There is only a continuum between very well executed experiments, and completely winging it off the seat of your pants.".

Teachers are also forever learning. This week I learnt that I have been confusing people for years about what a hypothesis really is. A conversation with @endesignonline and @OptimiseOrDie helped me understand my mistake (tl;dr I oversimplified, a lot).

Speed is not velocity, but this does not mean that speed is not important. The connection between speed and velocity is more complicated than it might at first seem. Mantras such as “fail fast” and “rapid iteration” hint at a deeper relationship.

There is a popular meme which is used as a response to “useless” meetings: “this meeting could have been an email”. I think replacing meetings with emails would be a good start, but replacing it with a wiki page would in many cases be even better.

Twelve hours later. I think she might be addicted.

Cat TV. 📺🐈

Sophie (7): Daddy, what exactly is falling at “nightfall”? Shouldn’t the fall of the night come at the end and “nightrise” at the beginning? 🤔.

“Won’t running all those experiments slow down our product development?” When speaking at conferences, I get this question a lot. And my answer is always the same: yes and no. Let me explain.

For me, if this is true, it raises two questions:. 1. Why is statistics not taught like this more broadly?.2. Why are linear models not more popular in analysis of A/B tests?.

This idea and the clear examples are blowing my mind. "Most of the common statistical models (t-test, correlation, ANOVA; chi-square, etc.) are special cases of linear models or a very close approximation [. ] it all comes down to y=a⋅x+b".

"Experimentation involves a great deal of learning by doing. [Our role] is to encourage teams to simply start to test, to hold their hand the first few attempts, and to provide critical but constructive feedback that helps teams improve their approach." 🥞

Pleasantly surprised I managed to nab my first name @mastodon.nl. Decentralization has its advantages, I guess. Ahead of the flock or migrating to a wasteland? Only time will tell. 🤷‍♂️

I don’t necessarily want to easiest solution either. This is a pet project in my spare time. I hope to learn from implementing the _best possible_ solution. Anyone have suggestions or experiences to share?.

My websites are statically hosted on GitHub Pages. I do not have access to servers (or logs). Hosting elsewhere would be possible but seems too drastic. I also want to be respectful of (the privacy of) visitors. “Just add Google Analytics” is not the solution I am looking for.

I don’t think these are unreasonable questions to ask. Answering them will help me decide whether to invest time improving these websites and creating more content. This would (hopefully) benefit future visitors.