Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Launch HN: Just words (YC W24) – Optimize your product's copy for user growth

news.ycombinator.com

Post reply on HN
Print view View original

51–60 of 80 posts

Re: Launch HN: Just words (YC W24) – Optimize your product's copy for user growth

#51

Post by nehagetschoice »

Areibman wrote: post #46

Very slick, straightforward demo. Question on my mind: The AI A/B testing space seems to be getting crowded. In what way do you see Just Words standing out?

So far, we have seen 1) Companies like stat sig that are stand alone AB test platforms 2) Customer engagement platforms like customer.io that allow you to do A/B testing on their campaigns 3) There a few website only platforms like coframe and mutiny that do not integrate with internal products like notifications and logged in pages

The things are make us stand out -

1) There is no tool afaik that looks at inferential learning based on past experiment results on copies. The experiment analysis and the continuous feedback loop is missing. 2) To make (1) happen, the first step is copy versioning and management. Without a tool that can abstract strings in a CRM, and monitor learnings over time, it's hard to make (1) possible. 3) We integrate with tier 0 services like notifications with a low latency solution that makes copy iterations possible for in-product features, outside of logged out surfaces like web pages.

Would be interested to know if you have seen software that solves for (1), (2) and (3)?

Re: Launch HN: Just words (YC W24) – Optimize your product's copy for user growth

#52

Post by nehagetschoice »

earthling998 wrote: post #50

How do you confidently report a “best version” on lower traffic sites where each variant may only see a couple dozen successes per day?

We either run experiments with higher traffic diverted to treatment (eg: 50% instead of 1%), or run it for a longer time. This is the reason why we need companies to have at least 2K monnthly users to run pilots with us.

Re: Launch HN: Just words (YC W24) – Optimize your product's copy for user growth

#53

Post by bawejakunal »

nprateem wrote: post #44

I'd like to learn more about ranking techniques, the bayesian approach mentioned in the intro as it pertains to this and other best practices. Can anyone recommend any resources please?

Yes, multi armed bandits have been tried a number of times, specifically for copy and UX optimization in industry. A good resource to begin with will be this blog https://towardsdatascience.com/multi-armed-bandits-part-1-b8...

A few examples from industry include: 1. Netflix artwork optimization https://netflixtechblog.com/artwork-personalization-c589f074... 2. Duolingo notifications optimization https://research.duolingo.com/papers/yancey.kdd20.pdf

Re: Launch HN: Just words (YC W24) – Optimize your product's copy for user growth

#55

Post by earthling998 »

nehagetschoice wrote: post #52
earthling998 wrote: post #50

How do you confidently report a “best version” on lower traffic sites where each variant may only see a couple dozen successes per day?

We either run experiments with higher traffic diverted to treatment (eg: 50% instead of 1%), or run it for a longer time. This is the reason why we need companies to have at least 2K monnthly users to run pilots with us.

Thanks! Cool idea on that front and in general; fun to see people taking copy seriously!

I am still a little fuzzy on how this might play out on a real site, since lots of monthly users doesn’t always mean lots of conversions (especially for one-time, high dollar transactions).

Let’s say I have a page that gets 100k unique visitors per month. I show them 5 different variants of a “nudge” widget. Some do better than others, but they all hover at While there may be a story to tell as far as winners and losers (e.g. mobile users converted to variant A at 2x the rate of desktop users), how do you confidently report a “winner” from what amounts to a few thousand conversions in a month?

Re: Launch HN: Just words (YC W24) – Optimize your product's copy for user growth

#56

Post by bawejakunal »

nprateem wrote: post #43
bawejakunal wrote: post #14

Earlier quoted context omitted.

Yes this can be tailored for individual users (or groups). To go more in-depth, we'll be using contextual bandits + more novel ways to optimize the best performing copy for user segments, that works on top of the anonymized knowledge about the user.

Can you explain more about this please. What are contextual bandits?

+1 a resource I linked in one more comment, sharing here as well.

This has been tried and tested a few times in industry, notably at Netflix and Duolingo for artwork and notifications

https://towardsdatascience.com/multi-armed-bandits-part-1-b8...

Re: Launch HN: Just words (YC W24) – Optimize your product's copy for user growth

#57

Post by helloworld »

Very cool! But maybe change copies to copy.

In my experience, copy is used the same way as code. And just as you probably wouldn't say, "I changed the codes for the three apps," most people wouldn't say, "I rewrote the copies for the three websites."

Re: Launch HN: Just words (YC W24) – Optimize your product's copy for user growth

#58

Post by Areibman »

nehagetschoice wrote: post #51
Areibman wrote: post #46

Very slick, straightforward demo. Question on my mind: The AI A/B testing space seems to be getting crowded. In what way do you see Just Words standing out?

So far, we have seen 1) Companies like stat sig that are stand alone AB test platforms 2) Customer engagement platforms like customer.io that allow you to do A/B testing on their campaigns 3) There a few website only platforms like coframe and mutiny that do not integrate with internal products like notifications and logged in pages The things are make us stand out - 1) There is no tool afaik that looks at inferentia…

Makes sense. Coframe (https://coframe.ai) comes top of mind

Re: Launch HN: Just words (YC W24) – Optimize your product's copy for user growth

#59

Post by antonvs »

helloworld wrote: post #57

Very cool! But maybe change copies to copy . In my experience, copy is used the same way as code. And just as you probably wouldn't say, "I changed the codes for the three apps," most people wouldn't say, "I rewrote the copies for the three websites."

> you probably wouldn't say, "I changed the codes for the three apps,"

Unless you’re - shudder - a data scientist

Re: Launch HN: Just words (YC W24) – Optimize your product's copy for user growth

#60

Post by vintagedave »

sgslo wrote: post #2

Neat product. Small thing on the demo: perhaps using Stripe in the demo isn't the most effective choice. Stripe surely has already A/B tested the heck out of their landing page; "Financial Infrastructure for the Internet" is (IMO) an incredibly strong tag line for the hero text. The alternatives generated by your tool pale in comparison. Perhaps it would be more effective to put a lower-quality landing page in as you…

> "Financial Infrastructure for the Internet" is (IMO) an incredibly strong tag line for the hero text. The alternatives generated by your tool pale in comparison.

Yes! Look at the alternatives the tool generated:

* Maximize Revenue with Secure Transaction Processing

* Elevate Digital Commerce with Trusted Payments

* Unlock Opportunities with Secure Transactions

* Simplify Your Online Payments

* Accelerate Your Online Business Growth

These are just... bland. Generic. I've seen a thousand webpages with taglines just like these.

AI can be quite bad at generating unique responses. Its answers are middling. This can be great for, eg, coding where you want a copilot to generate known algorithms and approaches, but terrible for marketing where you want a slice of brilliance and genius.

Would turning up the AI temperature and other settings help, maybe?

Post reply on HN