Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Tim Ferris way of testing ideas and how I did it.

docleyblog.com

Post reply on HN
Print view View original

11–20 of 38 posts

Re: The Tim Ferris way of testing ideas and how I did it.

#11

Post by tocomment »

patio11 wrote: post #7
teej wrote: post #2

I've been itching to try this for some time. I would appreciate any war stories HNers could share.

Email addresses don't covert on a 1 to 1 basis to sales. (If. Only.) After the autoresponder runs out (6 days in my case) they've only got only a moderately higher chance of conversion than my average trial user. (~5% vs. 2.8%) I really, really don't like the Put Up Fake AdWords Ad, See Who Gets Suckered minimum viable product. Aside from misleading customers and hurting the AdWords ecosystem (they don't actually get…

Those are really good points. Is there another type of market testing you recommend?

Re: The Tim Ferris way of testing ideas and how I did it.

#12

Post by ruchi »

dawie wrote: post #9
fookyong wrote: post #6

There is an ocean of difference between 14% of visitors committing their email address to something, and that same figure actually pulling out their credit card. A more prudent assumption would be 10% of that 14% will actually pay for your product when it is launched. I learned this first hand. Your mileage may vary of course, but it's incredibly easy to drive traffic and stimulate interest, enough that people will g…

I agree with you. Maybe a good idea would be to ask for a credit card, but not to charge it. I think this borders on being illegal though. I am still pretty happy with 10%, since this was my cut-off for going ahead and building the product.

Per fookyang, the conversion is just 1.4%.

Re: The Tim Ferris way of testing ideas and how I did it.

#13

Post by BobbyH »

dawie wrote: post #9
fookyong wrote: post #6

There is an ocean of difference between 14% of visitors committing their email address to something, and that same figure actually pulling out their credit card. A more prudent assumption would be 10% of that 14% will actually pay for your product when it is launched. I learned this first hand. Your mileage may vary of course, but it's incredibly easy to drive traffic and stimulate interest, enough that people will g…

I agree with you. Maybe a good idea would be to ask for a credit card, but not to charge it. I think this borders on being illegal though. I am still pretty happy with 10%, since this was my cut-off for going ahead and building the product.

In his book, I believe that Tim says that it's not illegal if you don't store a user's credit card information when selling a product that isn't fully-baked. But I agree that it is a bit shady...

Re: The Tim Ferris way of testing ideas and how I did it.

#14

Post by andymism »

dawie wrote: post #9
fookyong wrote: post #6

There is an ocean of difference between 14% of visitors committing their email address to something, and that same figure actually pulling out their credit card. A more prudent assumption would be 10% of that 14% will actually pay for your product when it is launched. I learned this first hand. Your mileage may vary of course, but it's incredibly easy to drive traffic and stimulate interest, enough that people will g…

I agree with you. Maybe a good idea would be to ask for a credit card, but not to charge it. I think this borders on being illegal though. I am still pretty happy with 10%, since this was my cut-off for going ahead and building the product.

Instead of asking for the credit card information (which does sound illegal), I've heard of others who have placed a "Buy Now"-type button on the landing page or fill out form fields just up to the credit card fields (say, for a multistep payment process). A user who clicks on "Buy now" or fills out the first part of the payment form is a pretty convincing datapoint.

I haven't tried any of this myself since it raises a big issue for me: Doing this makes a promise that you don't intend to keep (at that moment). If trust is what gets users to buy, isn't it counter-productive to violate a user's act of trust?

I'd like to get over this because I can see how experiments like this can be useful and I hope you'll keep us all posted with your success rate. Some things I'm curious to know are:

* How much time passed between your AdWords test and your MVP launch? * Of the emails captured, how many participated in the beta and eventually became paying users?

Best of luck!

Re: The Tim Ferris way of testing ideas and how I did it.

#15

Post by sgk284 »

patio11 wrote: post #7
teej wrote: post #2

I've been itching to try this for some time. I would appreciate any war stories HNers could share.

Email addresses don't covert on a 1 to 1 basis to sales. (If. Only.) After the autoresponder runs out (6 days in my case) they've only got only a moderately higher chance of conversion than my average trial user. (~5% vs. 2.8%) I really, really don't like the Put Up Fake AdWords Ad, See Who Gets Suckered minimum viable product. Aside from misleading customers and hurting the AdWords ecosystem (they don't actually get…

I think you've hit on some good points, but missed a big one. Running ad words drives traffic to a site that currently has no traffic, better yet that traffic has shown some interest in the product you're claiming to have. If some of those people try to buy (or express interest in) this non-existent product, that says something.

You're not measuring how much traffic ad words brings in. You're using ad words as a traffic generator (ideally generating interested traffic), and of this traffic that comes to your site, you can determine if people are clicking through.

For ~$100 you can determine if there is any interest at all. This can be powerful for people who really have no idea what the market will do. It's not perfect, but not useless.

Re: The Tim Ferris way of testing ideas and how I did it.

#16

Post by dstorrs »

utnick wrote: post #5
tocomment wrote: post #4

That seems hard to believe. I did the same testing strategy with my Job applier software ( http://fastjobapplier.weebly.com/ ) and first of all I only got around 30 ad clicks total, and no one filled out the form. (I was thoroughly disappointed.) Granted maybe I just suck at sales copy, or perhaps there just isn't demand out there for software that helps you apply for jobs. Still though, a 10%+ conversion rates seems…

yea 10% seems pretty high, I think it might have to do with saying the 1st month is free and you can cancel any time. Therefore, the 10% rate isn't really a 10% sales rate but more of a 10% willing to try your demo. But 10% is still pretty good for that I think.

Actually, I found it pretty believable that this percentage put their email in. I ran something similar...a SurveyMonkey questionnaire about the product, where the last question was "If you'd like to be notified when this launches, put your email here." 25-30% of those who finished the survey gave their email.

This is ONLY email, of course, not actual signups. I'm expecting 90+% of those people will drop out once we actually email them.

Re: The Tim Ferris way of testing ideas and how I did it.

#17

Post by patio11 »

sgk284 wrote: post #15
patio11 wrote: post #7

Earlier quoted context omitted.

Email addresses don't covert on a 1 to 1 basis to sales. (If. Only.) After the autoresponder runs out (6 days in my case) they've only got only a moderately higher chance of conversion than my average trial user. (~5% vs. 2.8%) I really, really don't like the Put Up Fake AdWords Ad, See Who Gets Suckered minimum viable product. Aside from misleading customers and hurting the AdWords ecosystem (they don't actually get…

I think you've hit on some good points, but missed a big one. Running ad words drives traffic to a site that currently has no traffic, better yet that traffic has shown some interest in the product you're claiming to have. If some of those people try to buy (or express interest in) this non-existent product, that says something. You're not measuring how much traffic ad words brings in. You're using ad words as a traf…

This can be powerful for people who really have no idea what the market will do.

These people should take their hands off the keyboard and not put it back on until they have spoken to at least one Real Person who has the problem that they are trying to solve. After they have done this, they will know if there is any interest at all. This produces actionable insights: if they tell you they have a problem, then at least some people have the problem. If they tell you that the industry's favorite way for solving the problem costs $1,000, that immediately establishes that there is a viable market for solving the problem.

Compare this to the AdWords campaign: can you tell me what numbers you can get that say "This is a good idea -- do this" and what numbers you can get that say "This is not a good idea -- don't do this"? No -- subjectively "good" results don't imply purchasing intent, subjectively "bad" results don't imply lack of it. You'll get noise -- very precise noise, noise you can measure down to the click, but still noise.

Re: The Tim Ferris way of testing ideas and how I did it.

#18

Post by dawie »

andymism wrote: post #14
dawie wrote: post #9

Earlier quoted context omitted.

I agree with you. Maybe a good idea would be to ask for a credit card, but not to charge it. I think this borders on being illegal though. I am still pretty happy with 10%, since this was my cut-off for going ahead and building the product.

Instead of asking for the credit card information (which does sound illegal), I've heard of others who have placed a "Buy Now"-type button on the landing page or fill out form fields just up to the credit card fields (say, for a multistep payment process). A user who clicks on "Buy now" or fills out the first part of the payment form is a pretty convincing datapoint. I haven't tried any of this myself since it raises…

I will definitely do a follow-up on this. It seems like there is quite a bit of interest in this type of testing. I will provide some numbers as soon as I have them.

Re: The Tim Ferris way of testing ideas and how I did it.

#19

Post by cschneid »

BobbyH wrote: post #13
dawie wrote: post #9

Earlier quoted context omitted.

I agree with you. Maybe a good idea would be to ask for a credit card, but not to charge it. I think this borders on being illegal though. I am still pretty happy with 10%, since this was my cut-off for going ahead and building the product.

In his book, I believe that Tim says that it's not illegal if you don't store a user's credit card information when selling a product that isn't fully-baked. But I agree that it is a bit shady...

"This product is currently in private beta. Enter your credit card details to guarantee your spot in the next phase of release. You won't be charged a cent until you get access"

Then do the authorize.net CC storage thing where you don't even need to worry about securing private details (pre-auth, don't capture payment till you have a product to sell).

Re: The Tim Ferris way of testing ideas and how I did it.

#20

Post by sgk284 »

patio11 wrote: post #17
sgk284 wrote: post #15

Earlier quoted context omitted.

I think you've hit on some good points, but missed a big one. Running ad words drives traffic to a site that currently has no traffic, better yet that traffic has shown some interest in the product you're claiming to have. If some of those people try to buy (or express interest in) this non-existent product, that says something. You're not measuring how much traffic ad words brings in. You're using ad words as a traf…

This can be powerful for people who really have no idea what the market will do. These people should take their hands off the keyboard and not put it back on until they have spoken to at least one Real Person who has the problem that they are trying to solve. After they have done this, they will know if there is any interest at all. This produces actionable insights : if they tell you they have a problem, then at lea…

Suppose I came up with an idea that I absolutely knew was brilliant, let's say edible shoelaces. Unfortunately these edible shoelaces aren't cheap to produce. I'm a one man shop and have almost no money.

I make a fully functional website to sell this product, only I don't have the product yet. When people browse the site and have gone through the pricing page, etc... and finally click "buy" I simply tell them that we're out of stock.

Now I want to see if people are actually trying to buy this thing, so I start marketing but have a limited budget. I throw $1000 at ad words. If I get 100 people trying to buy my edible shoelaces at $20 each then I know that maybe it's worth investing in manufacturing the product.

How is this not useful data? Instead of going all out and spending money on manufacturing and a website, you only pay for the website and hold off on the manufacturing expense until you're confident it'll pay off.

Post reply on HN