Hey Michael, apart from custom internal analytics tools and solutions similar to GA which other tools would you suggest using?
“Scientific Method for Startups”, and AMA with Michael Seibel
11–20 of 50 posts
Re: “Scientific Method for Startups”, and AMA with Michael Seibel
#12[deleted]
Re: “Scientific Method for Startups”, and AMA with Michael Seibel
#13I actually always translated the lean startup model as the scientific method for startups in my head and while explaining to others. My question: how do you apply that when you have tens of visitors/users? Not nearly enough to have a reliable ab test. Edit: so far he answered only one question. Not much of a AMA, right? Maybe remove it from the title.
It's not as scientific, but you can prepare a/b tests for paper prototypes and ask your users to go through them as they would go through the site.
Quantitative methods aren't reliable with such a small dataset so you will have to rely on assumptions and guesses. Unfortunately this makes your observations prone to bias from your hypothesis, so it's especially important to be mindful of this and keep them separate.
Edit: Didn't realize this was an AMA but hopefully my answer contributes to the discussion.
Re: “Scientific Method for Startups”, and AMA with Michael Seibel
#14I actually always translated the lean startup model as the scientific method for startups in my head and while explaining to others. My question: how do you apply that when you have tens of visitors/users? Not nearly enough to have a reliable ab test. Edit: so far he answered only one question. Not much of a AMA, right? Maybe remove it from the title.
It's not as scientific, but you can prepare a/b tests for paper prototypes and ask your users to go through them as they would go through the site. Quantitative methods aren't reliable with such a small dataset so you will have to rely on assumptions and guesses. Unfortunately this makes your observations prone to bias from your hypothesis, so it's especially important to be mindful of this and keep them separate. Ed…
Sure, it does! I just added it to register that is weird to publish as an AMA and not answer many questions.
I think your approach is right. One thing that helps me is to use hypothesis from users/visitors/customer as much as possible. Talk to customers and try to distill what are their hypothesis about a specific problem. Talking to customer goes a long way.
On the topic, I think that famous quote atributed to Ford "If I asked my customer what they wanted, I would be trying to create faster horses". I think is BS (and probably bogus). That ad absurdum scenario would only happens if he asked the wrong questions and/or followed the most superficial answers.
Re: “Scientific Method for Startups”, and AMA with Michael Seibel
#15What are some best practices for evaluating ideas before building anything. You mentioned talking to users which is good advice but assumes that an industry has been identified already.
>>> assumes that an industry has been identified already. This is one of the key take aways from customer discovery - is your customer actively looking for a solution to the pain / problem you've identified. If your customer is actively looking for a solution than it is easy(ier) to identify and talk with them. If no one is looking for a solution then it may be that you haven't identified a real pain, you haven't ide…
Ok, so following what I think you're saying, how does one find people with problems they're looking to solve?
Re: “Scientific Method for Startups”, and AMA with Michael Seibel
#16Earlier quoted context omitted.
>>> assumes that an industry has been identified already. This is one of the key take aways from customer discovery - is your customer actively looking for a solution to the pain / problem you've identified. If your customer is actively looking for a solution than it is easy(ier) to identify and talk with them. If no one is looking for a solution then it may be that you haven't identified a real pain, you haven't ide…
Ok, so following what I think you're saying, how does one find people with problems they're looking to solve?
Ideally you'd be a user of your own product, someone who is actively experiencing the pain and is looking for a solution. Then you'd look for people like yourself.
Otherwise you still want to follow the general pattern of talk to whomever stands to gain the most from your service. Note that the person that stands to gain the most might not be who you first assume it is.
Re: “Scientific Method for Startups”, and AMA with Michael Seibel
#17I actually always translated the lean startup model as the scientific method for startups in my head and while explaining to others. My question: how do you apply that when you have tens of visitors/users? Not nearly enough to have a reliable ab test. Edit: so far he answered only one question. Not much of a AMA, right? Maybe remove it from the title.
sorry for the delay :) - its really hard to do active a/b tests in the early days - often I tell founders that feedback cycle then has to rely of direct interaction with customers alone. Which I think is a good habit for all startups to keep even when they can get quantitive data.
Re: “Scientific Method for Startups”, and AMA with Michael Seibel
#18This was a better article than I expected. Finding ways to measure your assumptions is great, because if you can measure the behavior, you can measure the impact of changes.
Generally, I'm not a fan of "scientific method" as a phrase. It's too often used by people dismissing intuition in a very non-scientific way.
Re: “Scientific Method for Startups”, and AMA with Michael Seibel
#19Earlier quoted context omitted.
It's not as scientific, but you can prepare a/b tests for paper prototypes and ask your users to go through them as they would go through the site. Quantitative methods aren't reliable with such a small dataset so you will have to rely on assumptions and guesses. Unfortunately this makes your observations prone to bias from your hypothesis, so it's especially important to be mindful of this and keep them separate. Ed…
Sure, it does! I just added it to register that is weird to publish as an AMA and not answer many questions. I think your approach is right. One thing that helps me is to use hypothesis from users/visitors/customer as much as possible. Talk to customers and try to distill what are their hypothesis about a specific problem. Talking to customer goes a long way. On the topic, I think that famous quote atributed to Ford…
Its funny - I think the Ford quote is right. I always tell folks to not ask a customer what they want but ask them what their problem is. Its the customer task to describe and demonstrate their problem - its the founder's task to think up and test solutions for that problem.
Re: “Scientific Method for Startups”, and AMA with Michael Seibel
#20I always love the debate on whether great products are imagined or are discovered through experimentation (certainly a bit of both). This post details the advice I usually give to startups on this topic and I'm happy to answer questions and join the discussion.
In the creativity bucket, there's also a fair amount of inspiration from:
1. Studying other products and reasoning about them.
2. Conceptually dissecting multiple existing products and looking at different combinations of some subset of their components.
Any thoughts?
PS: Loved your essay