Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: A Platform For Self-Experimentation

doubleblinded.com

Post reply on HN
Print view View original

11–20 of 45 posts

Re: Show HN: A Platform For Self-Experimentation

#11

Post by aresant »

A very cool idea @ the high level, but one fatal flaw:

"At the end of each day you will fill out a short survey with a few questions evaluating the effects of the pills."

Self-reporting is an unsuitable mechanism to draw out scientific results.

There's an excellent detailed explanation available (1) but in TL;DR here are four of the most compelling factors at work:

1 - Honesty/Image management

2 - Introspective ability

3 - Understanding / Question Interpretation

4 - Response bias

Take Image Management & Response Bias - participants know that they will be able to see their results vs. the control group and it's not a leap to realize how easily our ego and even subconscious need for validation could dramatically skew the full study results.

(1) http://www.sciencebrainwaves.com/the-dangers-of-self-report/

Re: Show HN: A Platform For Self-Experimentation

#12

Post by detaro »

alexbock wrote: post #5

This makes me wonder if there is a simple procedure for doing a pseudo-double-blind study on oneself (without a third party). That is, trying something that can be prepared in pill form in such a way that you don't know during any given day/week/month if it was a placebo, but that you can still determine that later to collate the data. I know a lot of people that swear by particular unregulated supplements or dietary…

If you can pill it, you could add unique markers (maybe apply pseudo-random patterns to pill blanks) and take pictures before randomization and immediately before consumption. Or build something that dispenses a random choice from multiple boxes and keeps a log.

Re: Show HN: A Platform For Self-Experimentation

#13

Post by INTPnerd »

alexbock wrote: post #5

This makes me wonder if there is a simple procedure for doing a pseudo-double-blind study on oneself (without a third party). That is, trying something that can be prepared in pill form in such a way that you don't know during any given day/week/month if it was a placebo, but that you can still determine that later to collate the data. I know a lot of people that swear by particular unregulated supplements or dietary…

Yes, independent self experimentation is the way to go. You are not hoping that you can trust the people doing the study. You don't need to hope they did things the way they were reportedly done. You don't need to hope someone did not get paid off to tweak the numbers at the end. You just need to educate yourself on how to do proper studies and how to avoid false conclusions and techniques to bypass your own biases. But you also need to realize the results only apply to you at this part of your life. You can't promote your discoveries as being an absolute cure for other peoples problems. Even your own body can change over time and what used to work for you may no longer work.

Re: Show HN: A Platform For Self-Experimentation

#14

Post by chrisrxth »

tertius wrote: post #8
dangrossman wrote: post #7

It's hard to do a test like this when there's a significant chance you're buying nothing but powdered rice regardless of what the bottle says. http://well.blogs.nytimes.com/2015/02/03/sidebar-whats-in-th... http://www.nytimes.com/2013/11/05/science/herbal-supplements... Want to do a double-blind study on whether St. John's Wort improves your mood? None of the bottles tested from Walmart, Target, GNC or Walgreens actu…

For day to day things, this is what I consult. https://labdoor.com/

Good call. consumerlab.com is good too.

Re: Show HN: A Platform For Self-Experimentation

#15

Post by chrisrxth »

dangrossman wrote: post #7

It's hard to do a test like this when there's a significant chance you're buying nothing but powdered rice regardless of what the bottle says. http://well.blogs.nytimes.com/2015/02/03/sidebar-whats-in-th... http://www.nytimes.com/2013/11/05/science/herbal-supplements... Want to do a double-blind study on whether St. John's Wort improves your mood? None of the bottles tested from Walmart, Target, GNC or Walgreens actu…

That's a valid concern, and I agree that it's alarming. We will disclose where we are obtaining the L-Theanine from and provide a Certificate of Analysis.

Also, since this is a double blinded trial, if there are significant differences between placebo and real powders then that affirms that the powder was real.

Re: Show HN: A Platform For Self-Experimentation

#16

Post by Uptrenda »

Isn't there a huge potential for crowd sourced research platforms to take off and help advance the state of the art?

So far it seems like most studies are done by scientists / academics in a very closed off kind of way, but the true nature of science is open to everyone and with the right platform we might be able to use normal people to gather enough data to confirm or deny our hypotheses in a way that's much more open than the standard study.

Essentially, anyone would be able to test their hypotheses and either help advance scientific knowledge or at least provide extremely interesting markers that could be used as a basis to get funding for more rigorous / controlled studies. This approach obviously won't work well for every kind of question but just as the OP has highlighted: it's particularly well-suited for making self-experimentation more rigorous and I imagine it would work well for the social sciences.

Imagine if we gave independent researchers like Gwern the tools they needed to help answer more questions. If it was designed right: it would also be an incredible learning tool for the scientific method (since you could participate in the experiments + propose new ones.)

Are there any platforms like this in existence, I wonder? U-Uber for science?

Re: Show HN: A Platform For Self-Experimentation

#17

Post by peteretep »

alexbock wrote: post #5

This makes me wonder if there is a simple procedure for doing a pseudo-double-blind study on oneself (without a third party). That is, trying something that can be prepared in pill form in such a way that you don't know during any given day/week/month if it was a placebo, but that you can still determine that later to collate the data. I know a lot of people that swear by particular unregulated supplements or dietary…

If you're willing to involve a friend for 30 seconds, get them to label pillboxes and make a note of which pillboxes were labelled.

Re: Show HN: A Platform For Self-Experimentation

#18

Post by ryporter »

alexbock wrote: post #5

This makes me wonder if there is a simple procedure for doing a pseudo-double-blind study on oneself (without a third party). That is, trying something that can be prepared in pill form in such a way that you don't know during any given day/week/month if it was a placebo, but that you can still determine that later to collate the data. I know a lot of people that swear by particular unregulated supplements or dietary…

I, too, have thought about conducting a pseudo-double-blind experiment on myself. I think I could pull it off, but I'd also be willing to pay a third party to help me. For example, DoubleBlinded could partner with a respect supplement maker, and offer personal experiments in any supplement made by their partner. The consumer wouldn't have to worry about setting up the experiment correctly, and they would get their own results immediately upon completion. Aggregate studies can wait until enough customers have self-experimented on the same supplement, and DoubleBlinded can afford to wait that long, because the partnership model keeps their costs low.

Re: Show HN: A Platform For Self-Experimentation

#19

Post by meta_AU »

aresant wrote: post #11

A very cool idea @ the high level, but one fatal flaw: "At the end of each day you will fill out a short survey with a few questions evaluating the effects of the pills." Self-reporting is an unsuitable mechanism to draw out scientific results. There's an excellent detailed explanation available (1) but in TL;DR here are four of the most compelling factors at work: 1 - Honesty/Image management 2 - Introspective abili…

AFAICT, there isn't a control group. You have 4 weeks of pills and each week it could be real or fake. So you are comparing against yourself.

Re: Show HN: A Platform For Self-Experimentation

#20

Post by chrisrxth »

Uptrenda wrote: post #16

Isn't there a huge potential for crowd sourced research platforms to take off and help advance the state of the art? So far it seems like most studies are done by scientists / academics in a very closed off kind of way, but the true nature of science is open to everyone and with the right platform we might be able to use normal people to gather enough data to confirm or deny our hypotheses in a way that's much more o…

There are some in existence already:

- quantifiedmind.com for self-testing your mind, which has a few group experiments as well. I'm looking into integrating with them as part of doubleblinded.com.

- PACO (by Google) https://www.pacoapp.com/ Pretty cool and flexible platform, although not the most user friendly or easy to figure out. It is open source and has an app, which is cool.

- A few sites like 23andme and ubiome have a lengthy list of survey questions they ask you, which I only imagine they would pass through pattern recognition algorthms to detect correlations between whatever information you give them and your Genome (or gut genome in the case of 23andme).

So yeah, there is stuff out there. We want to do this for nootropics and supplements (and eventually much more).

Post reply on HN