Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: How to price a service that saves hundreds of engineering hours?

news.ycombinator.com

Post reply on HN
Print view View original

41–50 of 89 posts

Re: Ask HN: How to price a service that saves hundreds of engineering hours?

#41

Post by tracerbulletx »

zmgsabst wrote: post #31
Silhouette wrote: post #13

Earlier quoted context omitted.

100/day is not reasonable. If I'm understanding OP correctly then at that level this tool would be saving the customer's engineers 14 minutes of disruptive downtime 100 times per day. If that's correct then $100/day may indeed be unreasonable, but because it's way too low. I doubt the useful target market for a tool like this is in the $20/month or even $100/month category. What does a series A/B company that runs 10…

Okay — but running 60 copies of a small EC2 for 1 min is $0.009 to $0.04 Running 100x test suites would be $0.90 to $4, in server costs. Charging 25-100x the server costs is optimistic, at best.

The only time your costs should have anything to do with how you price something is when your customers know how much they are and they influence how much they'll pay, or setting lower limits so you don't go out of business.

Re: Ask HN: How to price a service that saves hundreds of engineering hours?

#42

Post by jackblemming »

polskibus wrote: post #15
VoidWhisperer wrote: post #3

With Cypress, atleast in what I've heard of (and seen), flaky tests seem to be a larger waste of developer time than the time it takes to run the test suite overall. Does this help improve that?

What has the reputation of being less flaky then Cypresss? My experience is that it is easier to write a non flaky test in Cypress than in Selenium.

Nothing. The problem is E2E UI tests are always going to be flaky.

That’s why many recommend unit tests or pseudo E2E tests that save a copy of an API response. Unfortunately this advice is rarely followed.

Re: Ask HN: How to price a service that saves hundreds of engineering hours?

#43

Post by OJFord »

> completes all tests (no matter how many you've got) in less than a minute.

> My thinking is to charge 2 cents per a test-minute

Doesn't that reduce to 2c/run?

That aside, my gut reaction was don't charge for time if your selling point is how much time you save.

Charge for #tests, or something even better targetted at what makes it slow usually (I'm not familiar) - you charge some (<100) percentage of the reason tests are expensive and make that go away, and it seems like a sound choice to use your service doesn't it?

Re: Ask HN: How to price a service that saves hundreds of engineering hours?

#44

Post by banana_giraffe »

punkpeye wrote: post #35
zmgsabst wrote: post #31

Earlier quoted context omitted.

Okay — but running 60 copies of a small EC2 for 1 min is $0.009 to $0.04 Running 100x test suites would be $0.90 to $4, in server costs. Charging 25-100x the server costs is optimistic, at best.

Unfortunately, EC2 is not an option if we want to deliver 1 minute promise. I need VMs that I can flash start with all the software needed to run the tests immediately available. The best I was able to achieve is 20 seconds using my current provider. However, they charge in hour increments. That being said, I am constantly experimenting how to reduce costs. I think the final result is going to be a fairly complicated…

Would Lambda be an option? Not sure what the stack looks like here, but a quick Google suggests people have gotten Cypress to run under Lambda, at least.

Re: Ask HN: How to price a service that saves hundreds of engineering hours?

#45

Post by Silhouette »

punkpeye wrote: post #32
Silhouette wrote: post #24

Earlier quoted context omitted.

Speaking from experience, the promise of completing tests in 1 minute or less is very appealing. Indeed. There is a qualitative difference between tests completing fast enough that a dev is OK with just waiting a few seconds for the results and tests becoming a "go do something else for a while" background activity. There is also a qualitative difference between tests running fast enough that if something does go wro…

Another thing to consider is that most engineers do not run all integration tests locally before committing them, simply because they take long time, i.e. A typical workflow is: commit changes -> raise a PR -> wait for all tests to complete -> [if tests fails, go back to step 1] This hugely impacts developer ability to switch context, because they need to stash code, switch branches, etc. If all integration tests cou…

I agree. That's another reasonable point.

If I were in your position I would absolutely be emphasizing these kinds of qualitative benefits in the marketing even if your pricing model is actually very quantitative in the end.

Anyone who runs the kind of engineering group you're currently aiming at won't have to think very hard to get an idea of the quantitative scale of the benefits they might enjoy as a result. Ideally you want authorising a few (or more than a few) thousand bucks a month to be a quick and obvious decision for them. "We provide a simple tool that enables radically better development processes with likely ROI between 10x and 100x" is a decent elevator pitch to people like VP Engineering and CFO who are going to end up making a purchasing decision.

Re: Ask HN: How to price a service that saves hundreds of engineering hours?

#46

Post by janosdebugs »

punkpeye wrote: post #35
zmgsabst wrote: post #31

Earlier quoted context omitted.

Okay — but running 60 copies of a small EC2 for 1 min is $0.009 to $0.04 Running 100x test suites would be $0.90 to $4, in server costs. Charging 25-100x the server costs is optimistic, at best.

Unfortunately, EC2 is not an option if we want to deliver 1 minute promise. I need VMs that I can flash start with all the software needed to run the tests immediately available. The best I was able to achieve is 20 seconds using my current provider. However, they charge in hour increments. That being said, I am constantly experimenting how to reduce costs. I think the final result is going to be a fairly complicated…

You could always be pre-pooling VMs once you hit a certain scale.

Re: Ask HN: How to price a service that saves hundreds of engineering hours?

#47

Post by janosdebugs »

banana_giraffe wrote: post #44
punkpeye wrote: post #35

Earlier quoted context omitted.

Unfortunately, EC2 is not an option if we want to deliver 1 minute promise. I need VMs that I can flash start with all the software needed to run the tests immediately available. The best I was able to achieve is 20 seconds using my current provider. However, they charge in hour increments. That being said, I am constantly experimenting how to reduce costs. I think the final result is going to be a fairly complicated…

Would Lambda be an option? Not sure what the stack looks like here, but a quick Google suggests people have gotten Cypress to run under Lambda, at least.

In my experience Lambda can have a pretty slow start when it's cold.

Re: Ask HN: How to price a service that saves hundreds of engineering hours?

#48

Post by karaterobot »

patio11 wrote: post #30

At Series A / Series B, they can’t think about serious commitments (and this sounds like one) in increments smaller than $10k. I’d try that and, if you get any pushback, ask what it would have to do to be worth $10k. (A month. Bill quarterly or yearly at their option; raise prices annually to capture increase in use.) Remember that their alternative is not “spin up VPSes” it is “staff an engineering rotation to maint…

That doesn't square with my experiences at startups. A $1k/mo service would have been something a manager could sign off on themselves, and all it took to do that was to get an engineer on the team be really excited about it. On the other hand, a $10K a month would have been a much harder conversation to have, falling outside the discretionary budget of a particular manager, and probably requiring meetings and demos with the head of the department to get approval.

Re: Ask HN: How to price a service that saves hundreds of engineering hours?

#49

Post by bequanna »

punkpeye wrote: post #26
Forge36 wrote: post #16

Ask your customers. Propose your current amount and be honest "do you think this is fair?" If they say yes great! If not, ask what they think is reasonable. Set yourself a lower limit to keep it profitable for yourself. "Sorry I can't price it below X". See how they respond.

I am trying to figure out how to scale this to the point where unit economics makes the most sense. If I simply transferred current costs on to the end user, it would be too much. It would be USD 0.20 x number of tests x number of hours, e.g. a company that works 9 to 5 and has 100 tests, that is 0.20 100 8=USD 160 per day. However, changes the more clients you have. If I have 10 clients paying for 100 concurrent tes…

Why are you assuming that you will need to maintain a static infrastructure?

Aren’t you able to (fairy quickly) autoscale up/down based on the size of some queue of test requests?

Re: Ask HN: How to price a service that saves hundreds of engineering hours?

#50

Post by Silhouette »

patio11 wrote: post #30

At Series A / Series B, they can’t think about serious commitments (and this sounds like one) in increments smaller than $10k. I’d try that and, if you get any pushback, ask what it would have to do to be worth $10k. (A month. Bill quarterly or yearly at their option; raise prices annually to capture increase in use.) Remember that their alternative is not “spin up VPSes” it is “staff an engineering rotation to maint…

Remember that their alternative is not “spin up VPSes” it is “staff an engineering rotation to maintain this service” which will cost them minimally a million annually.

But the default alternative is "continue making limited use of these integration tests and continue accepting the delays of running them slowly" which has a cost of $0 compared to status quo.

What proportion of series A companies that are trying to scale up and show results quickly so they are still around to raise a series B is going to spend $1M+ on a dedicated team to run a service for this? I'm guessing 0.000%. That means the comparison isn't a very compelling argument for the leaders in that organisation because really you're just arguing that your solution is better than something they obviously weren't going to do anyway.

Post reply on HN