Live data from Hacker News

Ask HN: How to price a service that saves hundreds of engineering hours?

news.ycombinator.com

21–30 of 89 posts

Re: Ask HN: How to price a service that saves hundreds of engineering hours?

#21
post #12

You can price based on eng time saved, but the rejoinder will be that the devs use test time to answer emails, respond in slack/jira, etc. And to some extent that's probably true. I do have some very strong advice on how not to charge: (1) do not charge based on your costs (customers won't care), and (2) do not charge based on fine-grained usage. The latter is important because if a VPE signs a contract for $3k/mo, h…

The latter is important because if a VPE signs a contract for $3k/mo, he/she doesn't want to sit there worrying that devs run a couple extra tests or add a few tests, etc, and boom: all of a sudden the VPE is going to the cfo to explain $10k in overage charges.

Counterpoint: I have personally seen companies of the size OP is talking about run into unexpected big jumps in spending on key infrastructure and tooling because they made some architectural change and then suddenly realised it was going to run the tests 10x as often or something. Without exception the response has been twofold. First they did take the immediate financial hit because they couldn't afford to be without a critical system. Second they diverted significant resources to getting the usage levels back under control very quickly so they didn't have to rework that critical infrastructure as well.

If anything, since bringing the costs of some important infrastructure down is exactly the problem OP's tool is solving and its value is directly demonstrable in terms of time and therefore money saved, your general argument should be a positive sign for OP that they probably have a viable business model.

I could buy an argument that tiered pricing might work better for OP than metered if someone showed up with data supporting the theory. But metered pricing seems to be working out pretty well for a lot of popular cloud-hosted dev tools and of course is almost universal among cloud infrastructure providers. People complain a lot about AWS having unpredictable pricing but I heard AWS has still had some success breaking into the cloud hosting market and probably it's making enough money that Amazon might keep it running. To be fair that's partly because although AWS lacks safeguards that would certainly be welcomed by many it does also have a reputation for being very decent about support in the occasional "Oh s@£t" scenarios.

Re: Ask HN: How to price a service that saves hundreds of engineering hours?

#22

You need to determine which type of customer you’re targeting. At the high end, some enterprise companies won’t blink at the prices you’re suggesting. However, they will expect a level of customer support and service agreements that are going to make you work hard to deliver what they ask. You’re also going to need to be heavily involved in sales cycles and support, which will start costing you many, many hours just…

> However, I suspect you’re massively overestimating the way that test times are a blocker for most engineer workflows. Even with long text suites, we don’t just sit and stare at the monitor while we wait. That’s time for responding to communications, checking other things, taking a short break and so on. You could have a hard time selling engineers on the idea that they can do more work and take fewer breaks while paying for the privilege.

Speaking from experience, my productivity is down by like 80% if I know that there is a test I am waiting to complete. I can do other things, but I will constantly switch back and forth to check on the progress. I know from talking with other engineers that they feel similar, esp. when flakky tests are involved.

However, I get what you are staying. I will think how to position this in a more compelling way that cannot be easily challenged.

Re: Ask HN: How to price a service that saves hundreds of engineering hours?

#23
post #9

With Cypress, atleast in what I've heard of (and seen), flaky tests seem to be a larger waste of developer time than the time it takes to run the test suite overall. Does this help improve that?

It sure does. The way Cypress works is that it injects their entire driver (test API) to the browser and then loads the tests themselves, also in the browser. It is a finicky process because they need to rewrite all JS and HTML ( https://github.com/cypress-io/cypress/tree/develop/packages/... ). It is slow because they have to proxy everything through HTTP 1.1 proxy ( https://github.com/cypress-io/cypress/tree/develo…

I wouldn’t give away too many technical details of how things work.

Re: Ask HN: How to price a service that saves hundreds of engineering hours?

#24

It is an interesting idea, but your approach to pricing seems off: 100/day is not reasonable, at least not for small/medium teams. Trying to do arithmetic on run times and hourly rates to convince people it is reasonable is not only overly simplistic, but actively distracts from the conversation you really want to have - how does your solution solve their problem? People buy when they believe you can solve a problem…

Speaking from experience, the promise of completing tests in 1 minute or less is very appealing. This is one thing I don't want to compromise. In the case of the two companies that are helping me to test this service, they both pay a fixed fee of USD 2000 for an agreed maximum parallelization of 50 browsers. This means that up to 50 concurrent tests, I can still deliver the promise of 1 minute tests. For context, Bro…

Speaking from experience, the promise of completing tests in 1 minute or less is very appealing.

Indeed. There is a qualitative difference between tests completing fast enough that a dev is OK with just waiting a few seconds for the results and tests becoming a "go do something else for a while" background activity. There is also a qualitative difference between tests running fast enough that if something does go wrong the code can often be backed out before it interferes with anyone else and tests that run slowly enough that the CI system can't keep up with merges from the team and changes start piling up that then need sorting out somehow every time a set of tests does fail.

Re: Ask HN: How to price a service that saves hundreds of engineering hours?

#25
Do you have a market strategy otherwise?

What is your customer base and target market? Both pricing models and numbers will come out very differently for the same underlying technical work depending on if you're aiming for fewer how-much-could-a-banana-cost-biggies (these need service agreements and someone who they can talk to on a first-name basis) or a turnkey system for every startup and corporate using Cypress out there (they just register a credit card and e-mail and start integrating).

While you are considering your margins, you're going by value-based pricing here - you focus less on how much it costs you to build and run and your bottom line and more on how valuable it is to the potential customer. Some told you hundreds of hours were saved.

It sounds like you have the pricing model for a mass-market SaaS with the pricing for the high-end.

One risk with such high value-based pricing is that assuming initial success, some happy customers might get restless after a few weeks/months/years and realize that for the price they're paying you they could actually save money by building and rolling something similar in-house. The customers who are ready to pay those prices are precisely the ones likely to do it, I think. In any case, unless your market is super niche or you have some crazy edge, you will get competitors if you're successful enough. Maybe even straight-up copycats. This is nothing bad at all but it can be good to keep in mind that you may lose customers if they're able to slaughter you on cost-performance.

Also consider that these higher-end clients likely care a lot about availability and stability. If you don't have brand-recognition or a personal reference, it can be hard to convince them to take a shot. Free trials will never be enough for them to be confident either. Initial underpricing (not too much obv) is a way to compensate for that initially.

But again, I think we need more context before we can make a fair assessment of if the price is reasonable or not.

Re: Ask HN: How to price a service that saves hundreds of engineering hours?

#26
post #16

Ask your customers. Propose your current amount and be honest "do you think this is fair?" If they say yes great! If not, ask what they think is reasonable. Set yourself a lower limit to keep it profitable for yourself. "Sorry I can't price it below X". See how they respond.

I am trying to figure out how to scale this to the point where unit economics makes the most sense.

If I simply transferred current costs on to the end user, it would be too much. It would be USD 0.20 x number of tests x number of hours, e.g. a company that works 9 to 5 and has 100 tests, that is 0.201008=USD 160 per day.

However, changes the more clients you have. If I have 10 clients paying for 100 concurrent test capacity, I wouldn't need 1000 VMs, as they are unlikely to be all running tests at the same time, all the time.

Re: Ask HN: How to price a service that saves hundreds of engineering hours?

#27
Different pricing strategies:

Market Pricing: what are your competitors charging.

Cost plus: What is your investment to date and what do you see ongoing marginal costs as, and what type of pricing do you need to make this worthwhile for you financially?

nearest substitute pricing: what’s the financial savings of your product versus the competitor? Charge a price that captures some of this value (what you’re proposing)

Also be mindful of Cypress possible taking steps to improve their own run times. What will you do to stay ahead and keep your marketshare? include this in your considerations as ongoing r&d.

Re: Ask HN: How to price a service that saves hundreds of engineering hours?

#28

I think your pricing math is a tad off. I don't think it's harmful to use that for marketing, like you're saving engineering hours, but when you use it for pricing it falls apart. The reason is that you're selling to engineers, and we all know as CI is running, we aren't blocked. So, it isn't a 1:1 minute saved, although, I wouldn't conflate that with zero saved. In my experience a 1-5 minute test run isn't much diff…

The only time I felt blocked by CI was at a company whose tests had a 50/50 chance of failing. The tests also took upwards of an hour to run (the engineering culture was a cult of True Believers in a terrible technology stack).

I think having something like this might have actually kept me at that job, at least a bit longer, though we had the same problem with our unit tests, and I don't even recall if Cypress was what we used (probably not, now that I think about it).

Re: Ask HN: How to price a service that saves hundreds of engineering hours?

#29

You need to determine which type of customer you’re targeting. At the high end, some enterprise companies won’t blink at the prices you’re suggesting. However, they will expect a level of customer support and service agreements that are going to make you work hard to deliver what they ask. You’re also going to need to be heavily involved in sales cycles and support, which will start costing you many, many hours just…

> However, I suspect you’re massively overestimating the way that test times are a blocker for most engineer workflows. Even with long text suites, we don’t just sit and stare at the monitor while we wait. That’s time for responding to communications, checking other things, taking a short break and so on. You could have a hard time selling engineers on the idea that they can do more work and take fewer breaks while p…

I agree. It's hard to work when you have a 10-minute test running, either I wait/do busywork, or I start doing something else and then I don't want to interrupt that line of work to context shift back to a failed test.

Re: Ask HN: How to price a service that saves hundreds of engineering hours?

#30
At Series A / Series B, they can’t think about serious commitments (and this sounds like one) in increments smaller than $10k. I’d try that and, if you get any pushback, ask what it would have to do to be worth $10k.

(A month. Bill quarterly or yearly at their option; raise prices annually to capture increase in use.)

Remember that their alternative is not “spin up VPSes” it is “staff an engineering rotation to maintain this service” which will cost them minimally a million annually.

Post reply on HN