maybe give them knobs on the amount of maximum concurrent tasks?
Ask HN: How to price a service that saves hundreds of engineering hours?
11–20 of 89 posts
Re: Ask HN: How to price a service that saves hundreds of engineering hours?
#12I do have some very strong advice on how not to charge: (1) do not charge based on your costs (customers won't care), and (2) do not charge based on fine-grained usage. The latter is important because if a VPE signs a contract for $3k/mo, he/she doesn't want to sit there worrying that devs run a couple extra tests or add a few tests, etc, and boom: all of a sudden the VPE is going to the cfo to explain $10k in overage charges.
So: price in broad tiers, and price based on value. If you save hundreds of eng hours/mo, charge for the first 20 hours saved. At $150 per, that's $3k/mo. Multiply by 5 to have a functional company (you can't do sales for under $10k; every successful sale -- which will be 20% if you're very good and lucky -- must pay for all failed sales, all marketing, all dev, etc. Oh, and you're doing sales.) So $15k/mo. That's within the approval authority for a vpe with 50 reports.
And accept that bigger companies don't worry about $2k-$3k/mo. So maybe your target should also be smaller shops that can't accept the infra overhead to build this out, but will buy a cheap solution that saves their 1-4 devs a ton of time. Dunno.
edit: and you need a strong cross-tenant security story.
Re: Ask HN: How to price a service that saves hundreds of engineering hours?
#13It is an interesting idea, but your approach to pricing seems off: 100/day is not reasonable, at least not for small/medium teams. Trying to do arithmetic on run times and hourly rates to convince people it is reasonable is not only overly simplistic, but actively distracts from the conversation you really want to have - how does your solution solve their problem? People buy when they believe you can solve a problem…
If I'm understanding OP correctly then at that level this tool would be saving the customer's engineers 14 minutes of disruptive downtime 100 times per day. If that's correct then $100/day may indeed be unreasonable, but because it's way too low.
I doubt the useful target market for a tool like this is in the $20/month or even $100/month category. What does a series A/B company that runs 100 slow rounds of integration testing per day spend on its cloud hosting every month? That's obviously going to be a much bigger number than they'd pay for a service like this alone but it will give you some idea of the magnitude of numbers that are reasonable to discuss. How much does running those tests slowly cost them not only in direct developer salary but also in lost productivity because of the slow feedback, reduced developer satisfaction leading to people being demotivated or even leaving if it gets too frustrating, etc? What is the extra risk to them of not having new projects ready fast enough to add serious value before the next funding round?
Re: Ask HN: How to price a service that saves hundreds of engineering hours?
#14With Cypress, atleast in what I've heard of (and seen), flaky tests seem to be a larger waste of developer time than the time it takes to run the test suite overall. Does this help improve that?
My experience with flaky tests is that they seem to leave some state lying around, or are impacted by some randomness that wasn't accounted for in the test writing (lists in different order, DB state etc). So I actually do think this would benefit that, as these tests are run "independantly", however that's just going to hide the problem of your bad/flaky tests, and you'll never solve the underlying problem.
Re: Ask HN: How to price a service that saves hundreds of engineering hours?
#15With Cypress, atleast in what I've heard of (and seen), flaky tests seem to be a larger waste of developer time than the time it takes to run the test suite overall. Does this help improve that?
Re: Ask HN: How to price a service that saves hundreds of engineering hours?
#16"Sorry I can't price it below X". See how they respond.
Re: Ask HN: How to price a service that saves hundreds of engineering hours?
#17Re: Ask HN: How to price a service that saves hundreds of engineering hours?
#18At the high end, some enterprise companies won’t blink at the prices you’re suggesting. However, they will expect a level of customer support and service agreements that are going to make you work hard to deliver what they ask. You’re also going to need to be heavily involved in sales cycles and support, which will start costing you many, many hours just to get the sale.
Are you prepared to handle this and stay responsive? If not, you will have mixed results pushing high priced offerings, especially if they have deal-breaker limitations (like your 30 second test limit) that could saddle the customer with even more work to integrate and use your tool. You have to acknowledge that your service isn’t purely savings for the customer. It’s work to implement and maintain on their end, as well as handle billing and so on. You cannot make sweeping calculations about savings without ignoring the costs it brings to a company to add your tool.
On the other end of the spectrum, you could target the self-serve market with lower priced services. These offerings can get away with a “take it or leave it” type of service where customers are mostly on their own to evaluate, pay for, and self-support the software. If the tool is good, engineers will try to make it work. However, I suspect you’re massively overestimating the way that test times are a blocker for most engineer workflows. Even with long text suites, we don’t just sit and stare at the monitor while we wait. That’s time for responding to communications, checking other things, taking a short break and so on. You could have a hard time selling engineers on the idea that they can do more work and take fewer breaks while paying for the privilege. Choose your marketing angle wisely.
Re: Ask HN: How to price a service that saves hundreds of engineering hours?
#19It is an interesting idea, but your approach to pricing seems off: 100/day is not reasonable, at least not for small/medium teams. Trying to do arithmetic on run times and hourly rates to convince people it is reasonable is not only overly simplistic, but actively distracts from the conversation you really want to have - how does your solution solve their problem? People buy when they believe you can solve a problem…
In the case of the two companies that are helping me to test this service, they both pay a fixed fee of USD 2000 for an agreed maximum parallelization of 50 browsers. This means that up to 50 concurrent tests, I can still deliver the promise of 1 minute tests.
For context, BrowserStack, which they used before, is charging USD 999 for 5 parallel browsers. I believe BrowserStack has this pricing model because they assume long running tests. It is a lot harder to share VMs if tests take 10-15 minutes than if they take 1 minute.
I could have pricing as USD 999 for 50 browser parallelization, and then charge extra if either of their tests take longer than 1 minute to execute. This model would ensure that we don't have one client hogging all of the resources.
Re: Ask HN: How to price a service that saves hundreds of engineering hours?
#20You need to determine which type of customer you’re targeting. At the high end, some enterprise companies won’t blink at the prices you’re suggesting. However, they will expect a level of customer support and service agreements that are going to make you work hard to deliver what they ask. You’re also going to need to be heavily involved in sales cycles and support, which will start costing you many, many hours just…