51–60 of 89 posts
Earlier quoted context omitted.
Would Lambda be an option? Not sure what the stack looks like here, but a quick Google suggests people have gotten Cypress to run under Lambda, at least.
In my experience Lambda can have a pretty slow start when it's cold.
> completes all tests (no matter how many you've got) in less than a minute. > My thinking is to charge 2 cents per a test-minute Doesn't that reduce to 2c/run? That aside, my gut reaction was don't charge for time if your selling point is how much time you save. Charge for #tests, or something even better targetted at what makes it slow usually (I'm not familiar) - you charge some (<100) percentage of the reason tes…
However, based on everything I've read in this thread, I am leaning towards max parallelization based model, e.g. maximum 50, maximum 100, etc, and charge extra for tests that take longer than 1 minute.
This allows straightforward sales while providing most of the value to an avg. client.
It is an interesting idea, but your approach to pricing seems off: 100/day is not reasonable, at least not for small/medium teams. Trying to do arithmetic on run times and hourly rates to convince people it is reasonable is not only overly simplistic, but actively distracts from the conversation you really want to have - how does your solution solve their problem? People buy when they believe you can solve a problem…
If you know your market, you know your price. It’s very easy to A/B SaaS pricing. Does the proposed pricing sounds too high? To me, yes, but go out there and test it. Worse thing that can happen is nobody buys it.
Earlier quoted context omitted.
I am trying to figure out how to scale this to the point where unit economics makes the most sense. If I simply transferred current costs on to the end user, it would be too much. It would be USD 0.20 x number of tests x number of hours, e.g. a company that works 9 to 5 and has 100 tests, that is 0.20 100 8=USD 160 per day. However, changes the more clients you have. If I have 10 clients paying for 100 concurrent tes…
Why are you assuming that you will need to maintain a static infrastructure? Aren’t you able to (fairy quickly) autoscale up/down based on the size of some queue of test requests?
As I have only two clients at the moment who are helping to test this, keeping a hot pool allows me to focus on other feature development while sacrificing some of the costs.
Keeping a hot pool makes it a lot easier to immediately start test runs. Run tests, restart browser, clear sessions, return to the pool.
What are you replacing? What else could they pay for to achieve an equivalent speed up?
What are your enabling? How much time are you freeing up that can be used for other purposes?
What is the impact on time to market / release inter-arrival time?
What is the shadow backlog of tests that are not being run but now could be?
What is the impact on quality: for example time to fix (MTTR) or production problems prevented because of increased testing?
You are welcome to schedule a no cost office hours session to walk around your situation: https://www.skmurphy.com/blog/2016/08/01/skmurphy-office-hou...
I'm interested in trialing this service. Cypress takes forever. Feel free to reach out.
Ask your customers. Propose your current amount and be honest "do you think this is fair?" If they say yes great! If not, ask what they think is reasonable. Set yourself a lower limit to keep it profitable for yourself. "Sorry I can't price it below X". See how they respond.
I am trying to figure out how to scale this to the point where unit economics makes the most sense. If I simply transferred current costs on to the end user, it would be too much. It would be USD 0.20 x number of tests x number of hours, e.g. a company that works 9 to 5 and has 100 tests, that is 0.20 100 8=USD 160 per day. However, changes the more clients you have. If I have 10 clients paying for 100 concurrent tes…
Charge fairly. If you get more customers and improve efficiency again, be honest, "thanks to some improvements in scalability I'm able to reduce your costs to X".
You'll also build a long-term relationship by being fair and taking advantage of your customers.
Earlier quoted context omitted.
Okay — but running 60 copies of a small EC2 for 1 min is $0.009 to $0.04 Running 100x test suites would be $0.90 to $4, in server costs. Charging 25-100x the server costs is optimistic, at best.
Unfortunately, EC2 is not an option if we want to deliver 1 minute promise. I need VMs that I can flash start with all the software needed to run the tests immediately available. The best I was able to achieve is 20 seconds using my current provider. However, they charge in hour increments. That being said, I am constantly experimenting how to reduce costs. I think the final result is going to be a fairly complicated…
That would be $0.05 per core for an hour… or $0.05 for 60 copies for a minute. (Price for m5.metal on demand; a1.metal is half the price; m6a.metal is around $0.04)
I think the idea is cool — and wish you the best of luck.
I’m just trying to share why as a senior SDE your product doesn’t make sense to me: 100 integration tests a day is a fairly substantial team… and you’re charging a lot for it, at $2-3k per month.
You are thinking too much. Just tell them $5K/month. If they balk re-think your product.
> existing customers are series A and series B companies
With 2 VC funded customers, try getting a referral for cloud credits. If you cant get $10K credits try to find a business-side co-founder.
Finally, would you be interested in some sort of experimental "guaranteed minimum activity" swap contract? No cost to you, but you may need to provide analytics. Email my puppet email which i dont check often enough
> completes all tests (no matter how many you've got) in less than a minute. > My thinking is to charge 2 cents per a test-minute Doesn't that reduce to 2c/run? That aside, my gut reaction was don't charge for time if your selling point is how much time you save. Charge for #tests, or something even better targetted at what makes it slow usually (I'm not familiar) - you charge some (<100) percentage of the reason tes…
test minute = test * minute, i.e. 50 tests = USD 1 However, based on everything I've read in this thread, I am leaning towards max parallelization based model, e.g. maximum 50, maximum 100, etc, and charge extra for tests that take longer than 1 minute. This allows straightforward sales while providing most of the value to an avg. client.
(But also now there are 'tests that take longer than 1 minute'? It's gone from wow to not really clear what you're offering beyond managed test-running (that's otherwise equivalent to self-hosted) to be honest.)
Edit: do you mean they're parallelised, so they all complete in however long the longest one takes (which in OP is <1m but in comments might be more) and you'll charge for #concurrent jobs?