Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Algorithms as a Service

mikesingleton.net

Post reply on HN
Print view View original

21–30 of 30 posts

Re: Algorithms as a Service

#21

Post by keefe »

mattmcknight wrote: post #7

This is where I usually draw the line at the "everything is a service" world view. It is much better to provide this as a library than as a service, particularly algorithms that run across large data sets. However, the exception I see to this is a service that applies the algorithm across a large dataset that is owned by the service. An example of this is geocoding, where you probably don't want to store the addresse…

I think the other area where AAS makes sense is in distributed algorithms, there is a genuinely large amount of work in writing parallel versions of algorithms and they also require clusters.

Re: Algorithms as a Service

#22

Post by akronim »

a-priori wrote: post #10
pierrefar wrote: post #8

Earlier quoted context omitted.

I was thinking that as I was typing my comment. Another solution, which S3 uses, is to ship hard disks by courrier. I guess the real metric here is cost per GB transfered in a unit time, say $/GB-hr.

At what point does it become ridiculous to move the data, which may be measured in TB or PB, when the algorithm itself would be measured in KB or MB?

In clusters working on large amounts of in memory data, the approach is often to load the data, then move the code (e.g. a java class implementing some data procesing interface) to the data as required, rather than move the data to the code.

Re: Algorithms as a Service

#23

Post by akronim »

mattmcknight wrote: post #7

This is where I usually draw the line at the "everything is a service" world view. It is much better to provide this as a library than as a service, particularly algorithms that run across large data sets. However, the exception I see to this is a service that applies the algorithm across a large dataset that is owned by the service. An example of this is geocoding, where you probably don't want to store the addresse…

At least if it's a library you're not really increasing your chance of failure. Whereas as you add external dependencies, the odds of one of them being down and taking your service with it increases. So you had better thinking about how you handle failure situations gracefully and so on, so the service you call had better be doing something special (e.g. giving you access to some huge dataset) to make it worth that additional complexity.

Re: Algorithms as a Service

#24

Post by kwamenum86 »

This would work very well for memory/computation intense algos with relatively small input and output. For example, a semantic analysis engine that accepts a URL and returns the extracted meaning from a web page. Input several bytes. Output less than a k. I think this is totally feasible and worth it as long as customers can monetize the output. To me a service like OpenCalais is AaaS. Sure they have software but the hard part of what they do is semantic analysis.

Re: Algorithms as a Service

#25

Post by topbanana »

This is effectively the same proposition that web services promised us ten years ago. For the most part we still develop closely-coupled software, and I don't see that changing in most cases

Re: Algorithms as a Service

#26

Post by robryan »

a-priori wrote: post #10
pierrefar wrote: post #8

Earlier quoted context omitted.

I was thinking that as I was typing my comment. Another solution, which S3 uses, is to ship hard disks by courrier. I guess the real metric here is cost per GB transfered in a unit time, say $/GB-hr.

At what point does it become ridiculous to move the data, which may be measured in TB or PB, when the algorithm itself would be measured in KB or MB?

There is always the stuff that goes the other way though like how Seti@Home does FFT's which is computationally expensive and benefits from a distributed system but the file size is quiet small.

Re: Algorithms as a Service

#27

Post by jacabado »

keefe wrote: post #21
mattmcknight wrote: post #7

This is where I usually draw the line at the "everything is a service" world view. It is much better to provide this as a library than as a service, particularly algorithms that run across large data sets. However, the exception I see to this is a service that applies the algorithm across a large dataset that is owned by the service. An example of this is geocoding, where you probably don't want to store the addresse…

I think the other area where AAS makes sense is in distributed algorithms, there is a genuinely large amount of work in writing parallel versions of algorithms and they also require clusters.

Google Wave?

Re: Algorithms as a Service

#28

Post by a-priori »

robryan wrote: post #26
a-priori wrote: post #10

Earlier quoted context omitted.

At what point does it become ridiculous to move the data, which may be measured in TB or PB, when the algorithm itself would be measured in KB or MB?

There is always the stuff that goes the other way though like how Seti@Home does FFT's which is computationally expensive and benefits from a distributed system but the file size is quiet small.

Yes, BOINC projects are cases where it is not ridiculous to move the data, because it is computation power that is the scare resource and the work units are typically only in the hundreds of kilobytes to single-digit megabytes.

Re: Algorithms as a Service

#29

Post by keefe »

jacabado wrote: post #27
keefe wrote: post #21

Earlier quoted context omitted.

I think the other area where AAS makes sense is in distributed algorithms, there is a genuinely large amount of work in writing parallel versions of algorithms and they also require clusters.

Google Wave?

do you have any reference on how distributed algos and wave mix? I'm a little behind the curve in understanding wave but I can't see the link here.

Re: Algorithms as a Service

#30

Post by jacabado »

keefe wrote: post #29
jacabado wrote: post #27

Earlier quoted context omitted.

Google Wave?

do you have any reference on how distributed algos and wave mix? I'm a little behind the curve in understanding wave but I can't see the link here.

Google Wave as a platform is a huge step in distributed computing abstraction. If you ever try/tried to implement any app on top of distributed algo you will value the wave possibilities!
Post reply on HN