Live data from Hacker News

Building an API in 60 seconds, without any server setup

api.blockspring.com

51–60 of 72 posts

Re: Building an API in 60 seconds, without any server setup

#51
post #41

I'd be more interested in seeing an infrastructure for scraping where the API functions are fixed, but the actual scraping functions are dynamically loaded so that the API user doesn't have to maintain it or re-pull/re-fork/re-compile when the website design changes. Bonus points if one can make an ORM out of it, e.g. for article in get_api('reddit.com').todayilearned.filter('new').limit(100): ... do something ... Wh…

I created http://scrape.ly with some of those points described. It's a bit of journey of getting there and I'd love to know in more detail about what you have in mind.

The API generation with Blockspring and Kimonolab is very nicely done but I like to focus solely on the web scraping more, as it represents very difficult set of challenges.

Re: Building an API in 60 seconds, without any server setup

#52

I'm not sure I understand what this is. Free hosted PaaS for single files, essentially? I know nothing about where the code is running? Is it performant? Is it reliable? How are they covering server costs / at what point would I be asked to pay, and how much?

Hey superuser, Great questions. Let me answer them one at a time! Where is the code running? Every script run happens in its own sandbox on our machines. We use docker, it's awesome. People are asking for dedicated machines specifically for their company, and others are asking to hook into their own machines. Once we figure out how people/companies want to use the site, we'll put up some options for where code will b…

You may be interested in my Python-on-ZeroVM-On-Docker Dockerfile[1]. This adds the security and isolation of ZeroVM on top of the connivence of Docker.

Note that ZeroVM isn't an x86 VM, so you need a custom Python (which that Dockerfile downloads). There are also no network sockets, so some things are difficult to make work, but you can work around that by using network code in the Docker container, and riskier code in ZeroVM.

I'd be pretty confident in that security model.

However, it's six months old now, and likely to need some updating. ZeroVM was changing pretty quickly when I was working on it.

[1] https://github.com/nlothian/Acuitra/tree/master/infrastructu...

Re: Building an API in 60 seconds, without any server setup

#54

Earlier quoted context omitted.

Another vote for Haskell.

Language support is super easy to add. Any list of libraries you'd really want with Haskell?

Prelude's support libraries plus at least the curl package: https://hackage.haskell.org/package/curl

Re: Building an API in 60 seconds, without any server setup

#56
I really don't understand what's happening here. I think we use the term "API" and "server" differently. When I think of an API, I think of a library or something that I can link to. For example the POSIX API, or the P-Threads API. These are available in one way or anther from any language.

This seems to think very differently about APIs. As if APIs are somehow tied to a specific language? When they use the term API, do they maybe mean library?

I think that under the hood, the idea is that, if you want to do something, in another language (rather than just writing it in the language) you somehow generate a stub in both languages and then proxy through a (network?) connection?

To make this more complex, it seems that the proxy connections are not just on the local machine, but over the internet to another machine. Presumably you have to pay for this?

Am I right? This seems like a lot of effort to go through just to avoid figuring out how to do something directly. With really sucky performance implications. I guess it depends on what you're doing. Most languages support a foreign interface to C, so maybe that is a more sensible way to do this rather than setting up "servers" and tunnelling commands and data back and forth.

EDIT: I just had another look at the website. So, when they use the term API - it would seem that they mean some sort of an HTTP based web interface? The idea is that you have some kind of script that does something for you and you can query it from you local script. I still don't understand why you'd want to do this over running it locally?

Re: Building an API in 60 seconds, without any server setup

#57

I really don't understand what's happening here. I think we use the term "API" and "server" differently. When I think of an API, I think of a library or something that I can link to. For example the POSIX API, or the P-Threads API. These are available in one way or anther from any language. This seems to think very differently about APIs. As if APIs are somehow tied to a specific language? When they use the term API,…

Hey Deadgrey,

You're totally right. I'm not sure API is the best word for it, but haven't found a better one. Would love your thoughts!

I've heard a bunch of crazy usecases from HN over the past day but my original one was to simply take some of the python and R scripts I had, and without much work (a few clicks actually) be able to execute them from my frontnd js, or from rails workers. It worked really well so we decided to see if others might find it useful too.

It's badass to see what others are coming up with in the API Library, hoping that can be a valuable page for everyone. And I imagine when we figure out fair pricing, we can actually make this significantly cheaper than a dedicated server for these processes.

Please, continue to send feedback like this. You're awesome.

Re: Building an API in 60 seconds, without any server setup

#58

I really don't understand what's happening here. I think we use the term "API" and "server" differently. When I think of an API, I think of a library or something that I can link to. For example the POSIX API, or the P-Threads API. These are available in one way or anther from any language. This seems to think very differently about APIs. As if APIs are somehow tied to a specific language? When they use the term API,…

I think the idea is that if you have a useful script, you can set it up here just once and it's automatically wrapped with an HTTP-based interface so you can use it like an API. Now you can use that script from any Internet-connected device, with basically zero setup on each device. I'm not sure if I can think of something I'd want to use it for, but this is an interesting idea and a great proof-of-concept implementation.

I think perhaps the most interesting part of this is that they're apparently spinning up/down Python (and other languages) sandbox environments on the fly, nearly instantaneously.

Re: Building an API in 60 seconds, without any server setup

#59
post #28

So my buddy Sean (@tilmonedwards) just built something pretty similar as a command-line app, just for yucks: https://github.com/seanedwards/svcizer

"The request body will be passed to stdin." "STDOUT and STDERR from cmd will be dumped to the response body." "HTTP headers and query parameters will be passed as environment variables." Uhhh, your friend just reinvented CGI: https://en.wikipedia.org/wiki/Common_Gateway_Interface

Yeah--that's neat, to me. :-) I've written against CGI before. The kicker to me is that it converges back on what we'd already done. History is fun.

Re: Building an API in 60 seconds, without any server setup

#60

Earlier quoted context omitted.

Another vote for Haskell.

Language support is super easy to add. Any list of libraries you'd really want with Haskell?

Some random highly used libraries:

containers, bytestring, mtl, text, transformers, time, array, aeson, split, blaze-builder, blaze-html lens, safe, resourcet

I'm sure aeson examples would be very popular. mtl/transformers examples for those who are getting to them in the learning process.

Here is a complete list of the top reverse dependencies for libraries on hackage:

https://docs.google.com/spreadsheets/d/1o7K_tEDIULPHNv8RpP0E...

Post reply on HN