Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Improve AI – Optimize Your App Easily with AI

improve.ai

Post reply on HN
Print view View original

1–10 of 20 posts

Re: Improve AI – Optimize Your App Easily with AI

#6

Post by ntonozzi »

This looks awesome, I'm excited to try it out.

It'd be cool to be able to dynamically deploy new configuration options without deploying new device code, or remove bad configurations. Looks like that could be possible but it would depend on your app to supply the configuration.

Is there any support for continuous variables (e.g. find me the best price between $10 and $20)?

Re: Improve AI – Optimize Your App Easily with AI

#7

Post by orasis »

ntonozzi wrote: post #6

This looks awesome, I'm excited to try it out. It'd be cool to be able to dynamically deploy new configuration options without deploying new device code, or remove bad configurations. Looks like that could be possible but it would depend on your app to supply the configuration. Is there any support for continuous variables (e.g. find me the best price between $10 and $20)?

Thanks! For updating variants remotely you can query them from a database or use something like Firebase Remote Config. We've intentionally separated updating variants from the decision of which variant to choose so that decisions don't block on network requests.

The machine learning models actually perform wonderfully on continuous variables, but for this initial launch we're having developers specify the increments of those variables manually, so you'd put in [10, 11, 12, ...]. Soon we'll also likely provide a 'Variant Generator' class that automatically hill climbs to find the best value. If that's something you'd like to have let me know and we can make it a priority.

Re: Improve AI – Optimize Your App Easily with AI

#8

Post by gavinray »

orasis wrote: post #2

Hi there, Justin here, the creator of Improve AI. I would love any feedback and feel free to ask me anything. Thanks!

Going to sound cynical/snarky below, promise I'm not. Also disclaimer, I know almost nothing about ML:

- The homepage makes a vague statement about "AI". This is a personal thing, but to me "AI" sets off warning bells. It feels like a vague term, compared to "ML".

- I couldn't find any FAQ or references on the site to technical architecture. What underlying framework/library is used for the models, and what kind of NN architecture?

- If I were going to do this myself, I would probably do something like the below. Why not do this?

  > Dump tabular data of feature/option/scores and model serialized as binary in SQLite DB
  > Use a tool to autogenerate a CRUD API over SQL DB, host that as model server
  > Use Litestream to replicate SQLite DB to S3
  > Pick one of the beginner-friendly ML toolkits like PyTorch/Fast.ai/TF, use tabular data API to train prediction/scoring model from historical data
  > Option to fetch the SQLite DB to local devices so that it's offline-capable
- For option-weighting, why use ML instead of simple regression? Facebook's time-series forecasting tools are based on regression analysis.

Also as a note, the quickstart guide redirects to iOS SDK Github

Re: Improve AI – Optimize Your App Easily with AI

#9

Post by orasis »

gavinray wrote: post #8
orasis wrote: post #2

Hi there, Justin here, the creator of Improve AI. I would love any feedback and feel free to ask me anything. Thanks!

Going to sound cynical/snarky below, promise I'm not. Also disclaimer, I know almost nothing about ML: - The homepage makes a vague statement about "AI". This is a personal thing, but to me "AI" sets off warning bells. It feels like a vague term, compared to "ML". - I couldn't find any FAQ or references on the site to technical architecture. What underlying framework/library is used for the models, and what kind of N…

I hear you on the AI vs ML thing. I doubt the debate will ever be resolved, but the domain name is .ai so I went with that.

The reason to use a full machine learning model instead of a simple statistical model is three-fold:

1) The framework supports any arbitrary JSON encodable data structure as a variant, this includes nested dictionaries, lists, etc. A full ML model allows capturing the entire complexity of the data stored in the variant - numbers, strings, booleans, etc and those can even be changed and the model will still be able to predict performance.

2) A full ML model allows generalization across complex variants. So if later on you introduce new variants, it may already be imply some things about how that variant will perform. For the most part simple statistical models must learn each variant from scratch.

3) The models don't just seek to make decisions that are a global optimum, like you would do with A/B testing. They are seeking to find a contextual optimum given the current conditions. This type of modeling also requires the generalization that ML gives you.

Post reply on HN