Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Three Months to Scale NewsBlur

blog.newsblur.com

Post reply on HN
Print view View original

91–100 of 146 posts

Re: Three Months to Scale NewsBlur

#91

Post by bsg75 »

duncan_bayne wrote: post #54
conesus wrote: post #20

Earlier quoted context omitted.

This is something I'll be working on soon enough. There are many, many dependencies (not only for the web app, but there are three DBs that you have to have installed as prerequisites - mongo, postgres/mysql, and redis). The real problem with setting up your own instance of NewsBlur is that you'll have to do your own feed fetching. This is effectively what you're paying for when you pay for NewsBlur. Let me break it…

Three DBs? I don't suppose you have a blog post or something that explains your architecture decisions? Not criticising - I'm sure you had good reasons - but I'm rather curious as to what they were :)

I too am interested, esp what goes into Mongo and what into Postgres (assuming Redis is a form of cache?)

I suppose I could also look at the source ;)

Re: Three Months to Scale NewsBlur

#92

Post by randall »

conesus wrote: post #53
UberMouse wrote: post #48

Earlier quoted context omitted.

I wasn't aware you could do this. Locking it by default seems like it would make more sense. Also while I've got you, two issues I've noticed. 1. When I click "All Stories" to refresh my feeds it defaults to listing everything, even though I have it set so I only view one item at a time. As soon as I click the button/arrow to go to the next unread item it shows the first item as it should and everything works fine af…

1. Intentional. Otherwise you'd stare at a blank screen until you click a story. 2. That's an unfortunate bug, but one that will probably get sussed out when I work on scaling over the next month. Dev is largely just a re-skin. They share a backend.

up / down arrows to go to next story sucks too, fwiw.

Re: Three Months to Scale NewsBlur

#93

Post by jzelinskie »

tmzt wrote: post #84
btipling wrote: post #76

Earlier quoted context omitted.

> I am not a web developer so that can partially explain my struggle with fabric, Django and multitude of webdev and devops modules that you use. GitHub is not an app store. You can't expect to "install" a software as a service's code base. Open sourcing a project is opening it up to engineers, providing a self-hosted option on the other hand should be an easy to install alternative that probably requires another ful…

Seems like there's a need for a cloud infrastructure/language independent packaging format, to build a "github of installable applications" Maybe container images that can also be used as bootable images?

Something like JuJu?

Re: Three Months to Scale NewsBlur

#94

Post by glenra »

korg250 wrote: post #74
lazugod wrote: post #41

Earlier quoted context omitted.

I briefly tried both. * Feedly has a simple interface which is much more like Google Reader. But it depends on a plugin (or an app) rather than being an allwhere accessible website. Their Kindle Fire app seemed to load content extremely slowly, and its interface wasn't a simple list. And it's free, so I can't be a customer and I don't know how they earn money. * NewsBlur is complicated and unresponsive. There are a l…

"and I don't understand why an arrow follows the mouse." Really? It is a feature for better reading, so you can can easily see which line you are.

You mean so as to remember where you were last reading are on the page? But if you use the mouse to switch browser tabs, that arrow moves up to the top of the page, and if you use any of the controls at the bottom, it moves down. I guess one could very carefully leave the window at a specific place so as to preserve that arrow's position, but it's not an obvious thing to do. How are you using this feature that makes it an improvement over just letting your pointer itself mark the current reading position?

Re: Three Months to Scale NewsBlur

#96

Post by glenra »

bsg75 wrote: post #88
korg250 wrote: post #74

Earlier quoted context omitted.

"and I don't understand why an arrow follows the mouse." Really? It is a feature for better reading, so you can can easily see which line you are.

Make sense now that someone points it out! For those who mouse click to mark reading point.

Aha. It somehow hadn't occurred to me to click on that triangle. I'm still not sure it's a useful-to-me feature, but it at least makes a little more sense now. It's less annoying once locked down, anyway. Maybe it should default to being locked down somewhere and only unlock the first time you click on it, rather than default to floating around freely.

Re: Three Months to Scale NewsBlur

#97

Post by BoyWizard »

duncan_bayne wrote: post #54
conesus wrote: post #20

Earlier quoted context omitted.

This is something I'll be working on soon enough. There are many, many dependencies (not only for the web app, but there are three DBs that you have to have installed as prerequisites - mongo, postgres/mysql, and redis). The real problem with setting up your own instance of NewsBlur is that you'll have to do your own feed fetching. This is effectively what you're paying for when you pay for NewsBlur. Let me break it…

Three DBs? I don't suppose you have a blog post or something that explains your architecture decisions? Not criticising - I'm sure you had good reasons - but I'm rather curious as to what they were :)

Based on the Github page, I would imagine Redis as a cache, MongoDB to store the 'non-relational data' (quoted from the GH page - I'm guessing the actual articles and such) and Postgres for the other application stuff - accounts, preferences, subs, etc.

Re: Three Months to Scale NewsBlur

#99

Post by laureny »

conesus wrote: post #20
mynegation wrote: post #16

Earlier quoted context omitted.

I tried to install NewsBlur, but instructions were not very helpful. I tried to get as far as I can, editing fabfile and installing gajillions of new python modules, but still failed. Don't get me wrong, it is wonderful and very commendable of you to open source NewsBlur! Nor do I feel entitled to the source code or free ride (I will probably end up paying for the hosted solution). Just providing some feedback, given…

This is something I'll be working on soon enough. There are many, many dependencies (not only for the web app, but there are three DBs that you have to have installed as prerequisites - mongo, postgres/mysql, and redis). The real problem with setting up your own instance of NewsBlur is that you'll have to do your own feed fetching. This is effectively what you're paying for when you pay for NewsBlur. Let me break it…

> So you have to run 10 processes in parallel, and that's already beyond the capabilities of most single machines.

Er... really? My laptop is three years old and I run 150 threads on it every time I launch the server I'm working on (dozens of times a day). No problems at all.

It's in Java, in case it matters.

Re: Three Months to Scale NewsBlur

#100

Post by ceol »

BoyWizard wrote: post #97
duncan_bayne wrote: post #54

Earlier quoted context omitted.

Three DBs? I don't suppose you have a blog post or something that explains your architecture decisions? Not criticising - I'm sure you had good reasons - but I'm rather curious as to what they were :)

Based on the Github page, I would imagine Redis as a cache, MongoDB to store the 'non-relational data' (quoted from the GH page - I'm guessing the actual articles and such) and Postgres for the other application stuff - accounts, preferences, subs, etc.

https://github.com/samuelclay/NewsBlur/blob/master/apps/rss_...

It looks like MongoDB is being used for fetch/push history, feed icon data, and article data. It's also being used for some other stuff: https://github.com/search?l=Python&q=%22mongo.Document%2...

Post reply on HN