the toplevel links (implementation / deployment / configuration) don't work for me, they go to say /keratin/authn-server/docs/config.md which is a 404 presumably instead of /keratin/authn-server/blob/master/docs/config.md
Fixed, thanks!
Show HN: Keratin AuthN – Accounts and Auth Microservice in Go
41–50 of 57 posts
Re: Show HN: Keratin AuthN – Accounts and Auth Microservice in Go
#42the toplevel links (implementation / deployment / configuration) don't work for me, they go to say /keratin/authn-server/docs/config.md which is a 404 presumably instead of /keratin/authn-server/blob/master/docs/config.md
Fixed, thanks!
no problem, btw, the first time I did make test I got this, but after that I was not able to repro it again and make test always worked, pasting here in case you are interested
Creating authnserver_server_1 ... done
TEST_REDIS_URL=redis://127.0.0.1:8701/12 \
TEST_MYSQL_URL=mysql://root@127.0.0.1:8702/authnservertest \
go test ./data/... ./models/... ./tokens/... ./ops/... ./config/... ./lib/... ./api/... ./services/... .
[mysql] 2017/11/08 13:27:23 packets.go:33: unexpected EOF
[mysql] 2017/11/08 13:27:23 packets.go:33: unexpected EOF
[mysql] 2017/11/08 13:27:23 packets.go:33: unexpected EOF
--- FAIL: TestAccountStore (0.01s)
--- FAIL: TestAccountStore/MySQL (0.00s)
Error Trace: account_store_test.go:43
Error: Received unexpected error driver: bad connection
Connect
github.com/keratin/authn-server/data/mysql.ensureDB
/home/luser/go/external/src/github.com/keratin/authn-server/data/mysql/db.go:71
github.com/keratin/authn-server/data/mysql.TestDB
/home/luser/go/external/src/github.com/keratin/authn-server/data/mysql/db.go:27
github.com/keratin/authn-server/data_test.TestAccountStore.func3
/home/luser/go/external/src/github.com/keratin/authn-server/data/account_store_test.go:42
testing.tRunner
/usr/local/stow/go-1.9.0/go/src/testing/testing.go:746
runtime.goexit
/usr/local/stow/go-1.9.0/go/src/runtime/asm_amd64.s:2337
ensureDB
github.com/keratin/authn-server/data/mysql.TestDB
/home/luser/go/external/src/github.com/keratin/authn-server/data/mysql/db.go:29
github.com/keratin/authn-server/data_test.TestAccountStore.func3
/home/luser/go/external/src/github.com/keratin/authn-server/data/account_store_test.go:42
testing.tRunner
/usr/local/stow/go-1.9.0/go/src/testing/testing.go:746
runtime.goexit
/usr/local/stow/go-1.9.0/go/src/runtime/asm_amd64.s:2337
FAIL
FAIL github.com/keratin/authn-server/data 0.040s
? github.com/keratin/authn-server/data/mock [no test files]
? github.com/keratin/authn-server/data/mysql [no test files]
ok github.com/keratin/authn-server/data/redis 0.771s
? github.com/keratin/authn-server/data/sqlite3 [no test files]Re: Show HN: Keratin AuthN – Accounts and Auth Microservice in Go
#43Earlier quoted context omitted.
Sort of agree, that's a very noisy and broad statement. I'd argue that the underlying I/O and event loop implementation matters more. At the end of the day it's all about highly available systems. Am I wrong?
Microservices need to be divided up very carefully, with full knowledge of the latency trade-off of moving functionality across a network gap. A same-datacenter round-trip takes 5,000 times longer than a main memory access, and 1,000,000 times longer than an L1 cache access. There's no silver bullet in performance or scalability. Each solution will require trading off advantages that matter less to achieve business g…
The gain is decoupling deploy and implementation, not latency performance (or network reliability).
Re: Show HN: Keratin AuthN – Accounts and Auth Microservice in Go
#44Earlier quoted context omitted.
It looks like the /stats endpoint consults the Redis database and returns processed data. If we're aggregating metrics externally, it would be more idiomatic for this to be totally stateless and just expose hot metrics - it's the aggregator's job to store and process the data. This can be quite painless using something like the Go Prometheus library. Being tied to Redis is also not great. For instance, it looks like…
That all sounds like a direction I'd happily consider: * Google Cloud Storage implementations for data interfaces * Metrics interface with a Prometheus implementation (STATSD to follow) * Redis-backed HLL metrics are optional Feel free to open issues in the tracker if you reach that point!
Good stuff.
I have one other bit of feedback on the architecture, and I realise that this one might be too different from where AuthN is currently. I see that there are a small number of API endpoints that you've marked "public" that users are meant to be able to reach. In practice, the user's POSTs would always be terminated upstream from the AuthN server itself, either at a gateway or by just relaying the requests through from the front-end servers themselves. A service that's part public/part private is much more perilous than a service that is one or the other. All things considered, these public endpoints seem like a very minor convenience to the integrator. I would just make the AuthN API private, and ask users to implement the public endpoints themselves by making requests to the AuthN API.
If you DID make all endpoints private, you're no longer tied to HTTP, and you have a nice opportunity to use GRPC instead. I've recently started rolling it out in my own services, and it's unexpectedly great considering the track-record of similar ideas. It's very performant, pleasant to work with, has a mature ecosystem of surrounding tools, and you get client API implementations in a wide range of languages for "free" (or at least at a cut price).
Re: Show HN: Keratin AuthN – Accounts and Auth Microservice in Go
#45Earlier quoted context omitted.
It looks like the /stats endpoint consults the Redis database and returns processed data. If we're aggregating metrics externally, it would be more idiomatic for this to be totally stateless and just expose hot metrics - it's the aggregator's job to store and process the data. This can be quite painless using something like the Go Prometheus library. Being tied to Redis is also not great. For instance, it looks like…
That all sounds like a direction I'd happily consider: * Google Cloud Storage implementations for data interfaces * Metrics interface with a Prometheus implementation (STATSD to follow) * Redis-backed HLL metrics are optional Feel free to open issues in the tracker if you reach that point!
[deleted]
Re: Show HN: Keratin AuthN – Accounts and Auth Microservice in Go
#46"Microservices perform better, especially when written in Go." Nonsense.
This is true for extremely generous definitions of "perform".
Microservices can lead to better performance by making for smaller, more clearly defined codebases, fewer unnecessary imports, and so on. They can also be easy to scale, because you can scale specifically that one component (e.g. identity management) by moving it to a separate database server.
We use microservices, and we have probably close to 2TB of MySQL database, but because we have our services and their database schemas cleanly separated, we can break that up into a set of databases which all fit into memory and one database which, being basically append-only, doesn't need to access historical data and so doesn't need to fit entirely in RAM.
It also lets us easily look at our cluster stats and see where bottlenecks are, by easily seeing which servers or services are under load.
We do pay a penalty for this; layers of indirection, network latency, protocol overhead, serialization/deserialization, and so on, but designing our systems like that from the start lets us tackle those problems at the start and account for them in our design.
Re: Show HN: Keratin AuthN – Accounts and Auth Microservice in Go
#47Earlier quoted context omitted.
That all sounds like a direction I'd happily consider: * Google Cloud Storage implementations for data interfaces * Metrics interface with a Prometheus implementation (STATSD to follow) * Redis-backed HLL metrics are optional Feel free to open issues in the tracker if you reach that point!
Good stuff. I have one other bit of feedback on the architecture, and I realise that this one might be too different from where AuthN is currently. I see that there are a small number of API endpoints that you've marked "public" that users are meant to be able to reach. In practice, the user's POSTs would always be terminated upstream from the AuthN server itself, either at a gateway or by just relaying the requests…
My intention for the user-facing endpoints is that the host app will never need to see or accidentally log a user's password. It's a pattern inspired by credit card vaults.
Could you still achieve your deployment goals with a Gateway/WAF setup that proxies the user-facing endpoints? The only issue I'm presently aware of with this setup is https://github.com/keratin/authn-server/issues/8.
Re: Show HN: Keratin AuthN – Accounts and Auth Microservice in Go
#48Earlier quoted context omitted.
Good stuff. I have one other bit of feedback on the architecture, and I realise that this one might be too different from where AuthN is currently. I see that there are a small number of API endpoints that you've marked "public" that users are meant to be able to reach. In practice, the user's POSTs would always be terminated upstream from the AuthN server itself, either at a gateway or by just relaying the requests…
My intention for the user-facing endpoints is that the host app will never need to see or accidentally log a user's password. It's a pattern inspired by credit card vaults. Could you still achieve your deployment goals with a Gateway/WAF setup that proxies the user-facing endpoints? The only issue I'm presently aware of with this setup is https://github.com/keratin/authn-server/issues/8 .
Sure - in my case this would be a load balancer on Google Cloud configured through a Kubernetes ingress. All the public paths available on the Auth microservice would have to be listed explicitly. You have to be very sure in this scenario that only the public URLs are reachable.
I see the motivation for sending user passwords straight to AuthN, but I do wonder if on balance the dangers of correctly configuring a shared public/private service don't outweigh the dangers of relaying the password through your front-end (which has to be able to access private endpoints on the AuthN service anyway). If you really didn't want your front-end to touch passwords, you could have a tiny sidecar service that only exposes the public endpoints and speaks GRPC to AuthN.
Anyway, this is well into debatable "matter of opinion" territory, and I don't want to waste your time. Thanks for publishing AuthN - I'll keep a close watch as you progress.
Re: Show HN: Keratin AuthN – Accounts and Auth Microservice in Go
#49Earlier quoted context omitted.
Sort of agree, that's a very noisy and broad statement. I'd argue that the underlying I/O and event loop implementation matters more. At the end of the day it's all about highly available systems. Am I wrong?
Microservices need to be divided up very carefully, with full knowledge of the latency trade-off of moving functionality across a network gap. A same-datacenter round-trip takes 5,000 times longer than a main memory access, and 1,000,000 times longer than an L1 cache access. There's no silver bullet in performance or scalability. Each solution will require trading off advantages that matter less to achieve business g…
> A same-datacenter round-trip takes 5,000 times longer than a main memory access, and 1,000,000 times longer than an L1 cache access.
This is a ridiculous comparison though. When have you ever seen a situation where a micro service was accessed over the network to fetch something that would have been in L1 cache? And yes, having a value "in memory" is faster, but in most cases these days things aren't "in memory", they're "in memory in memcached, somewhere in the cluster, probably not on the local server so you have to go over the network anyway".
Just out of curiosity though, I tested on our network. Our RTT between servers is less than half a millisecond. Our fastest service can get a cached reply back to the calling service in 35ms, whereas many calls are >100ms and some are much longer (e.g. initial login calls). It would take a lot of external login calls to noticeably (to the user) increase even the fastest service.
We also use Redis and memcached, and spread those across other services. Excluding specifically HTTP overhead and just looking at network latency, a few memcached calls which hit non-local servers in the cluster cost us as much in latency as one HTTP call would.
Re: Show HN: Keratin AuthN – Accounts and Auth Microservice in Go
#50Earlier quoted context omitted.
Fixed, thanks!
no problem, btw, the first time I did make test I got this, but after that I was not able to repro it again and make test always worked, pasting here in case you are interested Creating authnserver_server_1 ... done TEST_REDIS_URL=redis://127.0.0.1:8701/12 \ TEST_MYSQL_URL=mysql://root@127.0.0.1:8702/authnservertest \ go test ./data/... ./models/... ./tokens/... ./ops/... ./config/... ./lib/... ./api/... ./services/.…
Thanks for the report! I believe I've tracked this down to an initialization routine that MySQL goes through on the first boot. It happens after docker-compose unblocks. Likely a wontfix. :/