61–70 of 109 posts
Mods, can we change "go" in the title to "golang"? That's the official name, and it makes it searchable by search engines?
I still see it written as "Go" most of the time. It's unfortunate because I've picked up the habit of searching with "golang".
Earlier quoted context omitted.
It's just pass through. The limit in this instance is probably packets per second of the legacy AWS network. Why is this difficult?
Read the analysis. First attempt was "just a pass through".
Does anybody here have any experience with Go's garbage collection pauses with large stack sizes? I've got a scala app that regularly consumes about 48G of ram, and I'm very happy with the response times during heavy loads like this, but the P99.5 is abysmal because of garbage collection. I've tried tuning it, but it doesn't seem like anything I do helps. I'll probably end up using an Azul JVM but I'm curious how oth…
When tuning a prod Scala deployment a while back, I encountered a nasty "pregnant pause" every once in a while similar to what you have experienced. So, below are what I used in annotated form and adapted for the deployment you describe (the max heap setting). Some of these may be completely obvious to you (or others reading), yet are included for completeness.
-server
This one should be obvious :-)
-Xmx54G
Memory is cheap, so give the JVM 54 gig so that GC
isn't forced to run when your system is in the
steady-state of 48G heap utilization.
-XX:PermSize=128m -XX:MaxPermSize=1024M
Ditto on the cheapness of memory.
-Xss1M
A stack size of 1 meg seems a bit much, but does
allow for recursive algorithms to operate with
impunity.
-XX:ReservedCodeCacheSize=128m
This one was needed for Scala. It likely should
be specified with a high value since it limits the
JIT's code cache.
-XX:+DoEscapeAnalysis
A nice way to releave some heap pressure[1].
-XX:+UseCodeCacheFlushing
Should the ReservedCodeCacheSize be exceeded, this
lets the JIT continue to do its thing in an LRU
type of fashion (I believe).
-XX:+UseParallelGC
This one is the most impactful one of all. A lot
of people will say "use UseConcMarkSweepGC!" They
are wrong for high volume server deployments. The
concurrent mark and sweep algorithm caused massive
"pregnant pauses" in prod for me! The Parallel GC
algorithm performs much better under load and
doesn't cause the VM to sit-and-spin for 20+
seconds.
-XX:+UseCondCardMark
Another tweak which had a major performance boost
for me[2].
-XX:+UseNUMA
If your servers are NUMA[3] based, then this can
significantly increase performance[4] as well.
1 - http://www.ibm.com/developerworks/java/library/j-jtp09275/in...
2 - https://blogs.oracle.com/dave/entry/false_sharing_induced_by...
3 - https://en.wikipedia.org/wiki/Non-uniform_memory_access
4 - http://jose-manuel.me/2011/06/numa-bb/
EDIT: Inserted newlines to eliminate horizontal scrolling.
There are two other solutions that spring to mind, which might require quite a bit less code: 1) Take the original code, do the upload exactly in place in the original request (not even spawning a goroutine). However: protect the upload with a semaphore which only allows N-in-flight. My reasoning is, well, if the system operates with low latency when operating nominally, blocking the incoming request isn't too painfu…
Arguably, UploadToS3 should not be a method of *Payload. Suggestion: Make an S3-uploader package with _internal_ connection pooling, upload queueing and concurrency handling.
Nice features include:
> Retry Everything: All http requests and every part is retried on both uploads and downloads.
> Configurable conncurrency
> Uses an io.Writer (you could actually start posting to S3 before all of the data gets in on your side.)
etc.
Earlier quoted context omitted.
Go is 1-2 orders of magnitude faster than Ruby, and much easier to write concurrent code in. It makes perfect sense to me. What would you have recommended them, for a reasonably-high-performance server implementation? (Please don't say C.) https://benchmarksgame.alioth.debian.org/u64q/benchmark.php?...
> Go is 1-2 orders of magnitude faster than Ruby, and much easier to write concurrent code in. Than MRI Ruby you mean. The advantage wouldn't be as much if Ruby designers cared to add AOT compilation in the same vein as Dylan or Common Lisp to the canonical implementation.
It's interesting this came up today. I'm looking for a new language to migrate my flask/uwsgi web service to. I'm having a terrible time making it scale. Are there any tutorials/templates/best practices for writing a small web service in Golang?
The best advice I can give is to write a minimal web server using net/http[0], get a feel for how to write a RESTful HTTP API by hand, and then start experimenting with web frameworks like Gin[1], Martini[2], and Beego[3]. [0] https://golang.org/doc/articles/wiki/ [1] https://github.com/gin-gonic/gin [2] https://github.com/go-martini/martini [3] https://github.com/astaxie/beego
I loved it when I started, then really hated it when I needed to refactor, there is just too much magic and you pay a speed cost for that magic.
Does anybody here have any experience with Go's garbage collection pauses with large stack sizes? I've got a scala app that regularly consumes about 48G of ram, and I'm very happy with the response times during heavy loads like this, but the P99.5 is abysmal because of garbage collection. I've tried tuning it, but it doesn't seem like anything I do helps. I'll probably end up using an Azul JVM but I'm curious how oth…
this article discusses how they handled 69GB of heap and up to 6 seconds of pause times: http://blog.golang.org/qihoo the new garbage collector in 1.5 should improve things.
https://talks.golang.org/2015/state-of-go-may.slide#6 (Slides 6 to 11)