I'll give you my take on this as a Java programmer of 10+ years experience.
The problem you will have with Java in a startup is that Java's frameworks are by and large built for "enterprises". I put that in quotes because in one sense it doesn't really mean anything. In another, it's more of an idea of what governments and large companies want.
Even when using Spring (which, in my opinion, is a "must have" for pretty much any Java Web project), the amount of boilerplate required to set up an endpoint, map form data to objects, map those "presentation objects" into "business objects", map those "business objects" into a DAO (data access object), etc is pure torture.
Now this isn't completely the fault of the language. The language certainly doesn't help matters by being statically typed. This just doesn't gel well with how "fast" Web development works. Compare that to PHP (as just one example), where form data is just a map (hash or simply "array" in PHP parlance). Add a new field? Not a problem.
Add a new field in Java and you'll be making class modifications in about eight different places.
The bigger part of the problem is not the static typing though, it's the philosophies that dominate the Java landscape. There is a joke about the Java programmer's response to any problem is "just add one more layer and it'll be OK". It's funny because it's true.
It's fair to say that Fowleresque division of responsibility based layering is pervasive.
Of course the dynamic languages can have other problems (eg no error when misspelling a form field) and Java's static typing has, in my opinion, made Java's IDEs the best of any language or platform bar none (IntelliJ in particular). IDEs seem less able with dynamic languages because it's much harder (if not downright impossible) for an IDE to, for example, derive the members (let alone the types of those members, if that concept has meaning) on the fly.
Weirdly some in the PHP world have tried to mimic Java's deep layering with these horrific (imho) MVC frameworks that (again imho) simply combine the worst of both worlds. Dynamic class loading, bootstrappers, magic (and sometimes unpredictable) file loading, enforced directory structures and so on are just the wrong approach most of the time.
Java does have some benefits though. There are basically three tiers of languages in terms of performance (from best to worst):
1. C/C++/Assembler: the true compiled languages;
2. Java, C# (and the other .Net languages): the bytecode or virtual machine "semi-compiled" languages; and
3. Python, Perl, Ruby, PHP, etc: the scripted languages.
I qualify this by saying: most of the time (particularly for Web apps and especially if you don't have Google-level scale problems) performance doesn't matter. What matters is productivity and productivity is similarly tiered from worst to best in that above list. Hardware is now so cheap that it's far more effective to throw hardware at most problems than it is to write things in C. This is almost universally true when it comes to the Web where network latency and the speed at which the browser can process your page (and Javascript) will dwarf serverside performance most of the time.
I did a bunch of the Facebook puzzles awhile ago. The breathazlyer one was quite interesting. I started doing that in Python but it is somewhat difficult to get a solution to pass in that. Java is an order of magnitude faster. C is one or more orders of magnitude faster yet again.
So back to your original question (now that my answer has some context): I would say that using Java may make sense if everyone knows it and is comfortable with it. If not you will find many speed barriers along the way of learning a new language that may just slow you down or (worse) may create huge problems (eg security issues you weren't aware of because something didn't work the way you simply assumed that it did).
None of these problems are insurmountable but the way I figure it is this: when writing something new, chances are that within a year you'll be best off throwing it out and starting again anyway as the problem changes, you identify your bottlenecks (often not what you thought they'd be to begin with) and your scale changes.
So don't try and find the perfect language or platform or framework. Write something now and don't try and solve every problem you may ever have today.