Google Analytics is associated with gleaning useful, actionable insights from your users' behavior on your web sites and in your apps, which was what the guy who sold you on the concept of a data lake was promising.
Who besides your hypothetical salesman is arguing that the raisin d’etre for data lakes is user web analytics?
In my experience people arguing for data lakes talk a lot about some unspecified future benefit. Companies typically want to learn things about their interactions with their customers that allow them to make more money, and thus GA or its equivalent represents the 80/20 — or more likely 80% of the benefit for 1% of the cost — solution.
Micro-services are just connected things which work together to accomplish something. But where are those connections described? In some tables somewhere. Maybe. Whereas if you write a single monolithic program its connections are described in code, preferably type-checked by a compiler. I think that gives you at least theoretically a better chance of understanding what are the things that connect, and how they conne…
There is such a language, and it is Erlang.
Literally this. Reading the article I kept thinking to myself "which is why Erlang/Elixir is great because it doesn't make you choose up front". It's wild that with how popular Elixir has gotten that it still isn't seen as a serious contender for many companies
I can guess that following pattern will emerge
"Develop like microservice but deploy like a monolith"
So a container will have group of services rather than hosting single service. Similar will be happen for databases where different databases will on same host.
i dont care about it not being great. its good enough as a language. but maven... but websphere... java is a hellish platform that ordinarily would not win from interpreted languages or those which focus on fast compilation. But it runs literally everywhere, including your toaster, but more importantly on mainframes which also run the real mvp aka Cobol. run once, run anywhere remains a killer feature no other platfo…
I think you are putting Java in the pillory today, for the worst aspects of its ecosystem a decade ago. Outside of perhaps stodgy banks, technical folks are not choosing to run their JVM projects today on Websphere. Gradle is pretty darn nice.
It's what my employer uses, they dont really have much choice because they're running on IBM hardware.
Spring, Hibernate and other "enterprise" monsters came from Java alone. FactoryFactory123.
Have you ever used CORBA and DCOM factories? Enterprise ORMs were created in Smalltalk, C++ and Objective-C, years before Java was created. Poet and Enterprise Object Framework were two well known ones. In fact J2EE was born of the ashes of a failed Objective-C project at Sun, Neo, after its collaboration with NeXT on OpenSTEP failed apart. https://en.m.wikipedia.org/wiki/Distributed_Objects_Everywhe... Things are no…
Teaching the parable of Chesterton's Fence to every programmer would save billions annually.
Its not a terrible idea but introducing a network boundary adds all sorts of constraints and issues that a normal program flow can safely ignore. Network partitions dont happen within a local system. I’d read up on CAP and Fallacies of Distributed Computing. They’ll more or less explain the challenges.
>a normal program flow can safely ignore. This isn't really true other than network errors are more likely than a machine getting shut down but you should really be writing your code as if something could go wrong at any moment.
It's a different class of error though. You can have code that ensures transactional integrity everywhere in the case of power failure, but that alone doesn't mean it handles network partitions, 429 responses, timeouts, token expirations, corrupt responses, etc efficiently or correctly.
> For example, let's say in an eCommerce application that the shipping calculator is getting hit a lot. You'd like to be able to scale this independently as a service, so you can handle all the requests without also having to replicate all of the other resources, such as the cart persistence, user sessions, etc. that are a lot more memory intensive. Assuming you allocate different resources for it. If you're using th…
I have worked with a service where we saved some money by splitting them up and specialized the VM SKUs like that. But it's far from as trivial as GP implies. You have to plan out the interface, shake out any shared data (shared in-memory cache etc), design it to not be so chatty, and as with everything perf-related, do lots of testing. So it's not like a GGGP's solution that automatically spins up microservices coul…
(The other thing I forgot to mention was having a good consistent hashing strategy in the load balancer, so that the instances of each service consistently speak to the same instances of the other service, even as machines are spun up and down, but still maintains an even spread of load. This helps greatly in terms of allowing each instance to cache only the data for requests that target that instance. With a round-robin load balancer you end up with a lot more variation in your incoming requests, and thus require either more memory or expect more cache misses. That was probably the hardest part of the project, since the built-in load balancers don't have the specific functionality we needed).
My company is currently moving to microservices, but for different reasons. The problem you raise was in fact fixed years ago in our org simply by properly decoupling our "monolithic" app into modules. The top-level build was simply a collection of all pre-built modules. After each dependency update, an automated regression would run, and if pass rate was less than X%, the change was not pushed upstream. Really, micr…
Modules help some, but being tied to a deployment cadence with say, hundreds of other developers, can be very painful. Having to delay my team's features because another team broke the build doesn't help my customers. Isolating CI pipelines by team can help that somewhat, but you still end up sharing fate with teams with whom you might have no real interdependencies.
I think this is pretty much it. You can have one microservice per team. Then you can do whatever you want in terms of tools, languages, deployment workflows, etc. As long as you honor your API contracts.
Monoliths are "part of the future". Data shows that there is dissatisfaction with using both monolithic and microservices architectures at the same time. In 2017, 66% were using hybrid approach to approach, but only 54% in 2019. See the interactive graphic at the bottom of https://thenewstack.io/observability-and-elk/.
Who besides your hypothetical salesman is arguing that the raisin d’etre for data lakes is user web analytics?
In my experience people arguing for data lakes talk a lot about some unspecified future benefit. Companies typically want to learn things about their interactions with their customers that allow them to make more money, and thus GA or its equivalent represents the 80/20 — or more likely 80% of the benefit for 1% of the cost — solution.
Your statement implies a lot of assumptions about a business’s model. Our company cares about user interactions in our app, software development metrics, quality metrics, sales metrics, etc. GA is just one small piece of the puzzle.
“Data lake” may not be the right answer, but GA certainly isn’t.