I am with you. The release could read:
"having subsumed the opensource version of ES, we are now relicensing, calling it our own, and would be really happy if the opensource community would lie to contribute, because actually we don't totally understand how this product works. Many thanks to all who help us"
The issue with AWS version are many fold, but the main one is that it forces extra usage of expensive EC2 units, for the following reasons:
1. Blue/Green updates -> Start a new cluster with new version, lock current cluster, copy over all the data (can take over a day), at the same time write to both clusters, when finished, lock both clusters while endpoints are swapped over, unlock new cluster, trash old cluster.
During this process Customer pays for both.
- Solution, done properly, fire up new node with new version, swap it in, wait for 'green', take out old... wait for 'green'.. rinse and repeat. Result.. system never goes down, endpoints remain the same, less cost to customer.
2.There are a minimum of the folloowing node types:
- Master (small) x3
- Data (heavy on storage, medium on memory)
- Hot Data (very heavy on memory as shards have to be
held in memory)
- Coordinating (query) nodes, heavy on memory, light on
storage (cos there is no real storage)
- Ingest (same as Query).
- voting only.. tiny
- AI/ML heavy on memory and storage.. cos they do real
work
In the AS world they have:
- Masters
- Data
- Hot Data (cos they are really pricey)
The Data nodes do all teh functionality of the data, ingest and query nodes. Th emain query always gores to a data node, while it passes to other data nodes to get the data out, then aggregates locally to it... so its incorrect usage of the system. A data node should only ever deal with its own data, and pass the results it finds back to a node away from the data.
As your system is squeezed.. you add data nodes... not coordinating/Query or Ingest nodes, which you probably need. But thats more money into the coffers.
3. Their userhasging function is also old (2011 vesrion of bcrypt) , and fails. Any static password produces different results everytime you use it.. at least on the current opendistro. So you are forced into either not using security, or using proper security, which can be cumbersome across the cluster (if rolling your own).
however.. on the cloud version they have base level security working, so thats Ok.. they arent using their own 'open' software.
I could go on.. but its all flaky, and misunderstood at the core developer level. Toput on record, I have spoken twice to teh core development team to describe how updates should happen.. the last time 18 months ago (by 'spoken' I mean a face to face video call) .. so they know hoe it should be done.. but dont do it.
4. I have noticed that there are some functions/methods available on the setup in Amazon Linux, that dont exist on other Linux versions (centOs/Debian) that are security related. AWS Linux is 'lifted' from Redhat.. so another piece of software that they didnt write.. but obviously Redhat are happy with this. Maybe they got the licensing deal sorted .. who knows.
basically gores like this:
"Who would actually install, in production, what essentially is very close to pirated software?"
Roll your own.. its a bit of grunt work up front, then 50% the cost of the cloud version.