Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We've been thinking about software integration wrong for the past 50 years

nerve-signals.pika.page

Post reply on HN
Print view View original

31–40 of 40 posts

Re: We've been thinking about software integration wrong for the past 50 years

#31

Post by DontchaKnowit »

Animats wrote: post #7

It's a clickbait ad for something called "Nerve". The original idea for "electronic data interchange"[1] was to have a standard format for inter-company invoices, purchase orders, bills of lading, customs clearance, etc. So you only need N implementations, not N^2. This got going in the 1970s. So the formats are ancient. This is a flight ticket availability request in the international EDIFACT format. UNA:+.? ' UNB+I…

Yeah I worked for a health car software (EHR) company a while back and they use similar data interchange formats for messaging across different healthcare systems.

Its terribly ugly to read and not easy to work woth, BUT it is dead simple, nothing complicated just a lot of terse data.

Re: We've been thinking about software integration wrong for the past 50 years

#32

Post by axelthegerman »

I agree that moving data between arbitrary systems is an admirable goal, but this whole article doesn't make any sense nor solves anything.

Data migration is more than just "a function" that we can "just apply". Maybe this would work if all integrations were written in compatible languages with some sort of common SDK/API.

There is a reason open standards exist, to achieve interoperability. They have drawbacks like slower innovation cycles due to being standardized and now hard to change - hence all our $bigcorp don't give a flying fart and instead keep building up their walls which often don't even provide the ability to implement point to point integrations

Re: We've been thinking about software integration wrong for the past 50 years

#33

Post by mprast »

apimade wrote: post #22
bemusedthrow75 wrote: post #16

Earlier quoted context omitted.

> it's a really neat product idea. The one thing this idea is not is neat , though. It's the opposite of it. It entrenches the opposite of neat.

At the end of the day it's SQL on top of my integration layer. I love that. I don't know how it'd work in practice, or if it's any better than adopting a data warehouse - but even for understanding state management or offering a tool to create a view of a "customer" data definition is valuable. There's value to be had, I don't love the concept the author is proposing but the overall thing they're building, provided i…

hey, thanks for your feedback! If you're up to discussing further, would be interested to hear more about what you like/dislike about the idea and what potential you see, if any (you can just email me at mprast@get-nerve.com). Either way, thank you for reading!

Re: We've been thinking about software integration wrong for the past 50 years

#34

Post by mprast »

jerf wrote: post #15

It may not be immediately obvious, but this is basically another cri de coeur to the effect of "Why can't we make the Semantic Web work?" That's a good question. But you need to start from the fact that people have actually tried really, really hard, rather than nobody having tried at all, and it in fact hasn't worked despite rather substantial investment. When you start from there, you may obtain some useful underst…

Hey, thanks for your feedback. You're right; this is basically what the Semantic Web was trying to do. In earlier attempts to explain the idea I had mentioned Web 3.0 (that is to say, the 'original' Web 3.0), but wasn't sure enough people remembered it to be pertinent (I could certainly be wrong on this point).

I think the Semantic Web had a lot of promise, but it simply got too academic, with layers and layers of committees dedicated to ontologizing the entire world in a way that became progressively more detached from the needs of users and developers. As far as I could tell from my research, this was a stark reversal of the earliest ideas behind Web 3.0, which were more about establishing the 'meaning' of a piece of data in a decentralized way. Had the Semantic Web stayed the course on decentralization I think we may have seen a dramatically different outcome.

If you feel that this post didn't pay enough homage to the work that came before, that's a fair critique and I apologize. The concept of a Schema Network is definitely inspired by a lot of ideas that were in vogue in the Web 3.0 days.

Re: We've been thinking about software integration wrong for the past 50 years

#35

Post by xyzzy123 »

bradleybuda wrote: post #14

This is a neat idea, but the unfortunate reality is that every link in an "integration chain" is almost guaranteed to be lossy. The author notes an obvious version of this (what if one node doesn't support a field that the other nodes do) but there are countless other types of loss in each link - structure (i.e. document data types), precision, limits (max field size, max number), other "exotic" datatypes (geography,…

Also integrations have semantics and support specific use cases. They're not just boxes and lines on an arch diagram!

Re: We've been thinking about software integration wrong for the past 50 years

#36

Post by daltonlp »

An academic architecture almost always has the following basic characteristics:

1) It offers both superior control and convenience.

2) It is simple to comprehend and cheaper to operate.

3) Systems are connected to each other with drawn lines.

4) The architecture "Decouples" systems to make them simpler, not more complex.

5) Data structures map cleanly to each other, with no loss.

6) Data structures are 1-dimensional collections of "fields".

7) The architecture is not used in production. No organizations depend on it economically.

.

On the other hand, a practical architecture can be distinguished by the following characteristics:

1) It is used in production. People rely on it daily.

2) It is expensive, and needs constant human attention to balance control and convenience.

3) It has no optimal state - only varying states of failure.

4) Its systems have myriad ways to connect. Some are incompatible with others.

5) Data structures are messy and arbitrary. Some are provably incorrect.

6) Unicode in particular is a problem.

7) It is not comprehensible by any single person.

.

With apologies to Admiral Rickover :)

Re: We've been thinking about software integration wrong for the past 50 years

#37

Post by kkfx »

The software integration wrong is wrong BEFORE the web, the idea of an OS like a container ship and applications as container on the ship, able to "interact each others only with cut&paste, DnD humanly, or programmatically with some "desktop bus" is a commercial move that fail the entire subsequent IT development.

The original model was an OS-framework where an application was just a bit of code added to the common live codebase.

The web 2.0 try to reimplement such model without admitting the commercial horror we are in. Hence it can only fails despite all the progresses. Web n.x can't solve the underlying issue. The solution we have is simply accept that software as a product and as a service can't exists in a proper IT sound world. Data can be sold, iron can be sold, not more.

Data can be XML, EDI, json, yaml, ... as long as we have the base we can have easy enough integration because agreeing with a real world standard is not something doable up front while having the flexible base, the ease user-level/high-level development of classic systems do solve much integration issue. Try Emacs and you'll understand.

Re: We've been thinking about software integration wrong for the past 50 years

#38

Post by viraptor »

apimade wrote: post #21
zer00eyz wrote: post #3

I... I am dumbfounded. With the current state of the market why in the name of all that is holy would you depend on one vendor to get to another? I dont need to be at the whim of one vendor to get to the next one. What happens when the vendor in-between goes out of business or wants to present their own service or... I have no words for how bad this idea is.

You're thinking like an engineer. Engineers are poor at estimations, poor communicators and can't give me a damn budget. I'll never understand why engineers don't see that abstraction, even through corporations is bad in one area, but good in others (libraries, dependencies). Zapier? Tines? Workato? That's $10B in marketshare that disagrees with this, without even considering the order of magnitude of revenue present…

The SSO example is really different from the other ones. We've got a few standards implemented there and the authN/Z interaction is well defined.

For other systems we don't have that. It's a customer in one system, page viewer in another and a spreadsheet row in the billing service. Even with some common format, there's just too many specialisations you need to rely on custom fields.

There's a good reason zapier has an option for "just give us the code to run".

Re: We've been thinking about software integration wrong for the past 50 years

#39

Post by bemusedthrow75 »

apimade wrote: post #21
zer00eyz wrote: post #3

I... I am dumbfounded. With the current state of the market why in the name of all that is holy would you depend on one vendor to get to another? I dont need to be at the whim of one vendor to get to the next one. What happens when the vendor in-between goes out of business or wants to present their own service or... I have no words for how bad this idea is.

You're thinking like an engineer. Engineers are poor at estimations, poor communicators and can't give me a damn budget. I'll never understand why engineers don't see that abstraction, even through corporations is bad in one area, but good in others (libraries, dependencies). Zapier? Tines? Workato? That's $10B in marketshare that disagrees with this, without even considering the order of magnitude of revenue present…

> You're thinking like an engineer. Engineers are poor at estimations, poor communicators and can't give me a damn budget.

And yet it is engineers who will be building this magical platform that you think is 'neat', that you think engineers generally somehow can't understand.

How is that going to work out?

Engineers are always greener on the other side of the fence.

Zapier, Tines, Workato (and doubtless some other gibberish names I haven't heard of) don't intend for a system like this to work, either. They won't bend over backwards.

Re: We've been thinking about software integration wrong for the past 50 years

#40

Post by zitterbewegung »

jerf wrote: post #19
zitterbewegung wrote: post #18

Earlier quoted context omitted.

The semantic web did work. Google was able to enrich their own searches and bought a large semantic web called Freebase and incrementally enriched their web searches . When they can’t show you a relative ad they want to capture your search regardless and show you relevant info so you don’t switch providers.

If the "semantic web" worked, they wouldn't have had to buy a startup. You don't need a startup to read kindly-provided, accurate, well-categorized RDF tuples out of a web page. One of my favorite ironies of the "semantic web" is almost every story cited as a success story by its advocates is in fact a failure story, in that it involves somebody having to do something that wouldn't have been necessary if the semantic…

Not claiming they even invested in RDF tuples it clearly was that the ability to understand the internet meaning of sites and so forth required freebase and enriching their web results . It took a large engineering force to actually make it work.
Post reply on HN