Live data from Hacker News

Billing systems are a nightmare for engineers

getlago.com

261–270 of 372 posts

Re: Billing systems are a nightmare for engineers

#261
(1 of 2)

In the 00's, I was part of an effort to get the State of Oregon to consider open source software. A 40 million billing system debacle was part of what drove this effort. The details are not important, save to say the ASA paid big to stall the legislation and that worked. It was stalled, and word got out that future bills would be toxic. So, that's over, but a great idea remains!

I am putting it here, because someone, somewhere should do it. Will save us all a lot of money and do a great public good. And it's likely to pay a fellowship of some kind very well over the next couple decades. I'm on a different project and need to finish it, and I'm likely to be paid well on that, so...

Instead of paying the likes of Oracle, or some consulting firm to write and support the billing system, the State should do it. So far, I've seen several bad contracts in my State where the State has a right to use the software, does not own the code, must pay for support, and on it goes. This is quite expensive, and frankly, it didn't work well.

Instead do this:

The State sets up an organization. This org is public, and those who work there are public servants, or some similar arrangement. The key thing is the billing system gets written on open code, uses open data, and the resulting software is owned by the public, through the State essentially. This code gets published on Github under an appropriate open license.

Fund this org, paying respectable developer salaries, and get the system written, tested, deployed, whatever is required. This funding comes from public tax dollars, and I would suggest the lottery as a primary source.

Once the initial system is complete, and let's say it's for water. Once it's complete, the system can be published and the whole thing becomes an export for the State that does it.

Any municipality anywhere can do one of a few basic things:

They can build and deploy the system and handle billing themselves.

They can hire the Org to help them do this, contact for support, whatever.

They can hire private entities to deploy the system.

Changes can come from anywhere, but the organization is best suited to implement them. This can be on a contract basis, funded reasonably in any number of ways.

The organization continues on to develop other public works tools, each funded by public dollars, each displacing some expensive thing or other, and each saving the people quite a bit over the course of say a decade.

Wash, rinse, repeat.

The end product is a nice pile of public works software needed by tons of municipalities most all of whom are paying hard for it now and they pay hard for basically everything, and continue to do that. The public could make these investments once and benefit for a very long time.

Perhaps more than one State wants to be a part of something like this, or whatever State gets going on it first, might want a few organizations, depending on what is to be done and the scope / scale required. Either way, the Organizations provide jobs and consulting / support and basically own the public works they develop and are funded to maintain those software works.

An advantage over contracts and proprietary license is cost, particularly when the likes of Oracle are involved (and no judgement here and no calling out as I'm only saying Oracle as a well known example among many out there and I just feel the public works can be done without Oracle and the cost associated with similar entities). Having the data be open is another advantage in that it can be used in all sorts of ways, and the public records can be made available to the public in simple, effective ways.

Mix in smart API's and the whole thing becomes very high value and an attractive base target for all sorts of public software development having to do with the basic machinery of society.

None of this is particularly sexy, but it does pack a big punch in terms of modeling how a more modern, digital society could employ open code, open data to get the work we need done, done in a cost effective, efficient way.

And that's it basically.

Over time, this first, or core organization will accumulate considerable domain knowledge. And it would become the basis for a lot of other work. Ideally, others started up in various places would all garner the knowledge needed by domain:

Utilities, water, power, internet (where it's actually municipal, and we know that's not popular with the big telecoms...), garbage, etc.

City functions, parking, various permits, anything that has a process that could be automated and or just performed on open code in simple, direct ways that are lean, consistent.

Vehicles: Registration, and all that tends to be tied into these systems. In my state, we've got vehicle registration, drivers licenses / ID Cards that link to facial biometrics (I don't like this, but that is another discussion), and it's also voter information central with a signature, party affiliation, address, and everything needed to manage the voting rolls, and execute Vote By Mail nicely, and in a trustworthy fashion. Oregon has done well on that front. Citizenship records are in this pool of software too.

Health Care: Right now, again in my State the Medicaid system insures people who are diabled, are poor, perhaps wards of the State. This is currently on Oracle, and has been such a mess it's ended up in court. I don't even know the state of it all right now, but it's a lot of money and a lot of grief.

Re: Billing systems are a nightmare for engineers

#262

(1 of 2) In the 00's, I was part of an effort to get the State of Oregon to consider open source software. A 40 million billing system debacle was part of what drove this effort. The details are not important, save to say the ASA paid big to stall the legislation and that worked. It was stalled, and word got out that future bills would be toxic. So, that's over, but a great idea remains! I am putting it here, because…

(2 of 2 --> I got "comment too long!)

Now, this won't be popular with a lot of us here, myself included, as doing stuff like this will displace some revenue streams companies and people depend on. However, a longer view looks considerably more attractive. The jobs created are needed. Tie this stuff into education, and it's a perfect "farm" training ground for up and coming developers to work on important projects and get good experiences.

A lean and mean digital civics is attractive and necessary for a lot of reasons. The case for doing it on open code and open data is super compelling and compliant with the basic ideas behind public works anyway.

As citizens, we get to experience our taxes actually getting more work done and at a lower cost. And yes, spending will move toward other areas where it's possible to extract revenue, and that's an ongoing problem. Real good can be done, and that's what I'm writing about.

Companies wanting to or needing to interoperate with the State government can do so employing everything from really old school, courier, papers, and the like, through to very new school, all digital, everything operating in ways we know can work well given the incentives are where they need to be and the organization is funded as it needs to be.

The second and third order effects could be very significant! Standards fall out of this kind of thing, and devices, protocols, and all manner of currently diverse and expensive to maintain systems could be made leaner, meaner, have much longer service lives and the knowledge needed will be out in the open and available to the people who need it.

Obviously security needs to be a consideration. And that's no different than what we have going on right now. The big difference is an open society can be built on open code and data and having it be funded in ways that maximize the value to the public while also keeping costs where they need to be, appropriate makes a ton of longer term sense.

In a scenario where this is all closed, the idea of competition is often cited as a reason to do it private, but the realities are corruption and other harsh realities tend toward scenarios where lock in type solutions favor the public getting the lowest value possible for the highest dollar amount possible. We see this over and over and over. And that's why the ASA (American Software Association) opposed the effort in Oregon and another one in Texas with such zeal it was kind of amazing really.

Should we do it in some fashion as I've hinted at here, the opposite becomes true. Initial value for the dollar might not be a whole lot different from whatever contract + proprietary software delivers. Over time, the trend will be toward getting very high value for the dollar.

Regarding Standards... Think center of gravity. Systems like these can have very attractive start costs. And cost of change will vary, and likely be made higher by current players wanting to leverage lock in to preserve revenue streams (and who can blame them?). Fair enough, but as more gets done, that center of gravity will prove compelling, and we all get the benefits working in a similar, more compatible way provide.

When we all require a resource or process, and let's just take power or water for a moment... Running these things at profit puts incentives in the wrong places. Maximizing profit is not the same as maximizing the public use value. And where that is wrong, everyone takes a small hit, and those tend to add right up.

Maximizing public use value can end rent seeking type arrangements that cause more grief than expected. Maintenance is one, tech debt is another of many that come to mind. Where these are out of public view, they tend to get ignored and risks accumulate, until there is an event, and suddenly, those risks play out, and we the public are faced with a large bill, and that's an old story, no need to say more.

I'll end with the belief this seems to be an excellent way to employ open code and data. And one of the big problems we find out there with open code and data is failure to work on "dull" or "uninteresting" problems. How many times do we have to find out that project X is being used by everyone, and nobody really owns making sure it's going to make sense to continue to use it?

Now I will just stop there. That's it. Some municipality somewhere would LOVE to get a State grant to get this started, and some State somewhere really needs this kind of thing enough to bite on the idea, and there are a ton of messes to clean up.

So maybe those should just start getting cleaned up!

Re: Billing systems are a nightmare for engineers

#263
post #61

Earlier quoted context omitted.

Invoicing gets even better in countries like Portugal, where you have to send to the tax authorities every invoice that you generate.

In Italy too. They specifically developed an XML format for that, and each time you issue an invoice you have to (well, with some exceptions, but they are gradually removing all of them) send a copy to the revenue agency, which will forward it to the recipient (and keep a copy). While it is a bit inconvenient, though, I don't think it is a bad idea. I am pretty sure it helps a lot making the life harder for people ev…

They should rather give a service to issue /cancel an invoice and people will happily use the service

Re: Billing systems are a nightmare for engineers

#264
post #251

Basically everything about enterprise is a nightmare for engineers. What amazes me (after having worked at some of the most profitable companies in the world) is just how little intelligence the leadership has about their own revenue or costs, beyond "wow huge amounts of money is coming in or going out". And how many critical processes are implemented manually, by individuals, with personal spreadsheets, on their lap…

> intelligence the leadership has about their own revenue or costs

I see the positionning of the CFO right next or below the CEO as a mechanism to let the CEO care about “big pitcure” money flow, while having someone in contact with reality guide decisions and veto stuff that won’t fly in cost/revenue terms.

Re: Billing systems are a nightmare for engineers

#265

Earlier quoted context omitted.

Could you expand on what kind of hacks/kludges were required?

Basically the mainframes were almost incapable of any data exchange that was meaningful in the 21st century. Instead of IBM doing something same the market just built software to scrape bills off the mainframes printer output, which you could intercept in electronic form. But then every accounts bill files were different. And sometimes the mainframes moved data around on the page, and so on and so forth.

That gives me flashbacks. I worked on a payment system that had to pull data from third-party service providers that only made it available in the form of downloadable spreadsheets from godawful slow and unreliable JavaScript-heavy web portals. Just getting to the point of downloading the spreadsheets was painful enough, and then the spreadsheets themselves were inconsistent and would sometimes flap back and forth between two different formats for a few weeks as the service provider repeatedly deployed and then rolled back new versions of their software.

Re: Billing systems are a nightmare for engineers

#266
post #259

Wait until you get into the complexities of taxation. If you think you know them, pay attention to https://twitter.com/aotearoa_ben/status/1526786701750050817 . That's a law that changes the tax rate if 1. The purchase happens in Texas. 2. The payment cleared in a specific 2 day period. 3. The item cost Seriously, calling out just one example, https://comptroller.texas.gov/taxes/publications/98-490/scho... points out…

I worked on a tax app. Total nightmare. I remember having to adjust tax rate for a couple parishes in Louisiana and a city in Florida for example. Trying to come up with a readable, testable, way to get that done was something I tell ya.

Re: Billing systems are a nightmare for engineers

#267
Anyone know of a company other than Paddle and PayPro Global that act as the merchant of record and remit taxes for you globally?

I'm a solo founder not interested in dealing with either the complexities of these billing issues nor with the taxes issue. But my product would easily be of interest globally and the API's I've built have more interenational customers than global customers so I suspect the same may be true for the product I'm building out of those apis...

Re: Billing systems are a nightmare for engineers

#268
post #259

Wait until you get into the complexities of taxation. If you think you know them, pay attention to https://twitter.com/aotearoa_ben/status/1526786701750050817 . That's a law that changes the tax rate if 1. The purchase happens in Texas. 2. The payment cleared in a specific 2 day period. 3. The item cost Seriously, calling out just one example, https://comptroller.texas.gov/taxes/publications/98-490/scho... points out…

I worked on a tax app. I was amazed to find that some tax tables necessary to compute taxes were not released until months after taxes were due. WTF?

Re: Billing systems are a nightmare for engineers

#269
post #194
post #183

Earlier quoted context omitted.

Yes, almost everything ended as temporal or bitemporal model. Pain to work with. Even for simple questions like "What is the customer #1234 company name?" the system needs to know when are you asking.

By "temporal model" do you mean relational tables of "customer X used Y amount of this resource at price Z from timestamp A to timestamp B" or ... ?

I assume they are using something like DB2, which has very specific meaning when it comes to temporal or bitemporal tables.

https://www.ibm.com/docs/en/db2/10.5?topic=tables-bitemporal

Re: Billing systems are a nightmare for engineers

#270
post #175

'Tax logic' is an oxymoron and a trap for smart technologists to fall into. Tax codes are not logically consistent or clearly defined. They're a bunch of illfitting laws, beauracratic/regulatory guidance and interpretations of judicial decisions. I tell folks working in this area to accept the illogical nature of it all and to be ready for all sorts of arbitrary last minute changes. As the article points out understa…

Tax is written by a bunch of people whose background is overwhelmingly from law, not finance, for reasons that have nothing to do with logic, consistency, financial literacy, understanding, sanity, or common sense. Politicians frequently pass laws without understanding what it will do or what its implications are, because they just didn't think that far ahead. It is quite possibly the worst system you could come up with for writing tax law.

That said, there is a logic to most of it, but not in a way that allows you to come up with general heuristics or universal abstractions.

Post reply on HN