UPI: Anatomy of a Payment Transaction

Yes, UPI works. But it's a terrible system for privacy and autonomy. It has so many intermediaries, requires phone number and linked to person's identity. It should not be called a peer-to-peer payment system in anyway. It is controlled by the government rather than Visa or Mastercard. That's the only the difference.

For those outside India and/or non-active users of UPI - it drives so many transactions that even for an engineer, I forget how often I use it:

  - Payments between family members
  - Payments for every tiny/small item - a bag a chips or a cup of tea for example
  - Payments for car mechanic, plumber, or other services
  - Payments for online shopping or services - yes web apps show the UPI QR code and I can pay from my phone by scanning. Mobile apps will simply open the UPI app's payment screen
  - Buses, flights, trams, taxis, trains - online or on road

On an average day in a city like Kolkata, me and my partner make up to 20-30 transactions. I live in a small Himalayan village most of the year and I still make roughly 6-8 transactions a day.

While I'm not an expert on the Indian system, I think fast, universal and free payments are a very desirable user experience, but they're not desirable in a financial system.

    > Real-time payment rails that works 24/7 365 days a year from any bank to any bank (domestic, no exceptions) for free is truly a game-changer. Compare that to US payment rails which is slow and expensive. Apart from UPI, India has 3 more payment rails – NEFT (similar to ACH – batch settlement), IMPS (similar to UPI, instantaneous - but different user experience), RTGS (real-time, intermediated by the central bank RBI, but only for high-value transactions)

The financial industry and central banks have done themselves no favors with their opaqueness, but sadly this has led to a state where very few people understand the mechanics of the financial system.

The systems you named aren't alike. You are (understandably) mixing together several different networks and layers that are structurally different from one another.

In the United States, there's a mix of private and public institutions and networks. Anyone is free to start their own network – a necessary freedom. There are in fact several different networks such as the Fed's FedACH, and at the private level, the ACH Network (governed / semi run (???) by National Automated Clearing House Association (NACHA) in a confusing amalgamation of the Fed's FedACH network and The Clearing House's private Electronic Payments Network).

The Clearing House Payments Company™® also operates one of the world's largest private settlement systems, Clearing House Interbank Payments System, which includes American entities and non-American ones in the EU, China and other places as well.

From my somewhat limited knowledge of India, there is a lot of state interference and control over access to these systems and their creation; whereas in the US, while there are some regulations, any set of institutions are free to create their own private settlement systems and networks. Note – this is different from being free to transact with each other; this is the creation of private infrastructure that allows them to transact with one another at different levels of trust.

For example, from my limited reading just now - and to quote you – "IMPS (similar to UPI, instantaneous - but different user experience)" is kinda sorta a hybrid between institutional level agreements and consumer level agreements / networks. There's an important distinction here to draw.

Just because your bank knows my bank doesn't mean that my bank trusts you to honor your obligations to your bank and then (by extension) its obligation to itself and then to me. That's the lowest level of counter-party risk present in the system. And it's where the famous consumer networks like Visa, Mastercard etc. come in.

There is an important distinction to be made here, and it sort of highlights what someone I know stated as a systemic risk inherent to one network systems like India's — within these networks, the risk of fraud / failure of settlement from the end user to the institution is taken on by the entities running the networks... which in fact brings us neatly to something I realized while studying these systems, but haven't seen it written up elsewhere — (note, this is from an older comment, https://news.ycombinator.com/item?id=25951783 )

The question at the heart of the banking system is quite simple, if banks take capital from customers and use it to provide debt to others, then how much money should they keep on hand for their customers' withdrawals and transfers?

This question is hard to answer. As there is a conflict between what the bank does (i.e. provide debt), and how it is supposed to provide it (by taking savings etc.). Everything else, from central banks "offering cheap liquidity" is an add on. They are mechanisms that allow - for example, a bank to easily borrow this money so that they can cancel it out/repay it from transactions coming into their banks.

What makes it all borked is that you can't trust bankers with their grandmas. If there is a flaw, they will exploit it. Every major change has led to an exploit. E.g. In 1918, the American Government introduced the Leased Wire System that used the telegraphs and a network of 12 Reserves to allow banks to transact with each other across CONUS. It reduced the average time for cheques to be cashed in at banks across the country from 5.4 days in 1912 to just 2.4 days. Theoretically, this reduced the risk taken by banks when they transacted with unknown banks across the country, with the Government acting as the escrow. It catalysed innovation and led to an explosion of financial services across the young country.

The system was supposed to be foolproof by reducing the time "credit" was needed to make transactions. Essentially, until one bank sent the money and the other got it, they were operating on a system of credit. And they would "net" the books at the end of the day/week to physically transfer assets. FedWire (Leased Wire System) made everyone feel safe by sending notes of the transactions across great distances. But the netting still took time. All it took was one bank to misprice risk and fall behind on current obligations to other banks to cause the chain of dominos that led to hundreds shutting down in weeks and then thousands... which then led to the Great Depression.

Note - this is a highly simplified / subjective / one system view of a complicated sequence. more here https://www.stlouisfed.org/on-the-economy/2019/november/fina... / https://en.wikipedia.org/wiki/Panic_of_1930#Bank_failures

Important people got together and made rule changes to fix the problem. But then they innovated again. The Federal Reserve started making Automatic Clearing Houses (ACHs) and Remote Check Processing Centres (RCPCs) to make settlement faster, starting in the 60s and precipitating in 1972. This made settlement faster therefore safer. And it led to great financial innovation. The magic of computers and innovation meant that people could use these same systems to transact across the world!

Until 1974, when the German lender Herstatt collapsed due to foreign exchange investments based in the Dollar, which caused the bank to fail to meet its settlement obligations...

    That day, a number of banks had released payment of Deutsche Marks (DEM) to Herstatt in Frankfurt in exchange for US dollars (USD) that were to be delivered in New York. The bank was closed at 16:30 German time, which was 10:30 New York time. Because of time zone differences, Herstatt ceased operations between the times of the respective payments. The counterparty banks did not receive their USD payments

https://en.wikipedia.org/wiki/Settlement_risk#Herstatt_risk

This is a simplified history. But the history of banking is the history of doing settlement while managing liquidity and counter-party risk.

---

    > all are 24/7/365 and free

"Free" is a poor metric on which to judge a financial system. I would argue that it is an undesirable one. There is no such thing as free lunch. The more you understand these systems, the more you realize just how dangerous that "free" line is.

Cost in these systems is only partially about the "cost" of moving the digits. All of those dollars are there to mitigate / price in the risks inherent to the system. Hence, credit card transactions have a higher percentage per transaction than debit ones and so on up and down the chain.

Real-Time Gross Settlement systems, ACHs, consumer payment systems etc. change who, where and how that risk is managed. At the end, it might fundamentally be the same "money" going from one layer to another to another (kind of like a packet) but who takes the risk at every layer changes.

And that's also why they're paid through extensive testing over literally a century; people realized that a system that has capital shored up within it / paid up by its users is a system that can handle systemic risk. A system without that...

And that's a problem with purely state-based "free" systems like the Indian network that I haven't seen highlighted elsewhere is that counter-party risk isn't balanced between entities — it's taken on by the state. Sooner or later, there is going to be a cascade failure within this system unless something fundamental is changed.

----

whew - edited tone + grammar + spelling errors after posting!

All payment rails in India are RBI regulated directly or indirectly. NPCI is a non-profit section 8 company which is basically owned by the major PSU and private banks of India. And NPCI operates not just UPI, it also operates NEFT, IMPS, AEPS (aadhaar based payments), NETC (fastag), NFS (ATM network), Rupay debit/credit card network and BBPS (billpay). Only RTGS is operated by RBI directly.

Money serves its purpose while it's in motion. Increasing the velocity of money is good for economy. All the payment rails above do that 24/7/365 with lowest friction – by making all modes of payment possible and for free.

Digital payment rails is an order of magnitude cheaper (all inclusive) compared to cash rails. Accepting notes, counting, and depositing them, doing book-keeping and reconciling it against sales receipts, paying workers and vendors, avoiding leakage and theft etc – all cost time and money. For small merchants, it costs them time away from their business to handle cash.

UPI person-to-merchant (p2m) payments puts money instantly in their bank account. Their bank statements showing P2M deposits help them borrow for working capital at better interest rates.

Risk of theft with cash is much higher than digital theft from their bank accounts. RBI mandated 2FA, velocity checks, cooling-off periods, and awareness campaigns etc help people avoid scams.

W.r.t taxes, GST surveillance does catch merchants who accept high volume of P2M payments but aren't filing GST returns. Conversely, filing GST returns again helps with credit ratings and borrowing on better terms for working capital.

Thanks for the kind feedback.

I am not very familiar with the PromptPay architecture, but a quick search reveals that the user-facing app for QR scanning is bank-owned, rather than a Third-Party Application Provider (TPAP) like Google Pay is for UPI. In the case of UPI, you can link multiple accounts from the same or different banks to a single TPAP.

Another potential difference is the interbank protocol implementation. For UPI, the messages involved in a transaction are asynchronous. When NPCI calls the remitter bank for a debit request (ReqPay message), the bank is only supposed to send back an acknowledgement message (Ack) in the same HTTP request. Once the processing is done, the bank's switch sends a response (RespPay message) to a callback endpoint at NPCI (which, in turn, returns an Ack to the remitter bank). A similar flow happens when NPCI sends a request to the beneficiary bank for credit.