Tips for Building Resilient Payment Systems (2022)

I wish they talked more about maintaining consistency. How exactly does the reconciling of their records with their financial partner’s records work? Is it an asynchronous job that’s run on a schedule?

Also I’m sure they have dual-write consistency problems, how exactly do they solve that? 2PC? Event sourcing? Outbox?

Re: Tips for Building Resilient Payment Systems (2022)

#6

Post by kalev » Sun, Jun 18, 2023, 5:18 AM UTC

I wholeheartedly recommend against using Shopify Payments, as they simply cut you off on a moments notice and they can’t explain why. This is the exact response we got from support:

> Regrettably we cannot share the specific reason or share further details about the rejection.

Like what? We applied, you approved, a lot of legitimate transactions happened and now you cut me off but can’t explain why? You really can’t trust them.

Re: Tips for Building Resilient Payment Systems (2022)

#7

Post by jpollock » Sun, Jun 18, 2023, 5:44 AM UTC

rohan_ wrote: post #5

I wish they talked more about maintaining consistency. How exactly does the reconciling of their records with their financial partner’s records work? Is it an asynchronous job that’s run on a schedule? Also I’m sure they have dual-write consistency problems, how exactly do they solve that? 2PC? Event sourcing? Outbox?

Bank reconciliation is done using files. They will arrive randomly, typically on a multi-day lag. Then the payments are matched against the requests that were sent to the bank. The amounts _can_ and _will_ differ.

I understand it gets... interesting.

Re: Tips for Building Resilient Payment Systems (2022)

#8

Post by Brystephor » Sun, Jun 18, 2023, 5:53 AM UTC

rohan_ wrote: post #5

I wish they talked more about maintaining consistency. How exactly does the reconciling of their records with their financial partner’s records work? Is it an asynchronous job that’s run on a schedule? Also I’m sure they have dual-write consistency problems, how exactly do they solve that? 2PC? Event sourcing? Outbox?

In general, reconciliation is done asynchronously. You send a payment out. The processor some number of days later sends out a file containing information about that payment. You validate that the payment the processor recognized is the same as the one you sent out. That's a high level summary.

Also regarding dual write consistency problems: they mention Google cloud. If they're using NoSQL such as Data store/firestore, they have distributed transactions. Another possibility is they must trust their payment clients to use idempotency keys correctly. If you get two requests with the same idempotency keys, then the requests are considered to be the same and therefore it wouldn't matter which one is processed first. If they're different requests with the same idempotency keys, they may have some validations.

I believe DynamoDB also has conditional writes that can essentially be "if entity X does not exist: write entity X". Then there is a race condition for different requests with identical idempotency keys, but if that does happen, it'd result in an error and someone would need to look into it and figure out why that was allowed to happen.

Re: Tips for Building Resilient Payment Systems (2022)

#9

Post by jmholla » Sun, Jun 18, 2023, 12:50 PM UTC

kalev wrote: post #6

I wholeheartedly recommend against using Shopify Payments, as they simply cut you off on a moments notice and they can’t explain why. This is the exact response we got from support: > Regrettably we cannot share the specific reason or share further details about the rejection. Like what? We applied, you approved, a lot of legitimate transactions happened and now you cut me off but can’t explain why? You really can’t…

Who would you recommend?

Re: Tips for Building Resilient Payment Systems (2022)

#10

Post by kalev » Sun, Jun 18, 2023, 3:48 PM UTC

jmholla wrote: post #9

kalev wrote: post #6

I wholeheartedly recommend against using Shopify Payments, as they simply cut you off on a moments notice and they can’t explain why. This is the exact response we got from support: > Regrettably we cannot share the specific reason or share further details about the rejection. Like what? We applied, you approved, a lot of legitimate transactions happened and now you cut me off but can’t explain why? You really can’t…

Who would you recommend?

For payment processing? Mollie has been good so far, however because it's an external payment processor we pay additional fee to Shopify.