跳转至

支付系统-架构设计

设计

目标

  • 简化逻辑：边界清晰、无冗余逻辑、方便测试
  • 分布式：容器化部署、无状态、可扩展、低耦合
  • 高用性：机房多主、自动扩容、数据库最终一致

拆分

  • 业务：代金券、活动
  • 支付中心：商品管理、价格管理、发货路由
  • 支付网关：下单、支付、代扣、通知、后督（订单二次确认）、对账、渠道服务

实现

网关体系

  • 支付网关：接受下单请求、校验签名、生成订单、操作DB、调用渠道服务
  • 代扣网关：处理签约请求、代扣请求、互斥逻辑
  • 通知网关：处理渠道通知

渠道服务集合

原则

  • 无db操作
  • 无订单操作
  • 只接受请求并响应对应数据
  • 不关心上层逻辑

微服务化

渠道微服务化，可以在国际化业务中带来更便捷的差异化部署

面向协议编程

  • pay(用户触发扣款或签约)
    • method：alipay/wechat/unionpay
    • from: WEB/MWEB/APP/SDK/BANK/SERVER
    • type: pay/sign_pay
  • confirm_pay(确认支付，用户二次验证)
    • from
    • type
    • method
  • refund(退款)
  • transfer(转账)

扩展&配置

  • etcd配置管理
    • 高可用
    • 实时变更配置
    • 跨地域同步
  • 简化配置
    • 使用field tag生成json
    • 管理后台根据field json生成配置表单

System Context diagram for Pay System«boundary»Company[Enterprise]«external_system»Business System Allows customers to viewinformation about theirinfo and orders.«system»Pay center system The system for biz.«system»Pay gateway System The system for user to payor auto_renew.Customer User«person»«external_system»Channel Pay System The real pay system tohandle users’ pay orders.Show products.Pay requestOrder requestNotifyOrder requestNotifyPayNotify

支付网关系统

«boundary»Pay Gateway[System]«container»Api Gateway[Service discovery, Canaryrelease, Parameters convert…Implements by ingress/zuul]«container»Pay Service[Generate order]«container»Sign Service[For Monthly/Quarterly/Yearlysign and renew]«container»Callback Service[Receive callback fromchannels.]«container»Notify Service[Notify to biz.]«container»Order Supervision[Query order status.]«container»Channel Services[Deliver pay requests betweengateways and channels.]«container»DB Service[Db operations.]«container»Queue[Message queue.]«container»Database[mysql,middlewares]«container»etcd[Config system.]«external_system»Pay center For biz«external_system»Pay Channels wechat/alipay/unionpay…Push messageQuery ordersPull message

支付调用时序图

Internal ServiceUserUserChannelChannelPayCenter(Biz)PayCenter(Biz)PayGatewayPayGatewayCallbackGatewayCallbackGatewayPayChannelsPayChannelsNotifyGatewayNotifyGatewayPayDatabasePayDatabaseMySQLMySQLQueueQueuePageCreate OrderVerifyGenerate OrderSave OrderSQLOKOKOrder RequestOrder RequestResponse DataResponseShow QrCode orredirect urlPayNotifyNotifyVerifyOKUpdateSQLOKOKProcess resultpushOKOKRedirect UrlShow PayResult PagelooppullNotifyalt[notify success]OKUpdate notify statusOK[notify failed]loop[10 times]Notify

签约

Internal ServiceUserUserChannelChannelVirtualAssetsSystemVirtualAssetsSystemPayCenter(Biz)PayCenter(Biz)PayServicePayServiceSignServiceSignServiceCallbackServiceCallbackServiceSignCallbackServiceSignCallbackServiceChannelServicesChannelServicesPayDatabasePayDatabase1Sign request2Sign(app_id, uid,product_id, amount)3IsExists(uid, app_id,product_id, channel)4OK5Sign request6Generate sign7Sign request8Response9Response data10Show QR or direct to url11Confirm12Notify13Query sign record14OK15Verify16OK17Update status18OK19OK20Direct21Check expire22Expire events(uid,product_id)23Renew(uid, sign_id)24Query sign record25OK26Pay27Generate order28OK29Order request30Generate message31Order request32OK33OK34OK35Notify36Query order37Verify38Notify39Notify40ok41ok

数据库高可用

  • 跨DC同步：基于otter进行同步，双向同步（多机房使用星型结构）
  • 同DC高可用：基于mycat和mgr，实现大容量、高可用db集群
  • mgr的心跳检测：二次开发mycat，对mgr节点状态实时检测并增删故障db
  • 应用层：去除自增主键，按机房、机器生成无冲突、有序的流水号，防止多机房数据冲突

«boundary»idc A (Master)«boundary»k8s cluster«system»Pay Gateway«boundary»Channel Services«system»DB«boundary»idc B«boundary»k8s cluster«system»Pay Gateway«boundary»Channel Services«system»DB«system»Otter Sync data«external_system»Mycat«container»Apps[Gateways]«container»Channel Wechat[Channel service]«container»Channel Alipay[Channel service]«container»Channel …[Channel service]«container»db[Group replication]«container»db[Group replication]«container»db[Group replication]«system»Otter Sync data«external_system»Mycat«container»Apps[Gateways]«container»Channel Wechat[Channel service]«container»Channel Alipay[Channel service]«container»Channel …[Channel service]«container»db[Group replication]«container»db[Group replication]«container»db[Group replication]W/RW/RW/RReplicationReplicationbinlogbinlogbinlogW/RW/RW/RReplicationReplicationbinlogbinlogbinlogsync

部署架构

KubernetesChannelsPayGatewayQueryGatewayRefundGatewayTransferGatewayCallbackGatewayNotifyGatewayPayDatabasePayManagerSystemOrderMonitorPayCenterChannel…ChannelWechatChannelAlipaymysql

交互

配置

核心交易系统是将配置信息存储在etcd容器内

渠道

  • 基础目录: /foo/bar/pay/config
  • 每个渠道占用一个文件夹，每个渠道账户占用一个文件，例如微信存放在/foo/bar/pay/config/wechat目录下，appId: 2088123456 所在的配置信息存储在/foo/bar/pay/config/wechat/2088123456