How to Integrate PayPal with Rails Application

Hi! This is one more tutorial on how to integrate PayPal with Ruby on Rails app. If you worked on developing apps with any payment systems before, the first thing that comes to your mind is ActiveMerchant. It works well when you have a simple checkout scenario in an online shop or when you need to support several payment systems. However, PayPal has a quite extensive API and the ActiveMerchant adapters for PayPal (Direct Payment and Express Checkout API) use only basic functionality - placing an order, payment processing, cancellation and refunding.

If a more advanced integration with PayPal is needed, ActiveMerchant doesn't suit. What can we do in this case? Should we write a new client from scratch? Why not. But don’t forget that PayPal has an official Ruby SDK for many Classic API: Adaptive Payments, Express Checkout, Permissions, Direct Payment (and for the REST API as well). This is what we will talk about.

Setting up access to PayPal API

On Github you can find a short description of how to set up access to API and an example of some API-request. Classic API provides both NVP (name-value pair) and SOAP-formats. Actually, Ruby SDK is a thin wrapper for the SOAP API.

The first step of PayPal integration process for any programming language is registration on the developer portal and getting access to the PayPal Sandbox environment. You need to create several buyer accounts (Personal account) and merchant accounts (Business/Premier account). Don’t forget to put enough money to buyer accounts. If after manual tests you run out of test money, the empty wallet can be refilled only by transferring money from another test account. Do not rashly delete test accounts, you won’t be able to use their emails again. The next step is to enable API to access your Business (merchant) account and generate the access credentials - password and signature.

Then you can set up the Rails app. Specify a username, password, signature, mode and probably app_id in config/paypal.yml. It makes sense to test the application in Sandbox first, that’s why we use an option mode: sandbox.

# config/paypal.yml

default: &default
  # Mode can be 'live' or 'sandbox'
  mode: sandbox

  # Credentials for Classic APIs
  username: username
  password: password
  signature: AQU0e5vuZCvSg-XJploSa.sGUDlpAC6auTE-xLN1ERo8.JTZ4A51PHwu
  app_id: app_id

Let’s check if our requests can reach PayPal. For the test, we make a request to the Permissions API - we initiate a scenario of getting the permissions for placing an order, reservation, money transfer and refund. RequestPermissions API-request:

api_caller = PayPal::SDK::Permissions::API.new
request = api_caller.build_request_permissions(
  :scope => ['EXPRESS_CHECKOUT', 'AUTH_CAPTURE', 'REFUND'],
  :callback => 'http://example.com/redirect-to'
)
response = api_caller.request_permissions(request)
# => Request[post]: https://svcs.sandbox.paypal.com/Permissions/RequestPermissions
# => Response[200]: OK, Duration: 1.296s
response.token
#=> "AAAAAAAfmW8C1Xq0qkpi"

If the request passes successfully and a token is received in response, then the access to PayPal is set correctly.

How basic API requests work

Let’s review a simple request for reserving funds on an account of buyer, who has made a purchase (TransactionID is a transaction identifier for creating an order, returned by PayPal earlier).

api_client = PayPal::SDK::Merchant::API.new
request = api_client.build_do_authorization(
  :TransactionID => 'O-1HX867885F889024F',
  :Amount => {
    :currencyID => 'GBP',
    :value => 150 }
)
response = api_client.do_authorization(request)
response.TransactionID
# => '681296425B698101R'

You can notice that the structure and naming both of parameters and of returned data exactly match the SOAP-format of DoAuthorization API request from the documentation (DoAuthorization API Operation (SOAP)).

The given above API request turns into an exchange of XML-documents:

The request, sent to PayPal:

<?xml version="1.0" encoding="UTF-8"?>
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:cc="urn:ebay:apis:CoreComponentTypes" xmlns:ebl="urn:ebay:apis:eBLBaseComponents" xmlns:ed="urn:ebay:apis:EnhancedDataTypes" xmlns:ns="urn:ebay:api:PayPalAPI">
   <soapenv:Header>
      <ns:RequesterCredentials>
         <ebl:Credentials>
            <ebl:Username>username</ebl:Username>
            <ebl:Password>password</ebl:Password>
            <ebl:Signature>AQU0e5vuZCvSg-XJploSa.sGUDlpAC6auTE-xLN1ERo8.JTZ4A51PHwu</ebl:Signature>
            <ebl:Subject>UAVRQCSDMQCKY</ebl:Subject>
         </ebl:Credentials>
      </ns:RequesterCredentials>
   </soapenv:Header>
   <soapenv:Body>
      <ns:DoAuthorizationReq>
         <ns:DoAuthorizationRequest>
            <ebl:Version>106.0</ebl:Version>
            <ns:TransactionID>O-1HX867885F889024F</ns:TransactionID>
            <ns:Amount currencyID="GBP">150.0</ns:Amount>
         </ns:DoAuthorizationRequest>
      </ns:DoAuthorizationReq>
   </soapenv:Body>
</soapenv:Envelope>

Successful response:

<?xml version="1.0" encoding="UTF-8"?>
<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" xmlns:SOAP-ENC="http://schemas.xmlsoap.org/soap/encoding/" xmlns:cc="urn:ebay:apis:CoreComponentTypes" xmlns:ds="http://www.w3.org/2000/09/xmldsig#" xmlns:ebl="urn:ebay:apis:eBLBaseComponents" xmlns:ed="urn:ebay:apis:EnhancedDataTypes" xmlns:ns="urn:ebay:api:PayPalAPI" xmlns:saml="urn:oasis:names:tc:SAML:1.0:assertion" xmlns:wsse="http://schemas.xmlsoap.org/ws/2002/12/secext" xmlns:wsu="http://schemas.xmlsoap.org/ws/2002/07/utility" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
   <SOAP-ENV:Header>
      <Security xmlns="http://schemas.xmlsoap.org/ws/2002/12/secext" xsi:type="wsse:SecurityType" />
      <RequesterCredentials xmlns="urn:ebay:api:PayPalAPI" xsi:type="ebl:CustomSecurityHeaderType">
         <Credentials xmlns="urn:ebay:apis:eBLBaseComponents" xsi:type="ebl:UserIdPasswordType">
            <Username xsi:type="xs:string" />
            <Password xsi:type="xs:string" />
            <Signature xsi:type="xs:string" />
            <Subject xsi:type="xs:string" />
         </Credentials>
      </RequesterCredentials>
   </SOAP-ENV:Header>
   <SOAP-ENV:Body id="_0">
      <DoAuthorizationResponse xmlns="urn:ebay:api:PayPalAPI">
         <Timestamp xmlns="urn:ebay:apis:eBLBaseComponents">2015-10-07T08:58:31Z</Timestamp>
         <Ack xmlns="urn:ebay:apis:eBLBaseComponents">Success</Ack>
         <CorrelationID xmlns="urn:ebay:apis:eBLBaseComponents">7e427a34b9612</CorrelationID>
         <Version xmlns="urn:ebay:apis:eBLBaseComponents">106.0</Version>
         <Build xmlns="urn:ebay:apis:eBLBaseComponents">000000</Build>
         <TransactionID>681296425B698101R</TransactionID>
         <Amount xsi:type="cc:BasicAmountType" currencyID="GBP">150.00</Amount>
         <AuthorizationInfo xmlns="urn:ebay:apis:eBLBaseComponents" xsi:type="ebl:AuthorizationInfoType">
            <PaymentStatus xsi:type="ebl:PaymentStatusCodeType">Pending</PaymentStatus>
            <PendingReason xsi:type="ebl:PendingStatusCodeType">authorization</PendingReason>
            <ProtectionEligibility xsi:type="xs:string">Eligible</ProtectionEligibility>
            <ProtectionEligibilityType xsi:type="xs:string">ItemNotReceivedEligible,UnauthorizedPaymentEligible</ProtectionEligibilityType>
         </AuthorizationInfo>
      </DoAuthorizationResponse>
   </SOAP-ENV:Body>
</SOAP-ENV:Envelope>

We can see that PayPal has returned an identifier of a new authorization transaction. The order status has changed for "Pending" and the reason of this (PendingReason field) is "authorization".

Sending requests on behalf of another account

PayPal allows making requests on behalf of another account. Let’s say, your partner provides/delegates permissions for making payments to you (this can be automated through the PayPal Permissions API). You have to specify your partner's PayPal account as subject in configuration. If you have a couple of partners with different subject, you have to set the configuration of PayPal client dynamically. It can be done like that:

def self.api(credentials)
  PayPal::SDK::Merchant::API.new.tap do |a|
    a.set_config(
      Rails.env,
      credentials.dup.extract!(:username, :password, :signature, :app_id, :subject))
  end
end

Handling errors

Handling errors is a separate and interesting part of PayPal implementation in Ruby on Rails web app. At least, the errors should be logged. Some of them are related to the problems in buyer accounts (e.g. not enough money), others - with merchant accounts. In the last case, you should respond immediately.

This is how you can get the list of errors if a request failed (yep, there can be several of them):

def errors
  @response.error.map { |er| er.message }
end

HTTPS request to PayPal can result in 500-error. Seriously, I saw it with my own eyes. When this happens, HTTP client simply throws an exception. That’s why it would be nice to catch them for every API request. Do not forget, it is right to catch only subclasses of the Standard Error class, for example:

module Payment::PayPal
  class BaseError < RuntimeError
  end
end

module Payment::PayPal
  class BadRequestError < BaseError
    attr_reader :code

    def initialize(message, code)
      super(message)
      @code = code
    end
  end
end

def self.with_error_handling
  yield.tap do |response|
    unless response.success?
      raise BadRequestError.new(
        "#{ response.errors.join('.') }", response.error_codes[0])
    end
  end
rescue StandardError
  raise Payment::PayPal::BaseError
end

Instant Payment Notification (IPN)

You are already familiar with sending requests, creating orders, handling authorization and payment capturing. But what about other important aspects? PayPal suggests using Instant Payment Notification (IPN), which is an HTTP request initiated by PayPal that informs you about various events, such as order status changes or complaints filed by buyers against your online shop. After each API request, you should wait for IPN to receive the new order status, as PayPal returns this status in response to the API request, but you should rely solely on IPN. While you can ignore IPN in most cases, it's essential at times, such as when your operation (such as reserving funds) is deemed suspicious and is processed manually, leading to a typical delay of up to 24 hours.

In theory, IPN can have a significant delay, but in practice, notifications are instantaneous. API requests generally take 2-3 seconds, and IPN, which is triggered by the API request, can arrive before the original API request is completed. Thus, you must take this asynchrony into account and record the successful (or failed) completion of a transaction even before your system has saved data about the start of the transaction to avoid losing an order for no reason.

Processing the IPN produces a dilemma: if handling the notification has resulted in an error, should we send a HTTP-code of success or error in the response to PayPal? If PayPal gets the "unsuccessful" code, it will keep trying to send the notification again for 2-3 days in the hope that your server will manage to process it. It's convenient - you see the error, fix it and with the next repeated IPN the scenario will continue succesfully. There is only one problem - after a certain quantity of "unsuccessfully" processed IPNs, PayPal switches off the IPN sending for your account and, as a result, your valid orders suffer because IPN will not be received for them too. PayPal usually sends a notification to inform that you are running out of the limit of unsuccessful IPNs and your IPN will be turned off soon.

PayPal also suggests to check if IPN was received earlier before processing it (i.e. it is recommended not just to log IPN but to store it in a database). Surprisingly, the duplicates (which were successfully processed before) really arrive and it's really needed to secure yourself with one more verification. The fields that uniquely identify IPN (at least in the context of Express Checkout) are txn_id, payment_status and pending_reason. You will have something like that:

def duplicate?
  Shop::IPNNotification.where(
    txn_id: txn_id, payment_status: payment_status, pending_reason: pending_reason
  ).any?
end

Conclusion

In conclusion, I’d like to mention that PayPal Classic API leaves a controversial impression. On one hand, it provides many opportunities and covers 99.9% of all user requests. On the other hand, there is an evident incompleteness and inconsistency of API. Some obvious (and really needed) features aren’t implemented.

Ruby SDK and PHP SDK for Express Checkout officially stopped being actively supported since the fall of 2014 and PayPal recommends to use REST API. REST API is attractive for its simplicity and logic nature but it can’t fully replace Express Checkout for now.

We have faced with the following difficulties of PayPal integration in Ruby on Rails app:

If there is a problem with API request e.g. it's already in production, and you want to contact PayPal support, you will be asked to give an internal identifier of API-request or even a request/response by itself in form of NVP or SOAP. As a result, there is a necessity to log every request and response. Ideally, you need a raw representation of the request and response text. Ruby-clients out of the box can’t do that. But they can be easily patched and the needed functionality can be added.
The format, in which the client returns errors, differs from gem to gem. For example, in Express Checkout the list of errors is available as getter method errors, in Permissions - error. Message, long message and error code also may have similiar, but not the same, names.
It often happens, that neither error code nor its description helps to figure out what the problem is. You will find nothing new in the documentation. From time to time, we had to apply to the support service in order to determine the reasons of a new error and how to solve it.
Contacting the support is like playing a roulette, if you are lucky to be matched with a proper person, you will solve the problem quickly. Otherwise, you may spend a few days trying to sort it out and you will get no clear solution or recommendation. A few times, we had to wait for an answer for a day or two. If you create a ticket with high priority, its priority can be quietly decreased.
Similarly to Sandbox and Production environment, PayPal sometimes has problems with the availability of service. For 1-2 days PayPal can repeatedly return "Internal Error" and this error may randomly occur for completely valid requests.

What conclusions we have reached:

Read the documentation and integration guides attentively. It’s a quite obvious advice but still any little thing can be important. Tricky points or restrictions may be not clearly defined and you will face them after releasing the app to production.
Log all requests to API. Preferably, save as much as possible data about transactions, their identifiers and time of events in the database and save all this data in the original state - it will speed up the investigation of errors.
PayPal support is a useful tool for solving the problems. Generally, you can also get help on Stack Overflow, where sensible questions are quickly answered by a guy with a very high rating, probably an employee of PayPal.