Incident response: email reminders incorrectly sent to customers
Yesterday afternoon on the 4th July 2017, between 2pm and 2:35pm BST (British Summer Time), we sent a number of email reminders to end customers (those wanting to pay one of our merchants), asking them to authorise historic direct debit mandates. The emails were sent to individuals who received a request from one of our merchants to set up a mandate, but who never completed the original request.
All our systems remain secure and uncompromised, no third parties were involved, and we have put in place changes to fix the root cause of this issue. We apologise for any inconvenience caused.
What should I do?
What you should do depends on whether you’re an end customer who received an email from us on 4th July or a merchant collecting money through GoCardless.
For end customers
If you haven’t already taken action, please ignore the email you received. We have deactivated the link within it.
If you opened the link and completed the form, and you don’t want to set up payments to the merchant, please contact the merchant or your bank to cancel the Direct Debit mandate, or alternatively contact us if you have any queries.
If your customers were impacted by the issue, we’ll be in touch with more detailed information.
We’ll be letting you know which customers were affected, and what options you have for handling any action taken by your customers in relation to the emails.
As always, our support team is happy to help.
In detail: how did this happen?
On the ‘Customers’ tab of our dashboard, we have a ‘Remind all’ button, which sends a reminder to all your customers who haven’t finished setting up a Direct Debit mandate with you.
As part of our phone support service, our staff can trigger these emails on behalf of a merchant.
The combination of a bug in our dashboard API and the way our dashboard works when staff are acting on behalf of merchants led to reminder emails being sent for all merchants that had pending customers, rather than just the merchant the support agent triggered reminders for.
As soon as we found out about the issue, we halted sending any further emails, triaged the bugs, and began working on our response.
We have fixed the bug, and performed multiple tests to ensure the fix is working as intended.
We want to reiterate our apology for the inconvenience caused by this issue.
As part of our incident response process, we’ll be conducting a detailed investigation into the issue and our response to it. We’ll be focused on finding any improvements we can - in both our systems and processes - that will reduce the likelihood and impact of this kind of issue happening again.
If you have any questions after reading this, please feel free to contact our support team.
Our response to Cloudflare's bug disclosure
When triggered, the bug led to request data intended for different websites to be mixed together. This meant that a page served by website A could include the contents of a request made to website B.
Because of how widely Cloudflare is used - some measurements put around 5% of the web behind Cloudflare - this is a true internet-scale problem, and warrants a quick response from everyone involved.
As users of Cloudflare's proxy service, we've taken the time necessary to properly assess the risk to our customers and we've done that in partnership with Cloudflare's support team. While the risk to GoCardless data is extremely low, we'd rather be clear with our customers about what's happened.