Building a Modern Affiliate Platform: My Tech Decisions for Speed, Scalability, and Self-Healing Automation

Introduction

The online financial market is one of the most competitive spaces on the internet. Loan, credit card, and account comparison sites have been around for years and already hold strong positions. So is there still room for a new project? I believe there is — but only if it’s built on technologies that ensure speed, clarity, and flexibility.

This article is not just a story of how my partners and I are building yet another comparison platform. It’s mainly a breakdown of the technology decisions that, in my view, create a real competitive edge today — from framework and hosting choices, through data integration, to automation and self-healing systems.

Project Introduction

Recently with two other partners we’ve decided to give a try to run an online affiliate business - a page with comparison features for loans, deposits, credit cards, accounts and other financial services. Since there are only three of us (so far), we don’t have a hard organizational structure or named roles, but to name my role it’s a combination of CTO, solution architect and all-in-one developer. 

Since the competition is already established on the market, I’ve made a decision that the tech stack behind the project must be a combination of the best available technologies. And here comes why I’m writing this article - to show off the decision process to prove my knowledge, skills, and the way I handle challenges.

Edge & Delivery

For the best user experience, loading times and SEO results I’ve decided that SSG (Static Site Generation) is a must-have in today's times. It made me go with Next.js which combines very well with one of the best available CDNs (Content Delivery Network) on the market - Vercel.

Such simplification to go serverless instead of containers and Kubernetes is even better due to limited budget and the fact that I’m the only developer working on the application and I don’t expect any trade-offs of this choice. Also based on my experience I know that Vercel technologies provides a very, very good developer experience which is even more relevant since it’s a side project to me.

Data Manipulation

The data of financial services available on the market changes frequently (more about it later). My partners aren’t very proficient with backend technologies, and to save my time by changing stuff manually in the database(s), a UI for the data manipulation of available offers is mandatory. Instead of building one CMS (Content Management System) from scratch by myself, I offered to use Google Sheets which is a familiar interface for nearly everyone today to store the offers and is very suitable to store financial services in tables. It’s also more than enough for the predicted amount of data (even with some data relationships) and it’s possible technologically due to Google API and the fact that data is pre-rendered before user visit so the application won’t hit Sheets API limits. 

To make it more subtle technologically, I combined it with ISR (Incremental Static Regeneration) – I integrated with the spreadsheet an Apps Script that makes a call to Next.js app on each change to make on-demand data revalidation on the page that has currently changed.

Similarly, instead of reinventing the wheel I’ve decided to use the already existing headless CMS for blog and articles data. My choice was Sanity.io which provides a great UI for non-tech users. What’s even more important is that it allows for a good customization of the fields and data structure.

Analytics & Privacy

Each modern online application obviously needs analytics to gain information on what exactly users are looking for, how often, what’s most interesting for them etc. The market standard is one thing, I also personally always try to pay as much attention as reasonable to measurable metrics and evaluation. I offered to go with multiple analytics tools and for more granular data to use both cookie-based and cookieless analytics tools. 

Since not every user accepts cookie consent, but it is legal to collect some of relevant telemetry data even without it, we use high-privacy tools like Cloudflare Analytics or Vercel Analytics. Yet cookie-based analytics tools like Google Analytics&GTM, Microsoft Clarity or Meta Pixel once a user agrees for cookies allow to collect more granular behaviour data.

Since my very limited trust in external CMPs (Consent Manage Platform), I made a decision to develop my own one integrated with application. In my opinion it’s actually very easy and gives a good control over the data if ever needed. Yet it’s worth mentioning to definitely not forget about Consent Mode v2 since non-compliance may get you limited on some of the ad/analytics services.

Automation & Self-Healing

To avoid manual effort of verifying all the offers available on the market which - like I mentioned earlier - change frequently and there are many of them, I’m planning an automation system for this process. It comes with a few very big challenges.

Almost none of the financial services provide a public API which could be used to collect relevant information about the current offers, but they are available on the Internet. It forces us to use webscraping techniques which are difficult to be reliable due to the diversity of the web pages and changing page structures. Thankly we already live in the age when artificial intelligence is already a common thing and AI providers offer structural output of the processed data which makes it possible for the relevant data to make it into the database automatically.

The banking industry also often has tech to prevent bot activity on their pages, which is beatable, but requires complex solutions and it’s difficult to code. Also the server which will perform the tasks requires a good computational power and may require VPN/proxy usage if any of the providers will detect suspicious activity from one IP address.

In my opinion a system like that to meet expectations and to be more helpful than problematic needs to be a self-healing system. It requires a lot of analysis, solid architecture, granular monitoring, and actual skills to build artificially living organism. But it’s a very cool challenge that I’ll be happy to continue working on.