Data Normalization / WebScraping
Challenge
Halfway through developing the Instant Replay App, I encountered a major roadblock: the data was unreliable. The API I initially chose suffered from critical issues — including missing or incorrect dates, broken or incomplete player images, and inconsistent naming conventions — all of which made it impossible to create a smooth and accurate user experience.
Additionally, the API charged per request, and retrieving a single game required three separate calls: one for the team, one for the season, and one for the game itself. That added up quickly and wasn’t sustainable.
To solve this, I took full ownership of the data pipeline. I built a custom backend server and a dedicated MongoDB database, giving me complete control over the structure and quality of the data. This allowed me to normalize and cache everything locally — reducing three paid API calls down to one and ensuring consistent, dependable access to game data moving forward.
Tech Stack
Python | NodeJS | BeautifulSoup | Selenium | MySQL | CLI Scripting | MongoDB | WebScraping | API Development | Data Normalization | RapidAPI
Solution
To meet budget constraints and cut down on development costs, I leveraged Calendly’s free tier to interface directly with the client’s Google Calendar and securely verify appointment availability. To eliminate the need for expensive automation tools like Zapier, I built a custom Node.js server hosted on DigitalOcean, enabling secure communication between the client’s WordPress site and the Calendly REST API, while also automating appointment notifications.
What began as a hardcoded Gutenberg block was later refactored into a fully functional WordPress plugin for long-term flexibility and ease of use by the client. The plugin integrates seamlessly into the existing site while maintaining a minimal project footprint.
The plugin includes an intuitive admin interface that empowers the business owner to create, update, edit, and delete services independently. It’s also built with scalability in mind, supporting multiple service pages across different parts of the website—all managed from a single plugin.
Would you like to start a project with us?
Your website should be an investment, not an expense. Contact us today to see how we can help your business thrive!