A platform that actively scrapes and aggregates information on popular files from various bit torrent websites and trackers. The server autonomously scrapes torrent sources and collects a time series download statistics on files.
The platform visits torrent sites to gather download statistics and collects time series download statistics of popular files. This project is setup with continuous integration and deployment, and is hosted on various cloud providers like Digital Ocean, Heroku, and Amazon Web Services.
The backend has a fully automated modularized scrape engine for scraping torrent sites utilizing Puppeteer to mimic an end user. The backend is a production ready Express server application with proxy setup, REST API, API rate limiting, compression, Redis cache of complex queries, and secure headers.
The frontend is a React application with Redux for the data store and web workers for searching locally stored data. I originally built the front end style using SASS without any CSS framework but I have now fully converted the site over to styled-components, and the website is optimized for mobile with a flexbox solution. Set up with continous integration and delivery.