Programming Category Archives

Integrating Rollbar to Scrapy

Scrapy logs

For the past three (3) years, I’ve worked on multiple Scrapy projects with lots of spiders. Most of them are scheduled via scrapyd, a JSON API to schedule spiders. Sometimes, this spiders go kaput for various reasons such as change in layout, change in URL, being blacklist, among others. Checking the logs for them one-by-one can be time-consuming – so here’s where Rollbar comes in. Rollbar is …

MRT3 website, hacked by AnonymousPH?

MRT3 has been all over the news lately, from the long queues, not enough trains to accommodate the riding public, the extortion scandal exposed by a diplomat – among others. Unfortunately, most, if not all of them are bad news. That’s what made me think on what app to build to help the riding public. I tried to brainstorm. First thing to do was to do an …