Implementing beanstalk to create a scaleable web scraper

Image Credit (http://blog.hqc.sk.ca/wp-content/uploads/2012/12/Queue-2012-12-11.jpg) Queues are often used to make applications scaleable by offloading the data and process them later. In this post I am going to use BeansTalk queue management system in Python. Before I get into real task, allow me to give a brief intro of Beanstalk. What is Beanstalk? From the official website: Beanstalk […]

Develop database driven applications in Python with Peewee

It is not uncommon that most of the applications these days are interacting with database these days. Specially with RDBMS based engines( DB engines that support SQL). Like any other languages Python also provides native and 3rd party libraries to interact with database. Normally you have to write SQL queries for CRUD operations. That’s OK […]

How to automate your deployment and SSH activities with Fabric

It is not uncommon for developers to interact with remote servers. Beside FTP clients many use terminals or consoles to carry out different tasks. SSH is usually used to connect with remote servers and execute different commands; from running git to initiating a web or db server, almost every thing can be done by using […]

Write your first web crawler in Python Scrapy

The scraping series will not get completed without discussing Scrapy. In this post I am going to write a web crawler that will scrape data from OLX’s Electronics & Appliances’ items. Before I get into the code, how about having a brief intro of Scrapy itself? What is Scrapy? From Wikipedia: Scrapy (/ˈskreɪpi/ skray-pee)[1] is a […]

Write your first web scraper in Python with Beautifulsoup

Ok so I am going to write the simplest web scraper in Python with the help of libraries like requests and BeautifulSoup. Before I move further, allow me to discuss what’s web/HTML scraping. What is Web scraping? According to Wikipedia: Web scraping (web harvesting or web data extraction) is a computer software technique of extracting […]

XBMC Plugin for Online Videos

I developed an XBMC plug for a client that is installed on an Android based set-top box. It fetches Greek channels’ videos from various online video hosting sites like Youtube, Vimeo etc. Subscribed users are allowed to view videos after providing their credentials. The add-on itself is connected with remote server via REST APIs to retrieve/store […]