Last week I was working on a web scraper for a client who needed to get around a million records from a real estate website. After a certain level, the scraper stopped working and the reason was I forgot to put certain checks as I was expecting the client would not go for that route but he DID! A few days back I shared a post about how to write a basic scraper in Python by using Beautifulsoup. In this post, I am going to discuss how to make your scraper more foolproof and user-friendly for non-technical people. 1- Check 200 status code It is always good to check the…
-
-
3 Free SEO tools you should use for your next blog post
Blogging is not only the best way to express your thoughts and feelings in words but also good to reach your potential audience. These days the best way to reach your audience is to let people visit your site by finding you via Google. How do they find you? Well, they write something(called Keyword) and search about it and click on the link they think would be more helpful and interesting. After all this is how Google or any search engine work right? Many software developers who maintain blogs usually don’t care about their audiences( I’m one of them), they are excited and passionate enough that they just share their findings…
-
Write your first web scraper in Python with Beautifulsoup
Ok, so I am going to write the simplest web scraper in Python with the help of libraries like requests and BeautifulSoup. Before I move further, allow me to discuss what’s web/HTML scraping. What is Web scraping? According to Wikipedia: Web scraping (web harvesting or web data extraction) is a computer software technique of extracting information from websites. This is accomplished by either directly implementing the Hypertext Transfer Protocol (on which the Web is based), or embedding a web browser. So use scraping technique to access the data from web pages and make it useful for various purposes (e.g: Analysis, aggregation etc). Scraping is not the only way to…
-
How I wrote my first Machine Learning program in 3 days
A few weeks back I was intrigued by Per Harald Borgen’s post Machine Learning in a Week which oversimplified the entire learning and implementing a Machine Learning algorithm in a week on a real dataset. He laid down the framework that how as a programmer one can get into ML thing without getting worried about heavy maths and statistics. It was a good excuse to give this ML thing a chance which I had been trying to do for many years after completing Coursera course. Alright. So on this Monday I started my mission. I had to find out some good dataset to achieve the task. Initially I wanted to use NASA’s meteorites…
-
Mbstring not found error while installing mailparse PHP extension
Today I had to install mailparse via PEAR. On installing I came across one of the errors: mbstring not found despite of it was right there. One of the simplest option to fix it is given below. Though I am working on MAMP but it will work with anyone since it’s part of PHP Source. Go to your PHP Source Library. In my case it is: /Applications/MAMP/bin/php/php7.0.8/include/php/ext/mbstring/libmbfl/mbfl and add following lines in mbfilter.h #undef HAVE_MBSTRING #define HAVE_MBSTRING 1 and then run PEAR command again pear install pecl/mailparse If all goes well it should give something like: Build process completed successfully Installing '/Applications/MAMP/bin/php/php7.0.8/lib/php/extensions/no-debug-non-zts-20151012/mailparse.so' install ok: channel://pecl.php.net/mailparse-3.0.1 configuration option "php_ini" is not set to…
-
Currently Reading – Chrome extension about social book reading
I love reading books, in fact lots of books. I’am sure many of you read books as well. Books on different topics and subjects help you to expand the horizon of knowledge and creativity. Often it happens that you extract an idea from a book and you give your own touch in it and produce a product which people love to use. In the age of social media when people love to let others know about their daily activities by updating a status, it ‘s not surprising that there are such social sites available which let your friends know about what you’re reading. Goodreads is one of them which…
-
Online JSON Diff Viewer
Just came across this awesome tool to compare JSON. Yeah, at time it’s needed to compare them. Check this out(http://json-diff.com/) The thing I like most was the interface.
-
Develop your first Facebook messenger bot in PHP
(Credit: Independent.co.uk) Facebook recently announced a Bot platform for it’s Messenger which provides businesses and individuals another way to communicate with people. What is a Chat bot? A computer program designed to simulate conversation with human users, especially over the Internet. Chat bot in PHP When I heard of it, my very first thought was to a bot in PHP. I started to find some SDK in this regard released by Facebook but none was present. I headed over to documentation which provided good information for starters. Ok! so without wasting further time, let’s build our first Bot Timebot In order to create an Fb bot you will need two things to…
-
Harsha Bhogle about excellence and achievements
I always considered Harsha just a commentator but O boy! what a marvelous speaker he is. He is sharing nuggets to achieve things in life.
-
The 2% mindset.