• Scraping dynamic websites using Scraper API and Python
    Learn how to efficiently and easily scrape modern Javascript enabled websites or Single Page Applications without installing a headless browser and Selenium

    In the last post of scraping series, I showed you how you can use Scraper API to scrape websites that use proxies hence your chance of getting blocked is reduced. Today I am going to show how you can use Scraper API to scrape websites that are using AJAX to render data with the help of JavaScript, Single Page Applications(SPAs) or scraping websites using frameworks like ReactJS, AngularJS or VueJS. I will be working on the same code I had written in the introductory post. Let’s work on a simple example. There is a website that tells your IP, called HttpBin. If you load via browser it will tell your…

  • Things every developer should know to improve site performance

    Introduction Hundreds of websites/web apps are being launched every day for different purposes. Poorly optimized websites not only leave a bad impact on visitors but also hurt businesses. While more and more people visiting websites on their mobile devices, it is very important to make them load fast. In this post, I am going to discuss a few things that every developer should know while building or updating a website to load them fast. I will categorize these issues into two sections: Frontend and Backend. Types of Optimization Frontend:- In which CSS, JS and at times HTML is refactored and optimized one way or other. Backend:- Databases; indices, queries etc.…

  • 5 Ways EBS Snapshots Can Make Your Life Easier

    Image source Guest Post by Gilad David Maayan Amazon Web Services is the leading cloud provider, offering a range of storage options to meet the complex and evolving needs of an organization. Along with the standard cloud storage option, S3, Amazon also offers EBS, which provides flexible storage for the data contained in applications. With Amazon’s high availability, customizability and variety of integrated cloud services, you can pursue a storage tiering strategy for your data, taking advantage of a key facet of Amazon—EBS snapshots. Read on to learn how snapshots can make your life on the cloud easier. What Is Amazon EBS? Amazon Elastic Block Store (EBS) is a cloud-based…

  • Create a simple image search engine in OpenCV and Flask

    I recently started playing with OpenCV, an open-source Computer Vision library for image processing. Luckily there are Python bindings available. More luck that the guys like Adrian has done a great service by releasing both book and blog on a similar topic. So what is it all about? Well, it is simple, or I say, the simplest demonstration of using OpenCV to load an image and finding color-related to that image and show insights in a Flask based web application. This is not a state of art application neither it is currently serving the way I intend to do but hey, it is just a start, not the end. I…

  • Getting Started with AWS Automation: EBS Snapshots, AWS Lambda and AWS Systems Manager

    Image source: Pixabay Guest Post by Gilad David Maayan Whether you’ve recently adopted cloud-based services or you’ve been using them for a while, you’re likely interested in reducing the amount of manual work that managing a cloud system requires. Luckily, AWS includes a variety of automation tools and integration options that you can put to use to accomplish just that.  We’ll look at three of your options here, one simple to implement and two a bit more challenging, to give you a better idea of how they work and how you can use them to maximum benefit. EBS Snapshots The Automation of EBS snapshots is one of the easiest places…

  • Data Visualization in Bokeh – Line Graph

    Almost one and half year back I create a data visualization series with Matplotlib which was appreciated by hundreds of developers. Today I am going to start another data visualization series to create interactive graphs and charts in Bokeh. What is Bokeh From the official website: Bokeh is an interactive visualization library that targets modern web browsers for presentation. Its goal is to provide elegant, concise construction of versatile graphics, and to extend this capability with high-performance interactivity over very large or streaming datasets. Bokeh can help anyone who would like to quickly and easily create interactive plots, dashboards, and data applications. In short, Bokeh allows Python developers to create…

  • Create your first web scraper with Scraper API and Python

    Recently I come across a tool that takes care of many of the issues you usually face while scraping websites. The tool is called Scraper API which provides an easy to use REST API to scrape a different kind of websites(Simple, JS enabled, Captcha, etc) with quite an ease. Before I proceed further, allow me to introduce about Scraper API. What is Scraper API If you visit their website you’d find their mission statement: Scraper API handles proxies, browsers, and CAPTCHAs, so you can get the HTML from any web page with a simple API call! As it suggests, it is offering you all the things to deal with the…

  • Create your first ETL Pipeline in Apache Spark and Python

    In this post, I am going to discuss Apache Spark and how you can create simple but robust ETL pipelines in it. You will learn how Spark provides APIs to transform different data format into Data frames and SQL for analysis purpose and how one data source could be transformed into another without any hassle. What is Apache Spark? According to Wikipedia: Apache Spark is an open-source distributed general-purpose cluster-computing framework. Spark provides an interface for programming entire clusters with implicit data parallelism and fault tolerance.   From Official Website: Apache Spark™ is a unified analytics engine for large-scale data processing. In short, Apache Spark is a framework which is…

  • Codelobster IDE – Free PHP, HTML, CSS, JavaScript editor

    Guest Post by Stas Ustimenko In this article, we suggest you to get acquainted with the free editor of web languages – Codelobster IDE. It is presented on the software market for a long time already, and it wins a lot of fans. Codelobster IDE allows you to edit PHP, HTML, CSS and JavaScript files, it highlights the syntax and gives hints for tags, functions and their parameters. This editor easily deals with those files that contain a mixed content. If you insert PHP code in your HTML template, then the editor correctly highlights both HTML tags and PHP functions. The same applies to CSS and JavaScript code, which is…

  • Getting started with Apache Cassandra and Python

    In this post, I am going to talk about Apache Cassandra, its purpose, usage, configuration, and setting up a cluster and in the end, how can you access it in your Python applications. At the end of the post, you should have an idea of it and could start playing it for your next project. What is Apache Cassandra? According to Wikipedia: Apache Cassandra is a free and open-source, distributed, wide column store, NoSQL database management system designed to handle large amounts of data across many commodity servers, providing high availability with no single point of failure. Cassandra offers robust support for clusters spanning multiple datacenters,[1] with asynchronous masterless replication…