OK, so I was kind of bored last week so thought of coming up something anyway, even if it is useless. So, I allocated a few hours and came up with ScrapeGen.
What is ScrapeGen?
It is a simple Python-based command-line tool that generates python web scrapers based on rules and details entered in a YAML file. When it runs, it generates a new file. Rules turned to separate functions which then are called to main parsing method. View the Demo:
Why is needed?
Such kind of tool could be good for companies and individuals who write many parsers and hardcode the rules within the .py files. Imagine the rule is changed and you gotta write it again. What if you just change the rule in YAML and the scraper starts working? Sounds cool, right?
The tool is available on Github
Planning to write a book about Web Scraping in Python. Click here to give your feedback
