Lopez Crawl Directives

Lopez Crawl Directives is a small language that tells Lopes exactly what to do in the Web. Fear not! If you are familiar with HTML, regular expressions and CSS, the Directives should be relatively straightforward to get started. If not, there are plenty of StackOverflow answers on each small topic and many tutorials on each of these topics.

These are all the topics on Lopez Crawl Directive to take you from zero to hero (it’s super small), in a good order:

• Boundaries and seeds
• The module system
• Selection rule sets
• Set-variables and configuration

Besides that, you will need some reference lists and cheat sheets to help you around writing your own Directives. These are listed below:

• List of aggregators
• List of extractors
• List of transformers
• The Standard Library