Python – Installing scrapy and Creating First Project

Scrapy is an open source and collaborative framework for extracting the data you need from web pages.

To install scrapy:

open command prompt as administrator

type in console line:

pip install scrapy

create a project

In command prompt navigate to the directory that you want your web web crawler project to be stored. Then enter:


scrapy startproject tutorial

This will create a tutorial directory with the following elements

    scrapy.cfg            # deploy configuration file

    tutorial/             # project's Python module, you'll import your code from here          # project items definition file    # project middlewares file      # project pipelines file       # project settings file

        spiders/          # a directory where you'll later put your spiders


%d bloggers like this: