Scrapy is a Python module and in order to use it, we first need to install python in our Computer/Laptop.
If the python is already installed in your machine then you can Go ahead and start installing Scrapy on your Computer, If not then you can Read How to Install Python in windows, MAC, Linux, Android, or iOS operating system.
If you are not sure that python is installed or not in your computer simply open the command prompt and type the following command there.
python -V
If it installed you will see the latest python version, If not it’ll display the error there.
Installing and setting up a virtual environment
It’s always a good idea to make a virtual environment for each of your different projects. In that way, you will know which modules that you actually need in case if you want to deploy to the server or something like this.
One of the simplest ways to use virtualenv to create a virtual environment.
- In your command prompt or PowerShell type this command for virtualenv installation.
pip install virtualenv
- This command will install virtualenv and all of its dependencies.
- Now its time to set up new virtualenv. To do this use this command.
virtualenv virtual_workspace - cd to virtual_workspace that we just created.
- Now to activate type this command
Scripts\activate
How to Install Scrapy
- Make sure the virtual environment that we created is active. If not activate it.
- Use this pip command to install Scrapy in your virtual environment.
pip install scrapy - Wait till the Scrapy will install and once it does you can import and use its all of the features available.
Creating Scrapy project
- Once the scrapy installed, Now we can create the scrapy project on our computer.
- Navigate to the folder where you want to create a project in your terminal and type this command to create a project.
scrapy startproject myproject
- Simply replace the myproject with the name of the project that you want.
- Now open our newly created project folder inside the Code editor of your choice and you can start working.