Getting Scrapy, a web crawler framework, working on Windows 8.1

2015-01-06 04:31:00 +0000 - Written by Carl Burks

First get setuptools installed to use PyPi:


This assumes you have python aliased to your instance of python in your environment: (Invoke-WebRequest | python -

python.exe .\

How can I tell that I have python setup properly in my environment?

(Get-ChildItem -Path env:* | Where-Object -Property Name -Contains -Value "Python").Length -gt 0

What does this return?

How about this:

(Get-ChildItem -Path env:Path).Value.Contains("Python")

If one doesn’t return true then you might need to setup python.

Adding to Environment:


Edit Path variable and add a semicolon and your path to Python.

Running the first command is going to fail because your system probably isn’t setup to run scripts from the Internet. This is probably a good thing. You can download and save the script before executing it, or change your execution policy to allow scripts from unknown sources. That last option is a mite dangerous.

You’ll need to unzip it so if you’ve got 7zip installed you can probably do something like this:

& 'C:\Program Files\7-Zip\7z.exe' "X" .\

Keeping modules in order might require something like Virtual Environments.

pip install virtualenv

pip install scrapy

if you haven’t installed it already you will need PyWin32