Getting Scrapy, a web crawler framework, working on Windows 8.1

2015-01-06 04:31:00 +0000 - Written by Carl Burks

First get setuptools installed to use PyPi:

https://pypi.python.org/pypi/setuptools#windows-powershell-3-or-later

</a>

This assumes you have python aliased to your instance of python in your environment: (Invoke-WebRequest https://bootstrap.pypa.io/ez_setup.py).Content | python -

https://pip.pypa.io/en/latest/installing.html

python.exe .\get-pip.py

How can I tell that I have python setup properly in my environment?

(Get-ChildItem -Path env:* | Where-Object -Property Name -Contains -Value "Python").Length -gt 0

What does this return?

How about this:

(Get-ChildItem -Path env:Path).Value.Contains("Python")

If one doesn’t return true then you might need to setup python.

https://docs.python.org/2/using/windows.html

Adding to Environment:

C:\Windows\System32\SystemPropertiesAdvanced.exe

Edit Path variable and add a semicolon and your path to Python.

Running the first command is going to fail because your system probably isn’t setup to run scripts from the Internet. This is probably a good thing. You can download and save the script before executing it, or change your execution policy to allow scripts from unknown sources. That last option is a mite dangerous.

You’ll need to unzip it so if you’ve got 7zip installed you can probably do something like this:

& 'C:\Program Files\7-Zip\7z.exe' "X" .\setuptools-11.3.zip

Keeping modules in order might require something like Virtual Environments.

pip install virtualenv

http://docs.python-guide.org/en/latest/dev/virtualenvs/

pip install scrapy

if you haven’t installed it already you will need PyWin32

http://sourceforge.net/projects/pywin32/