Python Web Scraping Best Practices

I've been scraping the web a lot as part of my quest to find the best scratch-off lottery ticket. I want to share some of the practices that I've discovered. I'll do so by posting the code itself in a style of "literate programming". (It's actually just heavily commented code.…