Viewing posts by lee

Maths of RSA for dummies

Background knowledge

The reader is assumed to know some basic mathematics, but nothing too complicated. These assumed concepts include the knowledge of what a prime number is, what taking the modulus of a number means and the ability to read basic mathematical notation.

Read more

Peeling the onion

This is an attempt of a mid-level overview of how Tor combines public-key encryption and symmetric-key cryptography to allow it to function. I should make it clear that, I’m not an expert and these ideas really are just a compendium of things I’ve read here and there, and ...

Read more

Launch AIrVPN without password prompt in linux

AirVPN is a great VPN service that works on many platforms including linux. If you are using their client on linux, you may have found yourself annoyed having to enter your admin password each time you launch the client. There is a way to get around this in linux.

Read more

Adapting Scrapy's images pipeline to use Azure Storage

Although it's not documented, the Scrapy images pipeline supports storing your downloaded images in an Amazon S3 bucket. This is very useful especially when scraping is over distributed nodes and you want a centralized image store.

Read more

How does Scrapy react to a blocked pipeline?

These are just some notes on how Scrapy reacts to a blocked pipeline.

Read more