168 reputation
16
bio website
location London, United Kingdom
age 21
visits member for 3 years, 5 months
seen Feb 28 at 17:38

Operations and Infrastructure Engineer at Songkick. RHCSA, Pythonista, Occasional Photographer, Frequent Starbuckser


Feb
28
comment Estimating time remaining in a queue with multiple workers
Great, this seems to be working pretty well. Thanks.
Feb
10
comment Lexical Analysis without regular expressions
@kevincline Why do people row across the Atlantic when there are perfectly good planes in the sky?
Feb
10
comment Lexical Analysis without regular expressions
I didn't mean C couldn't do regex's, I meant it has more powerful features for doing this sort of stuff. I'd imagine it's easier to build an advanced and performant lexer in C than a higher level language.
Sep
7
comment What factors should I be looking at to increase performance in image resizing?
We don't use a hosting company, we're on co-located servers so we have full admin. All the Python image libraries (I've seen) are blocking and getting them to work asynchronously with our Tornado servers would probably be more effort than required. We'll also probably end up with dedicated image processing servers so we'll need Gearman for that anyway.
Sep
7
comment What factors should I be looking at to increase performance in image resizing?
@Karolis Yes, but again to verify the image dimensions (We would verify on resolution and file size) we'd probably want to do that outside the Tornado IO loop. Also we have 7 different image resizes to do + the original image, and $Bandwidth > $CPU. We have to upload the files to S3 anyway so we're trying to keep our bandwidth charges down.
Sep
7
comment What factors should I be looking at to increase performance in image resizing?
@Karolis Not reliably enough, anyone can fake data sent to the server. With tools like Firebug you can even modify Javascript etc. so our users could edit the code that resizes the images for the thumbnails and every image size would be full resolution. Also the only upload implementations we have is HTML5 or forms for HTML4 so we'd need server-side resizing for HTML4. Add to that, most people using the site will be uploading lots of images at once, we don't want to crash out peoples browsers.