Fotografar ou filmar o terminal

Hoje no terminal da Lapa um aviso sonoro informou que não se pode fotografar ou filmar o terminal. Alguém sabe o porquê?

Vendedores ambulantes no trem de São Paulo

Semana passada eu comecei a pegar diariamente o trem aqui em São Paulo. Da estação Ceasa até a Vila Olímpia, e no final do dia voltando por esse caminho. Às sextas-feiras, e ocasionalmente se perder a minha carona, o caminho é um pouco diferente. Pego o ônibus até a estação da Lapa, baldeio na Presidente Altino, e desço na Vila Olímpia mesmo.

Estranhamente durante esses dez dias eu vi poucos vendedores ambulantes, mas não tinha percebido até ontem. Um vendedor entrou no vagão, anunciando seu amendoim (R$ 0,50), e eu, sem fome ou caridade, o ignorei. Entretanto notei que na abertura das portas, ele manteve o corpo dentro e apenas pescoçou para visualizar rapidamente nos dois lados da estação, e correu até o vagão de trás.

Ali percebi que o vendedor procurou por algum segurança na plataforma, e aí caiu a ficha como não tinha visto, durante essas duas semanas, outra pessoa vendendo algo, e devem ser coibidos ativamente.

Plugin do Eclipse para editar scripts bash

Meu projeto atual tem arquivos .php, .pm e .sh no mesmo diretório. Aos poucos fui achando os plugins para o Eclipse, e às vezes até múltiplos para a mesma linguagem de programação. E para bash, eu achei o ShellEd. Por enquanto está muito bom. Mesmo o projeto tendo sido criado para PHP, os arquivos .sh estão sendo abertos pelo ShellEd.

O infocídio de Mark Pilgrim

Meu ídolo e profílico autor de sites/livros, como diveintopython[3].org e diveintohtml5.org, Mark Pilgrim, aparentemente cometeu infocídio ontem. Todas as suas páginas estão retornando o cabeçalho http 410 Gone (The requested resource / is no longer available on this server and there is no forwarding address. Please remove all references to this resource), e ele também removeu as contas no Twitter e Google+. Na vida real ele está muito bem, o que me deixa muito feliz.

Mark Pilgrim

Twittando do meu novo

Mr. Gaigalas em meta-tweet achou uma pérola: http://twitter.com/alganet/status/119586562651594752 tem até punchline.

Clojure script to parse IIS logs from a directory and show slowest pages

Just having some fun today. The code runs actually pretty fast: It reads over 1.5 million lines from 1.1GB of logs and sorts them by time-taken in 5 seconds. I bet there’s room for improvement but it was enough for my needs.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
(ns iis_log
  (:require clojure.contrib.duck-streams clojure.contrib.io clojure.string clojure.test))
 
(def #^{:doc "Where the log files reside."} logs-directory "D:/userdata/jnetto/Desktop/zina_inside/W3SVC93479707")
 
(def #^{:doc "How many lines to show from logs."} top-to-show 25)
 
;; IIS log lines starting with # are comments
(defn is-not-comment?
  [^String s]
  (not= (.substring s 0 1) "#"))
 
(defn remove-comments
  [file-contents]
  (filter is-not-comment? file-contents))
 
;; Values on logs are separated by a space. Therefore, "-" is shown as a value when there is nothing there, instead of a space,
;; which would mess up the separator.
(defn nil-if-hyphen
  [s]
  (if (not= s "-") s))
 
(defstruct token :uri :time-taken :username)
 
(defn relevant-line-info
  "Returns a map with the path + querystring, time-taken and username from a log line."
  [line]
  (let [line-tokens (clojure.string/split line #" ")
        cs-uri-stem (nth line-tokens 6)
        cs-uri-query (nth line-tokens 7)
        cs-username (nth line-tokens 9)]
    (struct-map token :uri (str cs-uri-stem (nil-if-hyphen cs-uri-query))
                      :time-taken (read-string (nth line-tokens 20))
                      :username (nil-if-hyphen cs-username))))
 
(def accesses (ref []))
 
(doseq [child (.listFiles (clojure.contrib.io/file logs-directory))]
  (println "Processing" child) ;; Just for debugging purposes, to show something is happening
  (let [file-contents (clojure.contrib.duck-streams/read-lines child)
        file-without-comments (remove-comments file-contents)
        tokens (map relevant-line-info file-without-comments)]
    (dosync (alter accesses concat (take top-to-show tokens))))) ;; We 'take 'top-to-show here as an optimization. If we will print
                                                                 ;; only top-to-show, why take more from each file?
 
;; Until here it's actually pretty fast. 'sort-by and 'reverse are actually the slowest parts of this script.
(doseq [line (take top-to-show (reverse (sort-by :time-taken @accesses)))] 
  (println line))

Pong in Clojure

Me and Cesar Canassa decided to create a game. I suggested a roguelike, which we will build eventually, but as both newbies in game development, we are going to start small.

With Pong.

In the last couple months I have been teaching myself some Clojure, and said to Canassa I would like to try writing Pong using it. Now here’s where his personality shines: He had never written a single line of Clojure, but said “sure!”. Later on, while researching about functional programming, he mentioned how he had read in several places how one of the few areas of development where this paradigm is not recommended is… game development.

Ooops. Too late.

Beers in hands on a Saturday night, we promised to get something running. The plan didn’t go well. Our lack of Java and Clojure knowledge, and our lack of intelligence after ten beers got in the way. We managed to open a black window and draw white lines on it, but at the same time there was a sense of struggling, a fight against the language and its libraries. We were driving the car down the hill again and again.

Somehow on Sunday, Canassa created a repository on Bitbucket for Clojure Pong, and the first changesets started to roll in.

Today we finished the game. We thought about adding sound to the ball collision on racquets and walls but the game already does much more than training and learning code should do.

We are looking for criticism, specially on idiomatic Clojure. We tried to replace the usual maintaining of a “global” state in variables that is possible in other languages and paradigms, with a “game” map that flies around functions. It is a simple approach, but probably contains a lot of unnecessary overhead. Not in terms of performance, but in terms of function responsibility.

For example, the racquet collision function takes the whole game as parameter. Feels like cheating: Instead of using variables sprinkled to determine what is happening, the values are keys of “game”. It feels like we are still using state, but it is concentrated on a map structure.

But it works, and it is also probably more fun than training and learning code should be. There is a pause screen, the ball rolls slightly angled on game start and scoring, and a FPS counter.

We are going to use some of these stuff on our next project. Breakout is coming, and I will post again when we have some news.

Crowd review a resume

Another one of my ideas which I won’t implement. Ever had doubts if a candidate CV/resume is a good fit for the job? Pay US$ 19.90, upload the CV to a website, and let 10 people say yes/no. I bet recruiters would love this tool.

Let me expand a little bit:

Recruiting a person with a completely different skill set than yours is hard, right? Imagine the troubles a owner of a soon-to-open delicatessen store feels when hiring a baker. Or a recruiter trying to avoid an embarrassment when forwarding a CV she’s not so sure about.

Well, why not let people who are already professionals in a field tell you if the candidate has a good curriculum vitae? Post a job description, upload/paste CVs, and receive good feedback on it.

What’s in it for the professionals? Money. One dollar per rating, maybe more if the professional has a proven track of good rating on the website. More if you tweak the paid amount and the website revenue cut. At US$ 19.90, that’s good for 10 one dollar ratings, or 5 two dollars ratings (with 10 left for the website).

Quis custodiet ipsos custodes? Apply metamoderation, and recommendation, for example CV uploaders can invite known professionals to join the website. Don’t let invited professionals rate CVs sent by people close to whoever brought them to the website. Gently remind CV uploaders to come back to the website and say if the received ratings were good.

Let professionals suggest questions to ask the candidate. Rate/metamoderate questions.

Let professionals comment on the CVs. For example: “According to the job description, I recommend the candidate knows SOX.”

Hiring is so hard I bet the 20 bucks price could be increased, for example it starts at US$ 20, but recruiters could ask for an specific professional for her opinion, whom would set her price per rate. Or +US$ for expedite ratings (in 2/6/12/24 hours). Maybe that’s too complicated for pricing, though.

If you ever build this website, let me know 🙂

memcached monitoring and statistics

I wanted to know, overtime, how many cache hits and misses my instance was getting. Searched a lot and finally found this perl script:

http://code.google.com/p/memcache-top/

Usage is very simple. Save the file, give it execution permissions, then just run:

./memcache-top.pl –commands –cumulative

Here’s an output example:

memcache-top.pl output example

Seems like I’m doing fine, caching the right parts! Hit over 80% and only half of my allocated memcached memory being used… (that’s because I had recently restarted it, so usually it keeps filled).

Apache + mod_wsgi + Django + lighttpd

I’ve written how to configure Apache, mod_python and Django and how to put lighttpd behind Apache.

Recently I decided to host my most visited website on a different VPS provider[1], and started a quest to update my knowledge about Django deployment. I did things differently this time, using mod_wsgi (the recommended way of deploying Apache and Django), and configuring Apache behind lighttpd for dynamic content (in other words, lighttpd will serve static media).

I did everything below in the last couple days, and did not wrote things as I was doing it, because it involved a lot of experimentation (trial and error) for me. As such, I am writing this article based on memory and checking my config files. If you encounter any problems, please leave a comment and I will clarify any omissions.

Here’s how to do it:

Install the usual suspects

I choose Ubuntu as my Linux distro, and installation of anything is a breeze on it. sudo apt-get install package-name. This part is well covered around the web, so I won’t comment in details how it’s done. Sufficient to say, some of the packages I’ve installed were apache2, libapache2-mod-wsgi, and lighttpd.

Configure Apache and mod_wsgi to load your project

Since lighttpd will act as the primary server for my domain, I decided to move Apache to port 81:

sudo vi /etc/apache2/ports.conf

Overwrite existing ip:port lines with these:

NameVirtualHost 127.0.0.1:81
Listen 81

Where you put your own Python modules on newer Ubuntu installations has changed to /usr/local/lib/python2.6/dist-packages/. Therefore, I’ve uploaded Django, my project and other necessary modules (which weren’t installed by apt-get) to this directory, leaving me with the following structure:

/usr/local/lib/python2.6/dist-packages/django/
/usr/local/lib/python2.6/dist-packages/my_project_name/

The mod_wsgi documentation has an excellent article on Django integration, but it’s fairly length. You should read it anyway, since there are lots of options that you might want to use. Here’s a cheatsheet:

Create a document root for your domain name:

sudo mkdir /var/www/example.com

Create the file which will be loaded by mod_wsgi with your project configuration:

sudo mkdir /usr/local/lib/python2.6/dist-packages/project_name/apache/
sudo vi /usr/local/lib/python2.6/dist-packages/project_name/apache/django.wsgi

With these contents:

import sys
import os

sys.path.append(‘/usr/local/lib/python2.6/dist-packages’)
os.environ[‘DJANGO_SETTINGS_MODULE’] = ‘project_name.settings’

import django.core.handlers.wsgi

application = django.core.handlers.wsgi.WSGIHandler()

Create a domain configuration file for Apache:

sudo vi /etc/apache2/sites-available/example.com

With these contents:

<VirtualHost 127.0.0.1:81>
ServerName example.com
ServerAdmin [email protected]

DocumentRoot /var/www/example.com

Alias /media/ /usr/local/lib/python2.6/dist-packages/django/contrib/admin/media/
<Directory /usr/local/lib/python2.6/dist-packages/django/contrib/admin/media>
Options -Indexes
Order deny,allow
Allow from all
</Directory>

Alias /project_media_dir/ /usr/local/lib/python2.6/dist-packages/project_name/templates/project_media_dir/
<Directory /usr/local/lib/python2.6/dist-packages/project_name/templates/project_media_dir>
Options -Indexes
Order deny,allow
Allow from all
</Directory>

WSGIScriptAlias / /usr/local/lib/python2.6/dist-packages/project_name/apache/django.wsgi
WSGIDaemonProcess example.com
WSGIProcessGroup example.com

<Directory /usr/local/lib/python2.6/dist-packages/project_name/apache>
Order deny,allow
Allow from all
</Directory>
</VirtualHost>

Activate it:

cd /etc/apache2/sites-enabled/
sudo ln -s ../sites-available/example.com

Configure lighttpd to proxy non-static media requests to Apache

I used MySQL Performance Blog’s “Lighttpd as reverse proxy” article as a basis for my own configuration. Therefore, we’ll have an url http://example.com/server-status, which will require authentication, enabling us to see Apache’s server status.

Create a directory for error logs:

sudo mkdir /var/log/lighttpd/example.com

Create a domain file configuration for lighttpd:

sudo vi /etc/lighttpd/conf-available/20-example.com.conf

With these contents:

server.modules += ( “mod_auth”,
“mod_status”,
“mod_proxy”,
)

$HTTP[“host”] =~ “(^|\.)example\.com$” {
$HTTP[“url”] !~ “\.(js|css|gif|jpg|png|ico|txt|swf|html|htm)$” {
proxy.server = ( “” => (
( “host” => “127.0.0.1”, “port” => 81 )
))
}

server.document-root = “/var/www/example.com/”
server.errorlog = “/var/log/lighttpd/example.com/error.log”
dir-listing.activate = “disable”

auth.backend = “htpasswd”
auth.backend.htpasswd.userfile = “/var/www/.htpasswd”
auth.require = ( “/server-status” => (
“method” => “basic”,
“realm” => “status”,
“require” => “valid-user”
)
)
}

There are lines worthy of mention in the configuration above:

$HTTP[“host”] =~ “(^|\.)example\.com$” {

This will wrap the directives inside to only apply for example.com requests.

$HTTP[“url”] !~ “\.(js|css|gif|jpg|png|ico|txt|swf|html|htm)$” {
proxy.server = ( “” => (
( “host” => “127.0.0.1”, “port” => 81 )
))
}

These will send any requests for documents not ending in the specified extensions to ip 127.0.0.1, port 81, where Apache lives. Essentially, everything which is static content (or more accurately, specified by the | separated regular expression), will be served by lighttpd.

cd /etc/lighttpd/conf-enabled/
sudo ln -s ../conf-available/example.com.conf

Tell the filesystem where you project and Django’s admin static content are located:

sudo ln -s /usr/local/lib/python2.6/dist-packages/django/contrib/admin/media/ /var/www/example.com/media
sudo ln -s /usr/local/lib/python2.6/dist-packages/project_name/templates/project_media_dir/ /var/www/example.com/project_media_dir/

Finally, restart everything so the newest configuration can be applied

sudo /etc/init.d/apache2 restart
sudo /etc/init.d/lighttpd restart

So, what actually happens?

When a visitor goes to your website (example.com), the request will hit lighttpd first. If the document path does not end with a string in our list of static content extensions, the request will be proxied to Apache on port 81, otherwise lighttpd will serve itself.

And that’s it, if my memory is correct. Did I miss anything? Comment at will.

[1] Linode, if you’re curious. Mainly because bandwidth is cheaper. If you’re looking for a Linode referral, Linode discount code or Linode promotion code, sign up using this link to credit me as referral. Thanks 🙂