Tuesday, March 25, 2008
Friday, February 1, 2008
Fun at the turing tar pit
<fumanchu> http://www.yosefk.com/blog/fun-at-the-turing-tar-pit.htmlSeriously, Fun at the turing tar pit is a great read.
*cough*Twisted*cough*
<lakin> lol
<jtate> fumanchu: definitely twisted
<fumanchu> yesterday our CEO sat down next to me and said, "even Glyph says it doesn't scale" :P
<lawouach> :)
<jamwt> haha.. great post
<fumanchu> I love the phrase "turing tar pit" -- brilliant
Tuesday, January 15, 2008
Mozilla Hired Humanized Founders
Wow. I never even knew about this company, called Humanize. But Mozilla just hired 3 of the main guys [1]. I just checked out Songza [2] and it's got a very simple, very cool interface. Not to mention that it lets you just listen to music. The interface is so simple, so clean, I immediately started using it.
It's always fun to learn about cool companies who are making great products. Specifically I google aza raskin and found his wikipedia page, which linked to a very good article "never use a warning when you mean undo" [3].
[1] - http://www.techcrunch.com/2008/01/15/breaking-mozilla-buying-humanized/
[2] - http://www.songza.com/
[3] - http://www.alistapart.com/articles/neveruseawarning
It's always fun to learn about cool companies who are making great products. Specifically I google aza raskin and found his wikipedia page, which linked to a very good article "never use a warning when you mean undo" [3].
[1] - http://www.techcrunch.com/2008/01/15/breaking-mozilla-buying-humanized/
[2] - http://www.songza.com/
[3] - http://www.alistapart.com/articles/neveruseawarning
Monday, January 14, 2008
How to simulate webfaction's apache/django setup in ubuntu.
I was recently interested in profiling my personal website which is a django application. Unfortunately, I use a shared hosting account on Webfaction. I haven't asked but I'm sure they don't appreciate having their servers hammered by an ab or siege process. Not to mention that it would use up your own bandwidth unecessarily. Also I wanted to be able to quickly and easily see the affects that various tweaks might have on performance and memory usage of my apache processes.
So I set about trying to replicate the basic apache setup that they provide. Typically, Webfaction provides the end-user with their own apache instance and config which runs the long running process, and then Webfaction proxies to this application for requests that come in on this domain. I don't have access to their proxy-server configuration so I haven't yet bothered setting one up, and for the purposes of this post will assume that profiling the application using just the long-running process apache to which the user has a configuration is good enough.
First, I set about installing mod_python, which (as always with Debian based systems) was dead simple:
Then I grabbed a copy of my current config:
The ServerRoot for apache2 in ubuntu is "/etc/apache2". All of the modules can be loaded using two lines. The following example is for the mime_module. I would suggest you follow this method when there is an appropriate .load and .conf file in /etc/apache2/mods-available/. Although updating the module's .so location is possible some (like the mime) module require some extra configuration:
Others, like the log_config_module are built-in. Which means our apache setup won't be exactly like webfaction but it'll be close. Also, for those modules which don't have an entry in the mods-available directory, you'll need to update their path. The log directory locations need to be updated as do the user and group that the server runs as and you'll need to add a line which specifies where to put the PIDFile. To finish, you'll need to modify the PythonPath line to update this configuration to where your code lives. My final result was:
The syntax of the config file can be checked with a single command:
And the server can be started and stopped easily as well using:
So I set about trying to replicate the basic apache setup that they provide. Typically, Webfaction provides the end-user with their own apache instance and config which runs the long running process, and then Webfaction proxies to this application for requests that come in on this domain. I don't have access to their proxy-server configuration so I haven't yet bothered setting one up, and for the purposes of this post will assume that profiling the application using just the long-running process apache to which the user has a configuration is good enough.
First, I set about installing mod_python, which (as always with Debian based systems) was dead simple:
sudo apt-get install libapache2-mod-python
Then I grabbed a copy of my current config:
ServerRoot "/home2/lakin/webapps/lakin_weckers_net/apache2"
LoadModule python_module /home2/lakin/webapps/lakin_weckers_net/apache2/modules/mod_python.so
LoadModule cgi_module /home2/lakin/webapps/lakin_weckers_net/apache2/modules/mod_cgi.so
LoadModule log_config_module /home2/lakin/webapps/lakin_weckers_net/apache2/modules/mod_log_config.so
LoadModule mime_module /home2/lakin/webapps/lakin_weckers_net/apache2/modules/mod_mime.so
LoadModule env_module /home2/lakin/webapps/lakin_weckers_net/apache2/modules/mod_env.so
LoadModule dir_module /home2/lakin/webapps/lakin_weckers_net/apache2/modules/mod_dir.so
LoadModule autoindex_module /home2/lakin/webapps/lakin_weckers_net/apache2/modules/mod_autoindex.so
Listen 3077
User lakin
Group lakin
LogFormat "%{X-Forwarded-For}i %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
CustomLog /home2/lakin/webapps/lakin_weckers_net/apache2/logs/access.log combined
Errorlog /home2/lakin/webapps/lakin_weckers_net/apache2/logs/error.log
ServerLimit 2
DocumentRoot /home2/lakin/webapps/lakin_weckers_net/lcw_trunk/htdocs
<location>
SetHandler python-program
PythonHandler django.core.handlers.modpython
SetEnv DJANGO_SETTINGS_MODULE lcw.settings
PythonPath "['/home2/lakin/webapps/lakin_weckers_net/lcw_trunk', '/home2/lakin/webapps/lakin_weckers_net/swim_trunk', '/home2/lakin/webapps/lakin_weckers_net/django_trunk'] + sys.path"
PythonDebug On
</location>
<locationmatch>
SetHandler None
AllowOverride None
</locationmatch>
The ServerRoot for apache2 in ubuntu is "/etc/apache2". All of the modules can be loaded using two lines. The following example is for the mime_module. I would suggest you follow this method when there is an appropriate .load and .conf file in /etc/apache2/mods-available/. Although updating the module's .so location is possible some (like the mime) module require some extra configuration:
Include /etc/apache2/mods-available/mime.load
Include /etc/apache2/mods-available/mime.conf
Others, like the log_config_module are built-in. Which means our apache setup won't be exactly like webfaction but it'll be close. Also, for those modules which don't have an entry in the mods-available directory, you'll need to update their path. The log directory locations need to be updated as do the user and group that the server runs as and you'll need to add a line which specifies where to put the PIDFile. To finish, you'll need to modify the PythonPath line to update this configuration to where your code lives. My final result was:
ServerRoot "/etc/apache2"
LoadModule python_module /usr/lib/apache2/modules/mod_python.so
Include /etc/apache2/mods-available/cgi.load
Include /etc/apache2/mods-available/mime.load
Include /etc/apache2/mods-available/mime.conf
Include /etc/apache2/mods-available/env.load
Include /etc/apache2/mods-available/dir.load
Include /etc/apache2/mods-available/dir.conf
Include /etc/apache2/mods-available/autoindex.load
Include /etc/apache2/mods-available/autoindex.conf
Listen 8080
User lakin
Group lakin
LogFormat "%{X-Forwarded-For}i %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
CustomLog /home/lakin/Desktop/lcw_test/logs/access.log combined
Errorlog /home/lakin/Desktop/lcw_test/logs/error.log
PidFile /home/lakin/Desktop/lcw_test/apache2.pid
ServerLimit 2
DocumentRoot /home/lakin/Projects/lakin.weckers.net/trunk/htdocs
<Location "/">
SetHandler python-program
PythonHandler django.core.handlers.modpython
SetEnv DJANGO_SETTINGS_MODULE lcw.settings
PythonPath "['/home/lakin/Projects/lakin.weckers.net/trunk', '/home/lakin/Projects/swim/trunk', '/home/lakin/Projects/django/trunk'] + sys.path"
PythonDebug On
</Location>
<LocationMatch "\.(css|js|jpg|gif|png|ico)$">
SetHandler None
</LocationMatch>
The syntax of the config file can be checked with a single command:
apache2 -t -f webfaction_django_lcw.conf
And the server can be started and stopped easily as well using:
apache2 -f webfaction_django_lcw.conf -k start
apache2 -f webfaction_django_lcw.conf -k stop
Wednesday, December 26, 2007
Free hosting for facebook apps
I recently stumbled upon free hosting for facebook apps. [1] One has to wonder what motivates companies to provide services such as these. It could be sheer benevolence, but I suspect there are some statistics and information mining that must be the driving factor. In any case it's something to be aware of.
[1] - http://www.joyent.com/developers/facebook/
[1] - http://www.joyent.com/developers/facebook/
Ever wished you had taken that college course?
A physics professor at the age of 71 has become an internet web star, because of his freely available physics lectures. [2]
From the website:
Wow. I may have found my new years resolution.
[1] - http://ocw.mit.edu/OcwWeb/web/home/home/index.htm
[2] - http://www.nytimes.com/2007/12/19/education/19physics.html?_r=2&hp&oref=slogin&oref=slogin
From the website:
OCW shares free lecture notes, exams, and other resources from more than 1800 courses spanning MIT's entire curriculum. [1]
Wow. I may have found my new years resolution.
[1] - http://ocw.mit.edu/OcwWeb/web/home/home/index.htm
[2] - http://www.nytimes.com/2007/12/19/education/19physics.html?_r=2&hp&oref=slogin&oref=slogin
Tuesday, December 18, 2007
Crowd Sourced Cloud Computing?
Projects like seti@home [1] have been around for awhile, I'm not sure how long. There are a few others that I can't seem to find right now. The articles about Google's cloud computing [2] got me to thinking, why isn't there a crowd-sourced general purpose version of these networks?
Some company creates the libraries and test facilities for writing MapReduce [3] programs. Then they provide a free program that people can download and run, like in [1]. Companies would pay to have their programs run on the network, and the profits would be passed on to the individuals who's computers ran some of the processing ...
I know there are a lot of tiny details to get right, security, validation of results (IE, rogue machines purposely calculating invalid results quickly to game the system), but it feels like it could be insanely profitable.
Someone has to have thought of this already. If someone out there knows of this, please let me know.
[1] - http://setiathome.berkeley.edu/
[2] - http://www.abetterkindofangry.com/2007/12/googles-algorithms.html
[3] - http://en.wikipedia.org/wiki/MapReduce
Some company creates the libraries and test facilities for writing MapReduce [3] programs. Then they provide a free program that people can download and run, like in [1]. Companies would pay to have their programs run on the network, and the profits would be passed on to the individuals who's computers ran some of the processing ...
I know there are a lot of tiny details to get right, security, validation of results (IE, rogue machines purposely calculating invalid results quickly to game the system), but it feels like it could be insanely profitable.
Someone has to have thought of this already. If someone out there knows of this, please let me know.
[1] - http://setiathome.berkeley.edu/
[2] - http://www.abetterkindofangry.com/2007/12/googles-algorithms.html
[3] - http://en.wikipedia.org/wiki/MapReduce
Subscribe to:
Posts (Atom)