Hacking for fun and profit

bye bye twitter: for x in `./twitter_backup.py -U tabo -P ‘*******************’ | cut -d’ ‘ -f 1 | sort -r`; do ./twixer -D “$x”; done;

A new version of Feedjack has been released: Feedjack 0.9.12.

Changes:

• Feedjack is now django-unicode compliant
• Using max_length instead of maxlength in models.py to prevent warnings in syncdb
• Patched fjlib to use the new django refactored database backends
• Disabling autoscaping for post contents in the included templates
• Handling of feedparser returning an empty etag
• Fixing encoding related problems in tag urls
• Non-integer page parameters should be handled correctly

This is a make-feedjack-work-with-django-trunk release. Apologies if you had to hack through the code to use it, and thanks to everybody that published guides on how to make everything work, like this ticket from Paul Bissex, this full guide from Alex Kuo and this patch to make unicode tags work.

Share and enjoy.

We interrupt these wonderful 6 months without posting in this blog to share the joy, directly from Django’s BFDLs, my christmas present to myself:

The Django Book! I bought it in Amazon and it was only 3 days late to Peru, not bad for christmas season.

I can’t wait to read this book. I did read the chapter previews in the site and they were very good. I’ll write a review of the book as soon as I finish reading it.

(btw, I’m sick of wordpress, it’s a buggy piece of ^*($#, is there a decent django powered blog with an import-from-wordpress feature?).

Thanks to Jaime Wong:

The “Chicha Simpsons” by Jaime Wong, featuring Antonio Ognio, Miguel Rabi, Cesar Villegas, Homer Simpson drinking chicha morada and yours truly (with a friendly Python).

Thanks a lot Jaime! I’m already using this as my avatar in pownce and in IM.

(ǝʇısqǝʍ dılɟ ǝpoɔıun ǝɥʇ ɟo dlǝɥ ǝɥʇ ɥʇıʍ ǝlqıssod uǝǝq sɐɥ ʇsod sıɥʇ)

˙ʇı ɥʇıʍ sɯǝlqoɹd ǝʌɐɥ noʎ ɟı ʍouʞ ǝɯ ʇǝl ˙ǝʇıs oƃuɐɾp ǝɥʇ uı ʇsılʞɔǝɥɔ ƃuıʇɹod ǝɥʇ ɥʇıʍ ʎsɐǝ ʎɹǝʌ sɐʍ ʇı ˙ʇı ʇɹoddns oʇ ǝƃuɐɥɔ ǝlqıʇɐdɯoɔuı spɹɐʍʞɔɐq ʇsɹıɟ (ʞuıɥʇ ı) ǝɥʇ ƃuıʇıɯɯoɔ ɯɐ ı os ‘ʞɔɐɾpǝǝɟ ɟo uoısɹǝʌ ʇsǝʇɐl ǝɥʇ ǝʞoɹq ǝpɐɹƃdn sıɥʇ

˙(ʇıɯɯoɔ ʇɐɥʇ ɟo ǝzıs ǝɥʇ ʇɐ ʞool ʇsnɾ) ɥɔuɐɹq sıɥʇ uı ʞɹoʍ ƃuızɐɯɐ sıɥ ɹoɟ ʞɔıuuıpǝɹʇ ɯloɔlɐɯ oʇ sopnʞ ˙ʇɹoddns ǝpoɔıun ʇɐǝɹƃ sɐɥ ‘sǝƃɐnƃuɐl ɹǝɥʇo ǝʞılun ‘uoɥʇʎd ǝsnɐɔǝq ǝlqıssod ƃuıɥʇǝɯos ‘ǝɹɐʍɐ ǝpoɔıun ʎlʇǝldɯoɔ ʇoƃ oƃuɐɾp oƃɐ sʎɐp ǝɯos

Some days ago Django got completly Unicode aware, something possible because Python, unlike other languages, has great Unicode support. Kudos to Malcolm Tredinnick for his amazing work in this branch (just look at the size of that commit).

This upgrade broke the latest version of Feedjack, so I am commiting the (I think) first backwards incompatible change to support it. It was very easy with the Porting Checklist in the Django site. Let me know if you have problems with it.

(this post has been possible with the help of the Unicode Flip website)

A new version of Feedjack has been released: Feedjack 0.9.10.

Changes:

• Fixing CSS tags for styles 4 and 5
• feedjack_update now strip()s tags before storing them
• feedjack_update shows the feed number being processed
• We no longer cache the tags in feedjack_update, it had encoding problems
  and makes the script safe to run more than one instance at the same time

I said that 0.9.9 would be the last release in the 0.9 branch. I lied. 0.9.10 is a maintenance/bugfix release. The 0.10 branch of feedjack is on the works.

Share and enjoy.

Taken from a Python up, Ruby down discussion in programming.reddit:

Ruby takes all the elegance and simplicity of Perl, and mixes it with the library support of Lisp…

- foonly

Previously…

A new version of Feedjack has been released: Feedjack 0.9.9.

Changes:

• Fixed i18n related bugs in the templates
• Fixed a bug related to the –settings option in feedjack_update.py
• The feeds per user, tag and user/tag are working again
• You can choose between rss2 and atom in the feeds for user, tag and
  user/tag
• The default /feed/ url now redirects to /feed/atom/ instead of
  /feed/rss/

You are encouraged to update to this version, it has all the acumulated bug fixes known at this moment (thanks Petar).

This will also be the last version of the 0.9 branch. The 0.10 branch (currently trunk) will have several modifications in the data model, so please be careful if you update your site via subversion. Just follow the right branch or install only official releases and you will be safe.

Also, if you are running a Feedjack site, please update your links to our new site: www.feedjack.org. You can also announce your site in the Feedjack mailing list so we can add a link in the project site.

Share and enjoy.

A new version of Feedjack has been released: Feedjack 0.9.8

Changes:

• Internationalization (petar)
• Fixed and infinite loop in feedjack_update.py (do’h!) (petar)
• Added Serbian (Latin) translation (petar)
• Updated the feedparser calls in feedjack_update.py (petar)
• Added Spanish translation (predius)

Thanks to Petar Marić for all his hard work! This release is 90% his work :-)

Some moths ago I wrote about the BDFL considering the use of the Django web framework.

Cronologically it went like this:

Please Teach me Web Frameworks for Python! (2006-01-27)

Literally a cry for help. He didn’t quite like the magic in Django, considering he used a pre magic-removal version.

Web Framework Redux (2006-01-30)

Perhaps WSGI represents the “blank slate” approach; Rails/Django represent the wizard approach; I’m still looking for the ideal mix-and-match solution.

Django vs. Cheetah: 1-0 (2006-01-31)

Guido is beginning to like the Django templates.

Which Part of “No XML” Don’t You Understand?

This one is related to his previous post. Guido just think that the use of XML in a template engine is WRONG. I couldn’t agree more.

Django Gaining Steam (2006-5-4)

Guido talks about Jacob’s Django talk in the Bay Area and Jeff Croft’s Django for non-programmers (a great article).

Months after that, Guido got interviewed in FLOSS weekly (2006-08-04) and he declared:

Leo La Porte (LL): Python doesn’t have a native GUI, there is TCL/tk… is that an issue?

Guido van Rossum (GvR): It seems to be coming less and less of an issue because more and more people are doing everything over the web

LL: The web is an interface, yeah

GvR: So of course that doesn’t really solves the problem because then you have, as I say, more web frameworks than keywords in the language. My personal favorite and I expect that will remain personal favorite for a long time is something named Django.

LL: I was going to ask you about Django. There was just a … just somebody published some article, interesting I think it was in the Rails website testing Django, Rails and a Perl framework and Django was by far the fastest.

GvR: Interesting! I didn’t hear about that.

Chris di Bona (CdB): How do you measure something like a web framework?

LL: Well they set a simple site and they used web testing applications to create lots of transactions and measure transactions and Django was like significally faster. So tell us about Django.

GvR: I am a very satisfied user of a very small part of Django. Django is sort of, I would call it probably a second generation web framework in Python where first generation would be things like Zope and Twisted. Django was originally started I think two guys who work for, believe it or not, a newspaper in Kansas. Not a very glorious location.

CdB: Well, it’s funny because Zope and Plone came out of the (??) newspaper.

LL: Well you know why, they have to streamline production workflows, that’s a big deal for a newspaper

GvR: Maybe that’s the case. This paper in Kansas decided that they wanted to set a local website with information for people in their town that was very responsive to the audience. They wanted to publish things very quickly but also not just add new articles to the site which everybody can do, but change the site completly, add new ideas, new features to the site, add new applications. They came up with endless number of examples, for example publish the sports, like the local sports results of the little league complete with hyperlinks to the teams and photos and all sort of interesting stuff. And they wanted to be able to roll that out very quickly and so I think they did that for maybe two years, and the two guys who did it and working with a bunch of editor who where providing the content, as they were doing that they realized that they needed a framework and they sort of grew a framework out of their first application. As they (??) what kind of things their editors were constantly asking them to them change to the site, they developed more flexibility in all those areas. And at some point they said let’s open source it and they got support from the newspaper. And then a very interesting thing hapenned. I suppose the newspaper is still using Django in some form (they are, and in fact they are selling the CMS they build). I think both of the original developers are no longer working there and they started Django the Open Source Project and what I found really great about that is I talked to those guys a couple of times and see them give presententations and I’ve seen how they work, and they really get open source. And they have a good license, but in my view even more important is the whole process, the way they work with the user community, the way they answer, they find a ballance between chaos and democracy and anarchy and sort of between Cathedral and Bazaar. They let lots of users add new features and provide ideas without losing the original thought and flexibility of the framework and I can think they are really doing a fantastic job at making Django a better product that goes way beyond what that original Kansas newspaper needed.

(now they talk about the Django vs Rails benchmark and how Django is an order of magnitude faster than Rails…)

LL: I will have to take a look at Django, because that’s pretty impressive.

GvR: Absolutely, I highly recommend it.

And yesterday (2006-08-17), at least two sources (Titus Brown and The Third Bit) are talking about what the BDFL said in SciPy 2006:

• Django is the web framework.
• It won’t be included in the standard library because of different development cycles, but will (should?) be as “standard” as PIL or NumPy
• He hopes that Django and TurboGears will converge

There is a discussion about this on reddit.

What do you guys think?

Things have been moving in the Feedjack front. There are new planets using it, like Debian Perú, Planet SCM and Planet Python@TW, but also apps that use use feedjack on the background like Ian Holman’s Economy Chat and VC Chat.

Also in the news: Feedjack just went into the FreeBSD ports, and I hope to see more Django apps get in there soon.

About the new home, Feedjack is now hosted in www.feedjack.org, and there are some new services too (thanks to Google):

• Subversion repository: http://feedjack.googlecode.com/svn/ (instructions)
• Issue tracker: http://code.google.com/p/feedjack/issues/list
• Mailing lists: one for discussion and another one for ticket updates and SVN commits

I’m planning to work again on Feedjack very soon, in the meantime don’t forget to report bugs and send your suggestions to the issue tracker.

Finally, Django 0.95 has been released (digg it here). Some of the new features and changes are:

• Django now uses a more consistent and natural filtering interface for retrieving objects from the database.
• User-defined models, functions and constants now appear in the module namespace they were defined in. (Previously everything was magically transferred to the django.models.* namespace.)
• Some optional applications, such as the FlatPage, Sites and Redirects apps, have been decoupled and moved into django.contrib. If you don’t want to use these applications, you no longer have to install their database tables.
• Django now has support for managing database transactions.
• We’ve added the ability to write custom authentication and authorization backends for authenticating users against alternate systems, such as LDAP.
• We’ve made it easier to add custom table-level functions to models, through a new “Manager” API.
• It’s now possible to use Django without a database.
• It’s now more explicit and natural to override save() and delete() methods on models, rather than needing to hook into the pre_save() and post_save() method hooks.
• Individual pieces of the framework now can be configured without requiring the setting of an environment variable. This permits use of, for example, the Django templating system inside other applications.
• More and more parts of the framework have been internationalized, as we’ve expanded internationalization (i18n) support. The Django codebase, including code and templates, has now been translated, at least in part, into 31 languages. From Arabic to Chinese to Hungarian to Welsh, it is now possible to use Django’s admin site in your native language.

You can also read the full release notes.

Greg Stein announced today a new Google Service in his talk in OSCON: Google Code Project Hosting (they need a shorter name), a hosting service of collaborative development enviroments featuring:

• Project workspaces with simple membership controls
• Version control via Subversion
• Issue tracking
• Mailing lists at groups.google.com

Obviously this is direct competition to Sourceforge.

Now, Sourceforge has has been suffering some problems for years:

• Downtime
• A very cluttered interface
• A search feature that just doesn’t work

Google Code Project Hosting is based on Subversion on Bigtable (instead of filesystem or BerkeleyDB) and features a trac-like issue tracker (written in Python!).

The interface is google-like of course: very simple and without creeping featuritis. There aren’t many projects yet in the system to test the search feature, but since searching is Google’s main strength I bet it will be better than Sourceforge’s.

Google already provides a great mailing list service in Google Groups and Code Hosting can send issue-tracker and SVN commits to the list of your choice.

Is it a Sourceforge killer?

No. It aims, at least at the moment, at different audiences. Google doesn’t offer shell accounts, tarball hosting or compile farms like Sourceforge does. The thing is most projects don’t make use of these features, so I guess lots of small-to-medium sized projects will flock from SF to Google once the dust settles down. For the larger projects, Google’s solution just doesn’t fit (yet).

Update: 0.9.7 released.

Some months ago I wrote about Feedjack, a Django powered Feed agregator. Finally I have decent templates to release it.

You can read all the details or directly download it.

The basic features are (taken from the readme since I’m that lazy):

Like the Planet feed aggregator:

• It downloads feeds and aggregate their contents in a single site
• The new aggregated site has a feed of its own (atom and rss)
• It uses Mark Pilgrim’s excelent FeedParser
• The subscriber list can be exported as OPML and FOAF

But FeedJack also has some advantages:

• It handles historical data, you can read old posts
• It parses a lot more info, including post categories
• It generates pages/feeds with posts of a certain category (example)
• It generates pages/feeds with posts from a certain subscriber (example)
• It generates pages/feeds with posts of a certain category from a certain subcriber (example)
• A cloud tag/folksonomy (hype 2.0 compliant) for every page and every subscriber
• It uses Django templates
• The administration is done via web (using Django’s kickass autogenerated and magical admin site), and can handle multiple planets.
• Extensive use of django’s internal cache engine. Most of the time you will have no database hits when serving pages.

Originally written to be used in ChichaPlanet, it is handling now a lot more planets in the same instance:

• Planet Gnome (mirror)
• Planet Debian (mirror)
• Planet Django (mirror)
• Planet Python (mirror)
• Planet Mozilla (mirror)
• Planet Ubuntu (mirror)
• Planet KDE (mirror)
• Planet Fedora (mirror)
• Planet Debian Perú (mirror)

Share and enjoy.

Just some minutes ago, Adrian Holovaty announced that Django’s magical removal branch has been merged to trunk. This means that now the mythical Django-without-the-magic is now the official development version.

We have been using the MR branch in Aureal for almost two months now. The last pre-magic-removal projects were migrated to MR the last week, since we could see that it was very stable. The merging of today just confirms this.

But what does Magic-Removal really mean?

It means mainly removing all the magic modules from django’s models. Also the way the ORM works is even better, and the programmer can do things that couldn’t be possible in pre-MR Django. Sadly, all these changes are backwards incompatible, but I can assure you that migrating a project is very simple, and it’s worth doing it considering the benefits you’ll get.

The Framework is a lot more Pythonic than it was before.

Remember the BDFL’s gripes about Django?

Similarly, I’m not keen on their object-relational mapping approach. There’s too much magic based on name correspondence, and the automatically generated APIs feel a bit unpythonic.

I hope he and all the people that didn’t like all the magic in Django can take another look at the framework, since it is all gone for good now.

This will be a _very_short post.

Guido van Rossum is considering the use of Django (among other python web frameworks) to develop a project in Google. He is having some minor gripes with it:

• The templating language is not pythonic: This is IMHO a Good Thing. It enforces separation of code and presentation, and is an ideal way for designers to work directly in the templates (most designers are not programmers).
• Too much magic: Guido should check the Removing The Magic page in the wiki about the magic-removal branch. The branch is available in subversion in http://code.djangoproject.com/svn/django/branches/magic-removal

Back to work now.

Update: Jacob Kaplan-Moss wrote an excelent piece: why django.

Looking in django’s del.icio.us page this morning, and I found out that Lawrence Journal-World finally made a website for Ellington, the Django framework powered CMS they wrote for sites like lawrence.com, LJWorld.com and KUSports.com.

The site itself is a little short on details. There are no demos, tours or pdf overviews, but there isn’t any marketroid babble neither, which is a Good Thing.

Ellington is Django’s killer app right now. While not opensource it demonstrates what can be done with the framework: professional, scalable and production-ready web applications.

The Apache HTTP server is by far the most used web server in the world. It is an excellent, feature packed and standards compliant web server. Extremely configurable, with an endless amount of modules, superb documentation and, due to its license, is being used commercially by companies like Oracle. There is only a small problem with Apache: It’s not the fastest server around. This is because historically, Apache’s priorities have been correctness and configurability, not performance. Correctness and configurability are the reasons why Apache powers almost 70% of the web today, but still, Apache has a big, fat ass (we are talking about system resources here).

(more…)

Update: Feedjack has been released!

Some nights ago I wrote a little Planet replacement in Django called FeedJack. There is still no source code available but I’ll release it as open source as soon as I polish it a little and write docs (if a software has no docs, it doesn’t exist).

You can see it in action in ChichaPlanet, probably one of the first “planets” after Planet Gnome and Planet Debian. It has some features in common with PlanetPlanet:

• It downloads feeds and aggregate their contents in a single site
• The new aggregated site has a feed of its own
• It uses Mark Pilgrim’s excelent FeedParser

But FeedJack also has some advantages:

• It handles historical data, you can read old posts
• It parses a lot more info than PlanetPlanet, including post categories
• It generates pages with posts of a certain category
• A cloud tag/folksonomy (hype 2.0 compliant)
• It uses Django templates

(more…)

It has been more than a month since I blogged about Django, the “Web Framework for perfectionists with deadlines”, and the more I use it, the more I know it is the right choice for my web development needs. There is a lot of activity in the project’s trac (this is more important IMHO than activity in the mailing lists ;-)

Some of the big improvements in the last weeks are:

• Internationalization support: both in code and templates.using gettext under the hood.
• A release: Django 0.9 was released this month, this should help people without subversion (or people that don’t know how to use it) installing django.
• More documentation: The documentation has been extended and improved. It is still incomplete, but it is of extremely high quality, probably one of the open source projects with the nicest documentation I’ve seen, and I’ve seen a lot.
• RSS/Atom syndication framework: “Man, it’s so easy it probably shouldn’t even be legal.”
• Pretty error pages: So pretty that now it is nice to have errors in your code ;-)

(more…)