I swear this particular feature of querysets was made especially for the archive listing of a blog. You know, the month and year listing in the side bar for all your previous posts. This solves that problem exactly.

Let’s say you have a BlogPost model, and a post_date field within said model. You want a list on the sidebar of all the unique month and year combinations. Here’s what you do:

queryset = BlogPost.objects.dates('post_date', 'month')

And that’s it. You’ll now have a query set full of datetime objects with all your unique months and years.

To get unique years only, substitute ‘year’ for ‘month’, and for unique day, month and year, substitute ‘day’ for ‘month’.

View Part 1
Beardy Geek Git Hub Repository

Welcome back. In part 2 I’ll be getting to the meat of the issue, which is retrieving the data from an existing Wordpress blog, and feeding the data into my own models.

Models

I’ve decided to start the models from scratch, rather than trying to copy the way Wordpress has laid them out. It should keep things simpler that way. I’ll be using the contrib comments system for the post comments, and I’ll also be creating 3 other models, Category, Tag, and Post. Here’s the code for the models:

from django.db import models
from django.contrib.auth.models import User
from datetime import datetime

POST_STATUS = (
               ('P', 'Published'),
               ('U', 'Unpublished'),
)

class Tag(models.Model):
    text = models.CharField(max_length=75)
    slug = models.CharField(max_length=75)

    def __unicode__(self):
        return self.text

class Category(models.Model):
    text = models.CharField(max_length=75)
    slug = models.CharField(max_length=75)

    def __unicode__(self):
        return self.text

    class Meta:
        verbose_name_plural = "categories"

class Post(models.Model):
    title = models.CharField(max_length=75)
    slug = models.CharField(max_length=75)
    content = models.TextField()
    author = models.ForeignKey(User)
    post_date = models.DateTimeField(default=datetime.now)
    status = models.CharField(max_length=1, choices=POST_STATUS)
    categories = models.ManyToManyField(Category)
    tags = models.ManyToManyField(Tag)

    def __unicode__(self):
        return self.title

Wordpress Export

Just a quick note: I am using Wordpress Version 2.6.1. Things may be different in other versions, but the code here works on this version. I’ll go through the code so you should see where the problem is if it doesn’t work with your version.

OK, if you didn’t already know, you can export all your Wordpress data into an xml file. Go to your dashboard, click ‘Manage’, and then ‘Export’. Select which authors to restrict (if any) and hit the download button. You should now have the data in an xml file.

XML File Editing

When I first tried to parse this file, I got an error as one of the namespaces used in the document is undeclared. To rectify this, you need to open up the file in a text editor (not an xml editor, you may get the same parse error). Near the top of the file, you should see something like this:

<rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:wp="http://wordpress.org/export/1.0/"
>

The missing namespace is ‘excerpt’, and we need to add this. It doesn’t matter what value you give it. I did this:

<rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:wp="http://wordpress.org/export/1.0/"
	xmlns:excerpt="mysite"
>

Save the file and we’re ready to start parsing the data.

XML Parsing

To parse the XML I have used ElementTree. This is included in Python 2.5, but if you’re using an earlier version, you can get ElementTree from Effbot.org.

I won’t include all the code here, just some snippets as examples. To view the full source, please check out the BeardyGeek Github Repository.

First we need to load our xml file.

tree = ElementTree.parse('c:/wordpress.xml')

Then find the top level under which our data resides, which is the ‘channel’ tag.

chan = tree.find('channel')

Now I want to create some shortcut variables for the namespaces that we will use when finding tags.

wp_ns = '{http://wordpress.org/export/1.0/}'
content_ns = '{http://purl.org/rss/1.0/modules/content/}'

Now we can get all the category, tag and item(post) entries:

cats = chan.findall('{http://wordpress.org/export/1.0/}category')
tags = chan.findall('{http://wordpress.org/export/1.0/}tag')
items = chan.findall('item')

This will give us lists of all the elements for those three items.

Finding and Saving Data

I’ll give an example of saving the data using the category tag.

for cat in cats:
        c = Category(text=cat.find(wp_ns + 'cat_name').text,
                 slug=cat.find(wp_ns + 'category_nicename').text)
        c.save()

The tag data is the same as the above.

The item data is a bit more complex. If you look at the xml you’ve exported, you’ll see that the item data includes both posts and pages. But it also gives all previous revisions of each post, which will include any drafts saved whilst writing a post. So we need to find all those with a status of ‘publish’ and a page type of ‘post’. We’ll deal with the ‘page’ data another time, using Flatpages.

if item.find(wp_ns + 'status').text == 'publish' and
               item.find(wp_ns + 'post_type').text == 'post':
            i = Post(title=item.find('title').text,
                     slug=item.find(wp_ns + 'post_name').text,
                     content=item.find(content_ns + 'encoded').text, author=u,
                     post_date=item.find(wp_ns + 'post_date').text,
                     status='P')
            i.save()

The ‘u’ (value for author) is a User object I create earlier in the code that I’ve used as the default author of each post (see source).

Post Categories

Now we have to find out which categories and tags this post has. Within each ‘item’ we have ‘category’ data. A bit confusingly this ‘category’ data also includes the tags, and to discover that you need to look at the ‘domain’ attribute to see which it is. Plus we only need the category with the ‘nicename’ in it (slugified).

post_cats = item.findall('category')

for pc in post_cats:
     #check for attributes
     if pc.get('nicename'):
         if pc.attrib['domain'] == 'category':
              c2 = Category.objects.get(slug=pc.attrib['nicename'])
              i.categories.add(c2)
         elif pc.attrib['domain'] == 'tag':
              t2 = Tag.objects.get(slug=pc.attrib['nicename'])
              i.tags.add(t2)

Comments

The last section deals with comments. I am using the django.contrib.comments module for this.

comments = item.findall(wp_ns + 'comment')

for comm in comments:

    if not comm.find(wp_ns + 'comment_author_email').text:

        comm_email = ''

    else:

        comm_email = comm.find(wp_ns + 'comment_author_email').text
        if not comm.find(wp_ns + 'comment_author_url').text:

            comm_url = ''

        else:

            comm_url = comm.find(wp_ns + 'comment_author_url').text
        db_comm = Comment(comment=comm.find(wp_ns + 'comment_content').text,

            ip_address=comm.find(wp_ns + 'comment_author_IP').text,

            object_pk=i.id, submit_date=comm.find(wp_ns + 'comment_date').text,

            user_email=comm_email,

            user_name=comm.find(wp_ns + 'comment_author').text[:50],

            user_url=comm_url,

            content_type=ct, site=site)

        db_comm.save()





Conclusion
Well that wraps it up for this post. I'll cover extracting the data for Flatpages in the next post, but this should give you enough to get started. You can see how to extract the required data from the xml document, so if you want to extend the models beyond what I have, you shouldn't have any problems.  Again, check out the fully code, plus the other file changes (url.py etc) at the Beardy Geek Git Hub Repository. Have fun.
 
 
					

Now don’t get me wrong, there’s nothing wrong with Wordpress. It’s just that I like to play with stuff, so I thought it would be fun to create a blog in Django, copy all my Wordpress posts across, and add at least some of the functionality that Wordpress has built in.

Open Source

Throughout this process, so that you can follow along, I will be storing the code in a GitHub repository. Click here to view the BeardyGeek repository

A Few Considerations

There are a few things to think about before getting started:

1. Data Conversion – getting the data from Wordpress into my Django models. Do I want to keep the same database structure as Wordpress, or create my own? If I create my own, how will I get the data across?
2. Services – how much do I try and implement? The main ones are comments, tags, pings, anti-spam.

I’m sure more will come to mind as I go through this

Conclusion

Well, I’d better stop jabbering and get on with it then! Keep an eye on the blog, or watch the Git repo for updates. I’ll get done whatever my current workload allows.

My day job involves working with software that automatically creates webservices on the .Net platform. Up until now I have used C# to create web applications to use these webservices.

But it would be nice to have the flexibility to use another programming language to create a solutions, and consume the webservices from there.

So, here’s a short tutorial on calling your .net webservice using Python.

Pre-requisites

1. I’m currently using Python 2.5, so I can’t speak for other versions
2. You will need the ElementSoap package from effbot.org. You can get it from here
3. You’re also going to need a .Net webservice with which to test this out. If you’re reading this tutorial, you probably already have one.

SOAP

Firstly we need to look at the SOAP examples on your .Net webservice. Go your webservice’s asmx page, and then click on one of your webservices to view the details. You will see some example SOAP requests and responses. The one we’re interested in here is SOAP 1.1. Mine looks like this:

POST /lookserver/webservices.asmx HTTP/1.1
Host: localhost
Content-Type: text/xml; charset=utf-8
Content-Length: length
SOAPAction: "http://tempuri.org/GetVersion"

<?xml version="1.0" encoding="utf-8"?>
<soap:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
  <soap:Body>
    <getVersion xmlns="http://tempuri.org/" />
  </soap:Body>
</soap:Envelope>

HTTP/1.1 200 OK
Content-Type: text/xml; charset=utf-8
Content-Length: length

<xml version="1.0" encoding="utf-8"?>
<soap:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:xsd="http://www.w3.org/2001/XMLSchema"
 xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
  <soap:Body>
    <getVersionResponse xmlns="http://tempuri.org/">
      <getVersionResult>string</getVersionResult>
    </getVersionResponse>
  </soap:Body>
</soap:Envelope>

As we can see here, I have a webservice called GetVersion, the namespace is http://tempuri.org, and the SOAPAction is http://tempuri.org/GetVersion. Note these down for your webservice.

Into the Python

Fire up an interactive session, and we’ll go through calling this service.

Firstly we need to import the ElementSOAP library:

from elementsoap import ElementSOAP as ES

If you get an error message, then you haven’t installed ElementSoap properly.

Next we create a SoapRequest:

sr = ES.SoapRequest("{http://tempuri.org/}GetVersion")

And now we initialize a SoapService object:

serv = ES.SoapService("http://localhost/lookserver/webservices.asmx")

Obviously put in your own url for your webservice.

Now we call the webservice, using the SOAPAction value:

result = serv.call("http://tempuri.org/GetVersion", sr)

The result is an xml element which should contain the response from your webservice. When I type result I get:

<element '{http://tempuri.org/}GetVersionResponse' at 00A1A8A8>

If you look at the xml at the top of the article, you’ll see that the result returns inside a GetVersionResponse tag. To get the value of the result we type:

result.find("{http://tempuri.org/}GetVersionResult").text

This finds the appropriate tag, and returns the text.

Conclusion

I hope this has helped. When I searched, I found that the other tutorials were either confusing, or out of date. Take a look at the various libraries at effbot.org, especially ElementTree for parsing XML, very useful. Enjoy!

I started adding a sprinkling of Ajax in one of my Django projects, so I thought I’d share a few snippets that might help you add some Ajax goodness into your app.

To handle the Ajax callbacks, I’ll be using Prototype, a lightweight Ajax library. And for a couple of effects, like fade-in, I’ll be using Scriptaculous.

Ajax Voting

I wanted to add a Digg-like voter to my app, where you have the number of votes showing with a vote button, and when you click on the vote button, the number disappears, then fades back in with an increase in the vote. The new vote will be added to the database. I also wanted to only allow each user to only vote on each item once (for obvious reasons!)

Overview

Here’s how it hangs together. The user presses the vote button. This calls back to a url, which has a view, which retrieves and/or stores data, which then renders the output to a template, which is then inserted by magic under the button with the new stored value.

Now on with some code.

Urls

First we need to define a url for the callback.

url(r'^callback/vote/(?P\d+)/$', 'vote', name='callback_vote'),

Models

For the purposes of this post, I’ll create 2 models. The first is an item model, where we can store an item description, and the number of the votes that item has:

class Item(models.Model):
    item_desc = models.CharField(max_length=200)
    votes = models.IntegerField()

Now we can create a Voter model, which stores who voted for which item, so we can prevent multiple voting for each user:

from django.contrib.auth.models import User

class Voter(models.Model):
    voter = models.ForeignKey(User)
    item = models.ForeignKey(Item)

Script Tags

Now we need to import the Javascript for Prototype and Scriptaculous. We can either do this in our base.html, which means the javascript will be loaded on every page, or we could add a block into the header, and just load the javascript on a per page basis. I have also stored the scripts in my MEDIA_URL.

<script type="text/javascript" src="{{ MEDIA_URL }}scripts/prototype.js"></script>
<script type="text/javascript"
    src="{{ MEDIA_URL }}scripts/scriptaculous.js?load=effects"></script>

You’ll see the load=effects option after the scriptaculous url. This is so it doesn’t load all of the modules. You can if you want, but for this example we only need the effect module.

Prototype Script

For the prototype script to work, and update our vote number, we need to have a page element to update. You could have a div or a span, but you need to give the element an id and a name.

Now for the script.

<script type="text/javascript">
	function addVote(item_id) {
		$('item').style.display = 'none';
		var ajax = new Ajax.Request('callback/vote/' + item_id + '/', {
				method: 'get',
				onSuccess: function(transport) {
					var result = $('item');
					result.update(transport.responseText);
				}
			});
		$('item').appear();
	}
</script>

I’m assuming we’ve called the span or div id ‘item’. The $(‘item’) is a short cut to get a reference to the element. Firstly we call the function with the item id as a parameter. Then we set the display to ‘none’ before doing the callback. The callback is done, we update the div element using result.update, and the final line uses the Scriptaculous ‘appear()’ function, which fades the div back in.

Don’t forget to add an onclick() or similar event to what ever you want to use to trigger the callback. In my case I used the tick image.

<a onclick="addVote({{ n.id }})"><img src"..." /></a>

I used n as a context for my item on the view for this page, in case you’re wondering what the {{ n.id }} is for.

Callback Code

Now we have the client side script, we need to take that item id and do something with it, then return the result for display.

First we need to create the vote view.

def vote(request, item_id):
    i = Item.objects.get(pk=item_id)
    #see if user has voted
    voter = Voter.objects.filter(voter=request.user, item=i)

    if not voter:
        v = Voter(voter=request.user, item=i)
        v.save()
        i.votes += 1
        i.save()

    return render_to_response('item/callback_vote.html', {'votes':i.votes})

We get our item object first, then check the Voter model to see if the current logged in user has already voted for this item. If they haven’t, then we create a Voter entry, and increase the number of votes by one. We then return the number of votes in the context.

Callback Template

Now we create a simple template to display the vote view.

{{ votes }}

Conclusion

And there we have it. That’s the basics of Ajax in Django. There’s a lot more to Scriptaculous than just the fade and appear, so check it out.

I am currently working on a django application that will include a profile page for each user. Like many web apps, it will have the ability to upload a profile picture, and have this picture shrunk to a suitable thumbnail size.

To complicate matters, I want the photo uploaded to a directory which will be named after the user.

I’m fairly new to Django, so it took some time to piece this all together, but someone out there may have had the same problems and I hope this post helps.

Prerequisites

You must have the Python Imaging Library installed. You can get this from the PIL homepage.

User Profiles

The first thing we need to do is create a model for our user profile.

from django.db import models
from django.contrib.auth.models import User

class UserProfile(models.Model):
    user = models.ForeignKey(User, unique=True)
    nickname = models.CharField(max_length=50, blank=True, null=True)
    photo = models.ImageField(upload_to='profile_pics', blank=True, null=True)
    thumbnail = models.ImageField(upload_to='profile_thumb', blank=True, null=True,
          editable=False)

I’ve made the upload_to attributes static for the time being, but we will be changing this later.

User Profile Setup

So that Django knows which model to use for you user profile, you have to add a setting to your settings file.

AUTH_PROFILE_MODULE = 'appname.UserProfile'

Make sure that this uses your app name, not your project name. I’ve used UserProfile for the model, but you can call it whatever you like.

Dynamic Upload Directory

To make the upload_to attribute dynamic based on the current user, we have to write a small function that returns the appropriately formed path. My thanks to Josh Ourisman and his post for this part.

Create a file called files.py and save it in your application directory.

def get_profile_path(instance, filename)
    dir = "%s/profile_pics/%s" % (instance.user, filename)
    return dir

This will make the profile picture save in the directory ‘username/profile_pics/filename’. You can move these around if you want a different directory structure. The MEDIA_ROOT value from your settings file will be prepended to this path automatically, so make sure you set that.

Amend the Model

Now back to our model, where we call our new function from the upload_to attribute.

from projname.appname.files import get_profile_path

class UserProfile(models.Model):
    user = models.ForeignKey(User, unique=True)
    nickname = models.CharField(max_length=50, blank=True, null=True)
    photo = models.ImageField(upload_to='get_profile_path', blank=True, null=True)
    thumbnail = models.ImageField(upload_to='profile_thumb', blank=True, null=True,
         editable=False)

We will leave the thumbnail upload_to field, as we will be dealing with that one manually.

Creating the View

I wanted to just create one view that would handle creating a new profile, and displaying and updating an existing profile. First I’ll create a form to display in the view.

from django import forms
from django.forms import ModelForm
from projname.appname.models import UserProfile

class UserProfileForm(ModelForm):
    class Meta:
        model = UserProfile
        exclude = ('user')

Now to the view

from django.shortcuts import render_to_response
from django.template import RequestContext
from famblog.blog.forms import UserProfileForm
from famblog.blog.models import UserProfile

from django.contrib.auth.decorators import login_required

@login_required
def profile(request):
    try:
        myprofile = request.user.get_profile()
    except:
        up = UserProfile(user=request.user)
        up.save()
        myprofile = request.user.get_profile()

    if request.method == 'POST':
        f = UserProfileForm(request.POST, request.FILES, instance=myprofile)
        if f.is_valid():
            f.save()
    else:
        f = UserProfileForm(instance=myprofile)

    return render_to_response('appname/profile.html', {'f':f, 'profile':myprofile},
        context_instance = RequestContext(request))

This view will try and retrieve the user profile based on the current logged in user. If the user profile doesn’t exist, then an exception is thrown. If this happens, we create a profile with just the username and call get_profile() again. We then deal either with a POST event, or we bind the retrieved profile instance to the form ready for updating by the user.

If you don’t want to create thumbnails of your images, you can leave it here. The images will have been loaded

Thumbnail Creation

To create the thumbnail, we now need to override the save def in our model.

def save(self, force_insert=False, force_update=False):
        #get mtime stats from file
        thumb_update = False

        if self.thumbnail:
            statinfo1 = os.stat(self.photo.path)
            statinfo2 = os.stat(self.thumbnail.path)
            if statinfo1 > statinfo2:
                thumb_update = True

        if self.photo and not self.thumbnail or thumb_update:
            from PIL import Image

            THUMB_SIZE = (150, 150)

            #self.thumbnail = self.photo

            image = Image.open(self.photo.path)

            if image.mode not in ('L', 'RGB'):
                image = image.convert('RGB')

            image.thumbnail(THUMB_SIZE, Image.ANTIALIAS)
            (head, tail) = os.path.split(self.photo.path)
            (a, b) = os.path.split(self.photo.name)

            if not os.path.isdir(head + '/thumbs'):
                os.mkdir(head + '/thumbs')

            image.save(head + '\\thumbs\\' + tail)

            self.thumbnail = a + '/thumbs/' + b

        super(UserProfile, self).save()

First we set a thumb_update var to false. Then, if the thumbnail exists, we check to see if the timestamp on the original image is newer than the existing thumbnail. If we didn’t check for this, then when we changed the image, we wouldn’t know whether or not to create the thumbnail.

The rest of the code deals with using the Imaging Library to resize the image. It also creates a ‘thumbs’ directory if one doesn’t already exist, saves the image, and updates the ‘thumbnail’ field in the model.

Conclusion

I hope this has been helpful. Like I said, I haven’t been using Django or Python for more than a few weeks, so if anyone has any suggestions for improving this code, then please comment.

Enjoy!

I’ve just recently started using Django and I wanted to set up an IDE for some coding niceties.

I already use Eclipse for PHP development, so I installed the PyDev plugin for Eclipse.

Now I get autocomplete in the py files, but when writing the Django html templates, there is no autocomplete for the various template tags.

So here’s a quick guide to adding your own tags in Eclipse.

1. In Eclipse, go to Window->Preferences
2. Go down to Web and XML.
3. Then to HTML files and finally Templates
4. You will see a list of all the various HTML templates there.
5. Click on New.
6. Give your tag a name. It can’t be one that’s already taken, but I found I didn’t have any issues. If you want to keep them all together you could start your tag name with ‘DJ’.
7. Select HTML Tag from the drop down.
8. Give your tag a description.
9. Type in the the tag pattern. If you want the full tag on autocomplete, you might type in {% block ${cursor} %}, otherwise you could start with block, which means you would need to type in the {% and then activate the autocomplete in the HTML editor. The ${cursor} places the cursor where you want it after the tag has been inserted.
10. Click Apply and OK.
11. Test this out in the HTML editor. Press CTRL and Space. You should get a list of available tags. Type b (or what ever the first character is for you tag name), and you’ll see your block tag with the description next to it.
12. Press enter, and your tag will appear.

Hopefully this will make your Django coding a little easier.