Jun
17

Django row-level permissions

yazzgothPython, News, DjangoNo Comments

It is a common need to have per-row (instance) permissions in a web application.

I’ve created a very simple and non intrusive module to help with that.

You can find the source code of the django-granular-permissions on Google Code here:

http://code.google.com/p/django-granular-permissions/

It’s fully functional, only thing missing are unittests and templatetag that would use the request.user provided by the session middleware context.

To use the package simply add it somewhere to your PYTHONPATH (ie. under your django project, or use

easy_install -U django-granular-permissions

to grab it from cheeseshop) and add the ‘django_granular_permissions’ to your INSTALLED_APPS in your projects settings.py

That’s it. From now on you can check, add and remove permissions for any instance of models class in your project like this:

Python [Show Plain Code]:
  1. # adding permission ‘edit’ to a user ‘Bart’ on an instance of a MyObject from myapp.models
  2.  from django.contrib.auth.models import User, Group
  3.  from myapp.models import MyObject
  4.  user = User.objects.get(username=‘Bart’)
  5.  obj = MyObject()
  6.  obj.save()
  7.  user.add_row_perm(obj, ‘edit’)
  8.  user.has_row_perm(obj, ‘edit’)
  9. True
  10.  user.has_row_perm(obj, ‘delete’)
  11. False
  12.  
  13. # similar for groups
  14.  group = Group.objects.get(pk=1) # get first group in the db
  15.  group.add_row_perm(obj, ‘read’)
  16.  
  17. # now we’ll add the user to the group and he will inherit the ‘read’ permission
  18.  user.groups.add(group)
  19.  user.has_row_perm(obj, ‘read’)
  20. True
  21.  
  22. # now to remove permission
  23.  user.del_row_perm(obj, ‘edit’)
  24.  user.has_row_perm(obj, ‘edit’)
  25. False
  26.  
  27. # note that when you try to remove a permission from a user that is granted to him through group nothing changes
  28.  user.del_row_perm(obj, ‘read’)
  29.  user.has_row_perm(obj, ‘read’)
  30. True
Apr
7

Timing a python function with a decorator and saving results to CSV file

yazzgothPythonNo Comments

Here’s a sample I wrote and find very useful for quick performance tests of python functions.

Python [Show Plain Code]:
  1. import timing
  2. from datetime import datetime
  3. def timeit(fn):
  4.     def wrapper(*args, *kw):
  5.         timing.start()
  6.         fn(*args, **kw)
  7.         timing.finish()
  8.         f = open("timing.csv", "a")
  9.         f.write("%s,%s,%d\n"%(datetime.now(), fn.__name__, timing.micro()))
  10.         f.flush()
  11.         f.close()
  12.     return wrapper

You can then import it to OpenOffice Calculator or MS Excel and do some fancy charts or whatever ;)
Note that this function is using disk I/O so by itself it is also causing some overhead (especially for functions called multiple times per second).
Obviously it’s not thread safe.

Jan
8

Non-blocking use of Django models API

yazzgothPythonNo Comments

If you’ve ever used Twisted framework and got through the first a bit hard learning curve, you most likely fell in love with it.
And if you’ve every used Django you most likely thought - isn’t there a way for it to run faster and in non-blocking fashion - especially if you want to write a web application that integrates other protocols.
One major block on the road is the DB models API which I’ll focus on now.

So let’s say you’re considering to use Django’s models in Twisted application.
As in my previous post you’ll have to prepare the environment to access the models, then use Twisted’s threaded deferred to make it non-blocking.

Python [Show Plain Code]:
  1. from twisted.internet import reactor, threads
  2. from django.core.management import setup_environ
  3. import settings
  4. import string
  5. setup_environ(settings)
  6. from foo import models
  7.  
  8. def getUser(user):
  9.    print "Hello %s %s"%(user.first_name, user.last_name)
  10.  
  11. def handleError(e):
  12.    print "Error "+str(e)
  13.  
  14. deferred = threads.deferToThread(models.User.objects.get, username=‘yazzgoth’)
  15. deferred.addCallback(getUser)
  16. deferred.addErrback(handleError)
  17. reactor.run()
Nov
8

Overriding name of file in ImageField(FileField) in Django

yazzgothPythonNo Comments

If you’re wondering how to override a name of a file you’re uploading to ImageField/FileField in Django, here’s one of the possible solutions:

Add this to your models.py

Python [Show Plain Code]:
  1. from django.dispatch import dispatcher
  2. import shutil, os, settings, re
  3.  
  4. class CustomImageField(models.ImageField):
  5.     """
  6.     Model class should have a method like:
  7.  
  8.         def personalize_name(self, field_attname):
  9.             return "path/to/file/%d.jpg" % self.id
  10.  
  11.     """
  12.     def contribute_to_class(self, cls, name):
  13.             super(CustomImageField, self).contribute_to_class(cls, name)
  14.             dispatcher.connect(self._save, models.signals.pre_save, sender=cls)
  15.  
  16.     def _save(self, instance):
  17.         oldname = getattr(instance, self.attname)
  18.         extension = re.findall(r‘\.[\w\d]+$’, oldname)
  19.         if len(extension)>0:
  20.             extension = extension[0]
  21.         else: extension = ""
  22.         filename = instance.personalize_name(self.attname)
  23.         filename = self.upload_to + ‘_’.join(re.findall(r‘[a-zA-Z0-9]+’, filename)) + extension # leave only alphanum characters and add extension and add extension
  24.         try:
  25.             shutil.move(os.path.join(settings.MEDIA_ROOT, oldname), os.path.join(settings.MEDIA_ROOT, filename))
  26.         except IOError:
  27.             filename = oldname
  28.         setattr(instance, self.attname, filename)
  29.        
  30.     def db_type(self):
  31.         """Required by Django for ORM."""
  32.         return ‘varchar(100)’

And to your class definition you need to add something like this:

Python [Show Plain Code]:
  1. class MyPhoto(models.Model):
  2.     photo = CustomImageField(upload_to="photos/")
  3.     owner = models.CharField(max_length=32)
  4.     uploaded = models.DateTimeField(auto_add_now=True)
  5.    
  6.     def personalize_name(self, field_attname):
  7.         return "%s %s" % ( self.owner, self.uploaded ) # don’t worry about the non-alphanum characters

Assuming that the owner will be Bartosz Ptaszynski and the uploaded date 08-11-2007 13:06:00 and the file is photo.jpg the saved file will be:

photos/Bartosz_Ptaszynski_08_11_2007_13_06_00.jpg

Inspired by this blog. Thanks Scott.

Oct
31

Kinetic Style Sheets (KSS) or how to AJAX without JavaScript

yazzgothJavaScript, PythonNo Comments

I’ve just found a very interesting framework designed for Python/AJAX developers.
It’s called Kinetic Style Sheets, or KSS for short, and it’s goal is to use CSS with some additional syntax to pass marshalled data from server back to the client to manipulate the DOM without using a single line of JavaScript (most of the times).

Apparently it’s included in PLONE 3.0, a new version of framework I’ve used some time back, but due to my migration to lighter frameworks (TurboGears/Django) existence of KSS had evaded my attention until now.
“So what’s so hot about Kinetic Style Sheets?” you may ask “Does it move things?”… Well pretty much :) But that’s not the killer feat.

It’s designed for Python developers who wish to keep their nose in Python code as much as possible avoiding hairy JavaScript(browser) world.

It’s also designed not to break compatibility with JavaScript-disabled browsers.

Well that’s that for wetting appetite… It’s time to get a bite of this framework, and then I’ll be back with more thoughts.

Oct
25

Generate random password in Python? how about one line

yazzgothPythonNo Comments
Python [Show Plain Code]:
  1. import string
  2. from random import Random
  3.  
  4. newpasswd = ”.join( Random().sample( string.letters + string.digits, 12) )
Oct
16

Django - overriding save() to change related object…

yazzgothPythonNo Comments

It may be not so obvious at first that when you override save() in models to change something in a related object (ie via ForeignKey) you have to explicitly call save() on that related object.

Python [Show Plain Code]:
  1. class FirstObject(models.Model):
  2.    related_object = models.ForeignKey("SecondObject")
  3.    def save(self):
  4.       self.related_object.counter = self.related_object + 1
  5.       self.related_object.save()
  6.       super(FirstObject, self).save()
  7.  
  8. class SecondObject(models.Model):
  9.    counter = models.IntegerField(default=0)
Sep
26

Writing an external script that can access your Django models

yazzgothPythonComment (1)

So you need to write an external script that would run outside Django and still have access to your app’s models?

Nothing could be more simple.

Python [Show Plain Code]:
  1. from django.core.management import setup_environ
  2. import settings
  3. setup_environ(settings)
  4.  
  5. from foo import models
  6.  
  7. # rest of your code…
Jun
20

macsearch.py - MAC search script for Motorola Canopy networks

yazzgothPython, Networking, Motorola CanopyComment (1)

This little program is what I’ve mentioned earlier on my blog. It connects to all devices it can find in specified IP ranges via SMTP and gathers their MAC tables.
The plan is to keep this data in a DB and update it once in a while (20min would seem appropriate ). Then you can easily pin point what route do packets take on the L2 of the network and what Subscriber Module (SM) has a client using specific IP address/MAC.
This can be very useful in Motorola Canopy based networks with open architecture.

Package still under development.

Currently to make it work you need:

* Motorola Canopy based network
* Net-SNMP binaries (snmpwalk)
* Python 2.5
* SQLObject
* IPy
* SQLite

1) Download and install Net-SNMP binaries (or compile it if you wish) and make sure the executables are within PATH.

2) Install Python 2.5

3) To make your life easier download and run ez_setup.py to install latest distutils.

4) From the command line type: easy_install SQLObject
which should download and install SQLObject (and most likely SQLite as well - if not download SQLite and install it)

5) From the command line type: easy_install IPy

6) Now you can use following script

macsearch.py

Python [Show Plain Code]:
  1. #!/usr/bin/env python
  2.  
  3. #  Copyright (c) 2007, Bartosz Ptaszynski
  4. #
  5. #  macsearch.py is free software; you can redistribute it and/or modify
  6. #  it under the terms of the GNU General Public License as published by
  7. #  the Free Software Foundation; either version 2 of the License, or
  8. #  (at your option) any later version.
  9.  
  10. """
  11.  
  12. @author: Bartosz Ptaszynski
  13. @copyright: (C) 2007 Bartosz Ptaszynski
  14. @license: GNU General Public License (GPL)
  15. """
  16.  
  17. # Standard Library imports
  18. from os import popen
  19. from sys import argv
  20. from optparse import OptionParser
  21. from datetime import datetime
  22.  
  23. # 3rd Party Library imports
  24. from sqlobject import *
  25. from IPy import IP
  26.  
  27.  
  28. class MAC(SQLObject):
  29.     ip = StringCol(length=16)
  30.     mac = StringCol(length=18)
  31.     device_type = StringCol(length=6)
  32.     last_seen = DateTimeCol(default=datetime.now())
  33.     site_name = StringCol(default="")
  34.  
  35. class Subnet(SQLObject):
  36.     subnet = StringCol(length=20, alternateID=True)
  37.  
  38. class MACDiscoveryAgent(object):   
  39.     snmp_site_name = " SNMPv2-MIB::sysName.0"
  40.     snmp_macs = " enterprises.161.19.3.3.4.1.1"
  41.     snmp_device_type = " enterprises.161.19.3.3.1.7.0"
  42.    
  43.     def __init__(self, community="Canopy", snmp_version="2c", db_uri="sqlite:/:memory:"):
  44.         self.community = community
  45.         self.snmp_version = snmp_version
  46.         self.snmpwalk = str("snmpwalk -c "+community+" -v "+snmp_version+" ")
  47.         self.db_uri = db_uri
  48.         self._db_connect()
  49.  
  50.     def _init_db(self):
  51.         try:
  52.             MAC.createTable()
  53.             Subnet.createTable()           
  54.             return True
  55.         except:
  56.             return False
  57.        
  58.     def _db_connect(self):
  59.         """ Connect to the Database
  60.         """
  61.         sqlhub.processConnection = connectionForURI(self.db_uri)
  62.         self._init_db()
  63.  
  64.     def _db_addMAC(self, ip, mac, device_type, site_name):
  65.         """ Add a MAC to the Database, refresh the last_seen if it already exists
  66.         """
  67.         oldmac = MAC.selectBy(mac=mac, ip=ip, device_type=device_type)
  68.  
  69.         try:
  70.             if oldmac.last_seen:
  71.                 oldmac.last_seen = datetime.now()
  72.         except AttributeError:       
  73.             m = MAC(ip=ip, mac=mac, device_type=device_type, site_name=site_name)
  74.             #print m
  75.    
  76.     def _db_removeSubnet(self, subnet):
  77.         """ Removes a subnet that the MAC lookup should sweep from the Database
  78.         """
  79.         pass
  80.  
  81.     def _db_removeMAC(self, mac):
  82.         """ Completely removes a MAC address from the database
  83.         """
  84.         pass
  85.    
  86.     def _db_lookupMAC(self, mac):
  87.         """ Looks up a MAC address in the Database
  88.         """
  89.         results = MAC.selectBy(mac=mac)
  90.         return results
  91.    
  92.     def _normalizeMAC(self, mac):
  93.         # TODO: use regexp here
  94.         if "-" in mac:
  95.             mac = mac.split("-")
  96.         elif ":" in mac:
  97.             mac = mac.split(":")
  98.         elif " " in mac:
  99.             mac = mac.split(" ")
  100.         mac = ”.join(mac)           
  101.         mac = mac.upper()
  102.         return mac
  103.  
  104.     def _getMACs(self, ip):
  105.         """ Get all MAC addresses from the device
  106.         """
  107.         print ip
  108.         device_type = popen( str(self.snmpwalk + ip + self.snmp_device_type) )
  109.         device_type = device_type.read()
  110.         device_type = device_type[52:-2] # TODO: I should use regexp here!
  111.         macs = popen( str(self.snmpwalk + ip + self.snmp_macs) )
  112.         macs = macs.readlines()
  113.         site_name = popen( str(self.snmpwalk + ip + self.snmp_site_name) )
  114.         site_name = site_name.read()
  115.         for mac in macs:
  116.             self._db_addMAC(ip=ip, mac=self._normalizeMAC(mac[-19:-2]), device_type=device_type, site_name=site_name[32:]) # TODO: use regexp!
  117.  
  118.     def addSubnet(self, subnet):
  119.         """ Add a subnet that the MAC lookup should sweep to the Database
  120.         """
  121.         try:
  122.             Subnet(subnet=subnet)
  123.             return True
  124.         except:
  125.             return False
  126.  
  127.     def sweep(self):
  128.         for subnet in Subnet.select():
  129.             for ip in IP(subnet.subnet):
  130.                 self._getMACs(str(ip))
  131.        
  132.     def lookupMAC(self, mac):
  133.         """ Lookup a MAC address in the Database
  134.  
  135.             If successful it returns IP address of the SM with provided MAC address.
  136.             If not successful it returns a list with all APs and BHs that have provided MAC address.
  137.             It’s a list of dictionaries. This list has a format [ {’device_type’: string, ‘ip’, string}, … ]
  138.            
  139.         """
  140.         mac = self._normalizeMAC(mac)
  141.         results = self._db_lookupMAC(mac)
  142.         for result in results:
  143.             if "SM" in str(result.device_type):
  144.                 return result
  145.         return results
  146.        
  147. # Example usage below…
  148.  
  149. m = MACDiscoveryAgent()
  150. m.addSubnet("192.168.105.44/30")
  151. print "Sweeping the network…"
  152. m.sweep()
  153. print "Sweeping completed."
  154. print "Looking up the MAC address…"
  155. res = m.lookupMAC(‘00 E0 4C A0 28 CC’)
  156. if type(res) is MAC:
  157.     print "Found it:", res.device_type,":", res.ip, ":",res.site_name,": last seen:", res.last_seen
  158. else:
  159.     print "Couldn’t find the SM, But found following items with this MAC in their MAC table"
  160.     for i in res:
  161.         print i.device_type, ":", i.ip, ":", i.site_name, ": last seen ", i.last_seen
Mar
28

Python and SNMP on Windows

yazzgothPython, NetworkingNo Comments

Since I’m working in a mixed environment (Windows and Linux) I’m expecting from every package I use to be multi-platform. I have started working on a Python script that would help me to locate a MAC address on a Motorola Canopy network, down to the Subscriber Module (SM).

I find PySNMP too complex, too difficult to learn in few minutes. It’s very low-level.
yapysnmp looks great, but it’s troublesome to compile (dependencies on outdated net-snmp).
SNMPy? Linux only….

So I’ve started writing my own code using popen and net-snmp binaries which should work on any platform.

I always like to keep things simple. Nice and easy for someone else to install and start using, so I’m always trying to keep the dependencies on 3rd party stuff to the minimum.

The idea I have is to utilize SQLObject + SQLite to keep track of the MAC addresses (incl. last seen info) so the MAC lookup will be very fast, and run a collector that would sweep through provided subnets collect MAC tables and save that information to the database.