Remote Logging with Python

If you find yourself developing a distributed system, it’s very useful to have a logging subsystem which allows you to see output from multiple nodes in one place.

Python’s Logging module is quite comprehensive, it gives a lot of ways to format messages and a good level of control over what happens. For any given Logger object, there can be multiple Handlers which control where log messages go.

For remote logging Python provides two handlers StreamHandler and DatagramHandler, for TCP and UPD connections respectively. This example focuses on using the DatagramHandler for a UDP connection.

Step 1: Adding the Handler

This part is easy, if you have a Logger object already instantiated then get the DatagramHandler class from logging.handlers and create an instance with the parameters for the IP address and the port of the remote listener.

from logging.handlers import DatagramHandler

ip_address = "127.0.0.1"  # Localhost, for demonstration
port = 5000  # Any port that is not in use

udp_handler = DatagramHandler(ip_adress, port)  # Create the handler
log.addHandler(udp_handler)  # Add the handler to the logger

So that bit’s easy enough! Note that no formatting information is needed at this point, that is handled by the receiver.

Step 2: Receiving the message

This is the part that gets difficult. The receiver will get the message, but rather than being a nice formatted string, it will receive a serialised (In Python terminology: pickled) object of type dict (A Python dictionary). This contains all message parameters we need to format the output.

First: we’re going to make use of four modules: socket, struct, cPickle, and logging.

Second: a socket is needed to receive information sent to it. We create the socket for use with IPv4 addresses and with intent to use UDP as shown. When the socket is bound using the bind() method, we can start accepting connections.

import socket
import struct
import cPickle
import logging

ip_address = "127.0.0.1"  # Localhost, for demonstration
port = 5000  # Any port that is not in use

# socket.AF_INET: Means this socket will operate using IPv4
# socket.SOCK_DGRAM: This socket will receive UDP Datagrams.
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.bind((ip_address, port))

Third: We need a Logger object to work with the LogRecord object we will decode. At this point any formatting rules can be used. In this example, a simple StreamHandler is used to print messages to StdOut.

log = logging.getLogger()  # New Logger instance
handle = logging.StreamHandler()  # Prints to StdOut
handle.setLevel(logging.DEBUG)  # Set effective level
log.addHandler(handle)  # Add to logger

As a tip, if you find messages that you expect to be displayed are being lost, ensure that the proper effective level has been set on both the Logger objects and the handlers.

At this point, the assets we need are in place and we can enter a loop to receive data.

while True:
    data = sock.recv(2048)  # Recieve 2048 bytes
    data_length = struct.unpack(">L", data[:4])[0]  # Get packet size
    try:
        unpickled_dict = cPickle.loads(data[4:data_length + 4])  # Unpickle
    except EOFError:
        print("Error while receiving data")  # Thrown in case of unpickle error
        continue
    record = logging.makeLogRecord(unpickled_dict)  # Dict -> Record
    log.handle(record)  # Print record out

The socket waits on the recv() method until data is sent to it, after which the socket tries to read 2KB of data from the stream. The stream of data contains two things: the length of the data packet (first 4 bytes) and the pickled information. Using the struct module, the first four bytes are translated into an integer, then the cPickle module is used to unpickle that data, from this we get the dictionary. The dictionary is simply turned into a LogRecord object by the logging module, then the Logger object can handle it like any other message.

As a note: if you expect datagrams to ever be longer than 2kb, you will need to use a loop to ensure you receive all data else you will fail to unpickle the data.

The recv() method takes data from the socket, but if the address of the other client is needed, use recvfrom() instead, which returns a tuple containing the data and the address of the client. If more information is needed, consider the following…

Step 3: Sending Additional Data (Optional)

Sometimes, certainly with distributed systems, it’s important to be able to show identifying information in the log. This could be placed in every single log message individually but that’s tedious and may result in a lot of unnecessary code. My solution was to add a dictionary of static information to the handler which is put in ever record it handles.

from logging.handlers import DatagramHandler

class CustomUDPHandler(DatagramHandler):

    def __init__(self, host, port, record_additions={}):
        super(CustomUDPHandler, self).__init__(host, port)
        self.record_additions = record_additions

    def handle(self, record):
        for key, value in self.record_additions.iteritems():
            if key in record.__dict__:
                raise AttributeError(
                    "Attempt to overwrite attribute %s in record." % (key,))
            record.__setattr__(key, value)
        super(CustomUDPHandler, self).handle(record)

Attributes of the LogRecord object are kept in its namespace, overriding the handle method in this way allows us to put all our attributes in the record before it’s pickled and sent. An AttributeError will be raised if you inadvertently attempt to overwrite any of these existing attributes.

On the other end, a formatter can be used to display the information. Say we create an instance of our CustomUDPHandler with the parameters: (‘127.0.0.1’, 3000, {‘ident’: “Client #5”}) then the attribute ident will be a part of the record sent to the other side. Access it as such:

style = "%(ident)> %(message)"  # Ident formatted into string
formatter = logging.Formatter(fmt=style)

stream = logging.StreamHandler()
stream.setFormatter(formatter)
log.addHandler(stream)

Allowing us to have a nice formatted message output from the receiving end that looks like this:

 Client #5> Everything is awesome!

Advertisements

Installing Citrix Receiver on Ubuntu 12.04

Citrix Receiver is a useful cross-platform tool for accessing programs and files on other networks. My university uses Citrix for their desktopAnywhere service but only provides support for MacOS and Windows which is a pain because installing Citrix on Ubuntu is not quite smooth sailing. So here’s a quick guide to cover install and all the hurdles I went through trying to set it up.

1. Download your packages

You can find all the files for Citrix Receiver on the Citrix website in various different flavours. 64 bit packages are at the bottom of the page. Be aware of the file type you’re downloading; Ubuntu users will want to download the .deb files. You can also download the USB Support Package which gives Citrix the ability to read and write from your USB flash drives during a session.

Note: If you’re unsure about which package you need, x86 (32 bit) or x86_64 (64 bit), then run the command uname -i in terminal and you’ll see the arch type of your machine.

Download Options

OpenMotif is required for the graphical configuration menu of Citrix Receiver but not required for general use so we won’t bother with it for the time being.

2. Installing your packages

Due to a few prompts that may pop up during installation, it’d be best to install through the command line. There’s also a bug that prevents successful installation on 64 bit systems so there’s two ways to go about this.

32 bit: Open the terminal and find where you downloaded the files, for me that was Downloads, and install with dpkg as such:

~$ cd Downloads
~/Downloads$ sudo dpkg -i icaclient-12.1.0_i386.deb

Use the same command to install the USB support package if you wish to do so.

~$ sudo dpkg -i ctxusb-2.2.0_i386.deb

64 bit: Open the terminal and find where you downloaded the files, for me that was Downloads, and we want to partially install with dpkg as such:

~$ cd Downloads
~/Downloads$ sudo dpkg --unpack icaclient-12.1.0_amd64.deb

This unpacks the .deb file but doesn’t configure it. The reason for this is an error in the installation script that fails to detect the system type leading to this message appearing:

Unpacking icaclient (from .../icaclient_12.1.0_amd64.deb) ... Setting up icaclient (12.1.0) ... dpkg: error processing icaclient (--install): subprocess installed post-installation script returned error exit status 2 Processing triggers for menu ... Errors were encountered while processing: icaclient

Open your favourite text editor (gedit will do) and find icaclient.postinst in the directory /var/lib/dpkg/info/

~$ sudo gedit /var/lib/dpkg/info/icaclient.postinst

It’s a bit of a long file but you want to find the text near line 2648 that reads

echo $Arch|grep "i[0-9]86" >/dev/null

Replace that line with this

echo $Arch|grep -E "i[0-9]86|x86_64" >/dev/null

How the text should appear.

Save the file and finally, run this command to configure the package.

~$ sudo dpkg --configure icaclient

ICA client should be installed. Should you wish, you can also take this opportunity to install USB support in the same way by using dpkg -i <filename here>.

3. Supplying authentication certificates.

The Citrix Client does not supply many of it’s own SSL certificates for secure connection authentication, fortunately for Ubuntu users – Firefox does! You can use the following command to copy them over.

~$ sudo cp /usr/share/ca-certificates/mozilla/* /usr/lib/ICAClient/keystore/cacerts/

Any specific CA certificates that your Citrix admin requires can be placed in that folder too. If you get the error You have not chosen to trust “<name here>“, the issuer of the server’s security certificate. then you’ll need to place the appropriate CA certificate in the /usr/lib/ICAClient/keystore/cacerts/ directory.

4. Give it a go!

I’d had a few issues installing the Citrix Receiver onto 64 bit Ubuntu but following those worked for me and… voilà!

Citrix Receiver running on Ubuntu 12.04 x64

Is hacktivism a genuine form of protest?

Cross-post, originally published on ORG-Zine.

In the online age of uncertain digital rights, it’s crucial for individuals to take a stand and raise awareness of the issues that affect all of us online. There’s several ways within the online world to demonstrate and raise awareness of causes such as e-petitions or social networking groups, but one of the more controversial methods is hacktivism. The term hacktivism is a portmanteau of the wordshacker and activist which tends to refer to the use of unauthorised computer access to further an agenda, usually political or social. Hacktivism itself can be a fairly dubious issue, for all the power it can grant hackers of varying shades of grey, it could potentially be an effective tool for promoting an important agenda.

Is hacktivism effective?

It’s very difficult for activists in small numbers to bring awareness to the issues that they campaign against. It’s tempting for activists to pull stunts in order to raise awareness of their cause; take the instance of Eddie Gorecki and Jonathan Stanesby, two members of Fathers 4 Justice, who scaled the Royal Courts of Justice dressed as Batman and Robin. Their protest managed to gain national recognition in the press, which rather successfully raised their profile. Days later two-thousand supporters marched in London with a tank!

So perhaps hacktivism is just that – the gimmick that raises the profile of a cause. Anonymous hacktivists have used the Low Orbit Ion Cannon (LOIC), a tool for Distributed Denial of Service (DDoS) attacks, to take down several websites of organisations supporting the Stop Online Piracy Act. The Department of Justice website was taken down as well as those owned by the FBI, MPAA, RIAA. Was it effective? Well SOPA failed, didn’t it?

That said, I wouldn’t attribute the success of the anti-SOPA campaign to Anonymous taking down websites. At most, I could imagine people trying to access those sites being irritated that “heir internet isn’t working properly.” Many of the examples of hacktivism I’ve seen are preaching to the converted; this style of hacktivism does not seem to do much to engage with the public beyond creating momentary annoyances.

I believe that most of the attention to the anti-SOPA campaign came from the blackout of prominent websites such as Wikipedia, Reddit and Craigslist. This seemed a brilliant strategy to confront end-users with the effects of censorship, perhaps more-so than a ‘500 Internal Error’ web page.

However, aside from black-outs, there is also the case of internet vigilantes such as ‘The Jester’ who put a lot of effort into disrupting the websites of alleged terrorist organisations. As of late, he’s also helped to put behind bars several ‘script-kiddies’ who disrupted the UK anti-terrorist hotline as well as disrupt the activities of Anonymous group LulzSec. If there’s such a thing as hacktivism in action – that’s it!

Is hacktivism ethical?

On the face of it, a lot of DDoS attacks can seem to simply be retaliation – an eye for an eye. I’ve never been comfortable with that stance. To me, activism should be about rectification rather than revenge. So can hacktivism ethically meet this criteria?

It could be seen that hacktivism in the form of website take-downs and take-overs is a method of censorship. Denying access to information from groups with opposing viewpoints could be seen as dodgy behaviour, however I’m not ready to brand this as censorship. I feel that censorship is a very strong term describing the suppression of ideas; this sort of website blocking is more comparable to graffiti than book burning. These actions often take websites offline for a few hours, nothing serious enough to cause lasting damage but just enough to raise attention to a cause. Consider as well that this behaviour could be a form of disobedience in refusing to accept services as-is; that perhaps attacks like these are comparable to blockading buildings like the recent protest in Mexico against biased TV reporting or even the Occupy movement.

It’s easy enough to imagine however that the owners of these websites won’t see having their own content effectively blocked as a legitimate act of protest. It’d be all too easy for groups of organised hackers to pick on somebody whose livelihood depends on their web presence. It’s unlikely to be that bad; targets of hacktivist attacks tend to be large multi-national corporations like Visa, to whom a few angry nerds with a DDoS script pose little risk. But the key question as to whether these attacks are justified does not have a blanket answer; it will always depend on the specific case.

A tactic that certainly should be of concern to digital rights campaigners is document droppingor as it’s referred to: “dox drops”. This is the practise of hackers stealing personal, or otherwise private, information pertaining to individuals and publishing it to the web. Personal details of executives of the pro-copyright lobby were published to the world as part of Anonymous’Operation Payback. Clearly that was a breach of individual privacy and served no purpose for pursuing an agenda, other than an incitement of retaliation.

I feel this comes down to an argument of “does the ends justify the means?” If we consider that using hacktivist methods could disrupt terrorist networks or promote a particular cause or ideology, are we willing to accept that it’s ok? At the end of the day, hacktivists have to accept that it’s as important for those they disagree with to share their opinion as it is for those they oppose.

So is hacktivism a genuine form of protest?

I’ve spent a while thinking about this but I believe that yes, hacktivism is a genuine form of protest. Clearly from the attention that has been given to causes utilising hacktivist methods it seems fair to say that hacktivism can be effective. I find it debateable though as to whether hacktivist methods can create sympathy for a cause, it’s difficult to ascertain the helpfulness of hacktivism for a particular cause. I do believe that it is entirely possible for hacktivist methods to be used in a justified and ethical way.

The serious nature of hacktivism necessitates that it be one of the final options of protest for when all other methods have failed. It’s a rather aggressive tactic that’s more likely to intimidate and aggravate rather than promote progressive discourse between two parties. Failing that, clever clogs hackers will no doubt find a way to promote their message.

Ubuntu 12.04 (And first post!)

Woo! First post!

So anyway, I recently re-installed Ubuntu on my desktop PC. I quite like Linux and I’d want to use it more often but I find that certain things don’t work so well. But anyway, here’s a few of the issues I’ve found with Ubuntu 12.04

Skype

Skype’s come a long way since it bumped up to version 4, the inclusion of a tabbed window interface really helps to streamline everything. Still, sound for 64-bit computers is still an issue. I’m thinking it’s a PulseAudio/Skype issue but all sound is badly distorted. It’s little things like this that compromise the usability for everyday users.

Unity

For all it’s bad press, I quite like the unity interface. The way the top bar becomes the title menus really does a lot to clear up the clutter everyday PC users face. However, I’ve found the integration of other applications to be somewhat lackluster, particularly in the system tray area which blocks other applications from displaying icons there unless you open the console and use gconf to change the white list.

The side-bar ‘dock’ that housed all of the application icons no-longer auto-hides which was a useful space-saving feature. Even better was when there was an option so you could customise it to your own personal preference! I don’t mind Ubuntu nabbing a few Mac OS-X UI features here and there because if there’s one thing Apple are good at it’s UI design (iOS Homescreen and notifications not withstanding); what I do have an issue with is Ubuntu taking Apple’s rather authoritarian approach to software development – there’s no space for personal customisation just like…

Volume Control

Ubuntu tended towards incremental steps of 6% if you change the volume, for headphone users that can be quite significant so if you’re not wanting to be deafened, you’d change that figure. The gconf utility used to allow you to do that just fine and dandy but now that option isn’t there. I got around this by using custom key bindings and this .volumeHack.sh file. I didn’t bother with the xbindkeys script and instead just used Ubuntu’s provided keyboard shortcut manager in system settings.

IM Clients

Empathy is still the default IM client on Ubuntu and I’m alright with that because I feel Pidgin has really lost a lot of ground to Empathy. The one feature Empathy still needs to implement to overtake Pidgin is combining contacts – to this day Trillian is the only IM application that achieves this properly. Combined contacts should:

  • Be able to be viewed as a mixed chat log from individual contacts, combined chronologically.
  • Prioritise a particular account protocol.
  • Easy switching and notifications of changing protocol within a single chat window.

Skype support would be lovely too but somewhat of a pipe dream at this point, though Skypekit is available for Linux embedded devices – surely should work on Linux desktop?

Rhythmbox

I can’t believe this issue made it past testing: the bottom of letters which hang below the line are cut off. Minor I know but mind boggling in it’s simplicity.

Chrome

Right-click on the tab-bar and tell it to use system title bar and borders to avoid displaying two title bars in full-screen mode.

Mounting

There really needs to be an easier way to manage mounted partitions in Ubuntu. Disk Utility is an excellent application but I feel it’s usefulness could be extended to managing mounted partitions, it’d be nice if it would automatically mount a particular partition that I use to store my important files but to do this I have to edit fstab – not too much of an issue for me but it’s minor things like this that will confuse Joe Bloggs.