Tag Archives: urllib2

Splunkd request using urrlib2, splunk-python-sdk

Splunk guys are so cool they put together a splunk python sdk. It comes stock standard with examples, explanations, documentation, etc…

So I’m all ready to play along and none of the examples work(for me at least).

When making a request to splunkd(the service interface) you need to be authenticated. That makes perfect sense. The logic the example uses doesn’t work though:

import httplib
import urllib
from xml.etree import ElementTree

HOST = “localhost”
PORT = 8089
USERNAME = “admin”
PASSWORD = “changeme”

# Present credentials to Splunk and retrieve the session key
connection = httplib.HTTPSConnection(HOST, PORT)
body = urllib.urlencode({‘username’: USERNAME, ‘password’: PASSWORD})
headers = {
‘Content-Type’: “application/x-www-form-urlencoded”,
‘Content-Length’: str(len(body)),
‘Host’: HOST,
‘User-Agent’: “”,
‘Accept’: “*/*”
connection.request(“POST”, “/services/auth/login”, body, headers)
response = connection.getresponse()

That bombs out immediately. Wiggled and jiggled some code and it still nothing. I then tried the very simple :

params = urllib.urlencode({‘username’:USERNAME, ‘password’:PASSWORD})

resp = urllib2.urlopen(url, params)


The response is a session key.

Works like a charm!!! I’m still not sure why the first example didn’t though. I have never used httplib and don’t intend to. But I shall investigate!!!

urllib and urllib2 have always been awesome.

Simplicity is priceless.




Posted by on February 1, 2012 in Uncategorized


Tags: , , , , , ,

urllib2, debug mode

So one of my work projects is sending some xml via http.

We use urllib, urllib2 to do all this work. (obviously)

I needed to check what was happening to the data being sent. Since most of the work is hidden by urllib my debugging was pretty much useless.

However urllib2 allows us to switch it to debug mode like such:

import urllib, urllib2
url = "some url"
xml = "some xml"

opener = urllib2.build_opener(httphandler)
opener.addheaders = [('Accept', 'application/xml'), ('Content-Type', 'application/xml'),]

req = urllib2.Request(url)
resp = urllib2.urlopen(req)


Leave a comment

Posted by on July 18, 2011 in Uncategorized


Tags: , , , , ,

Python , Send xml data as post request

So I needed to send xml data to a service. initially I did it like this:

import urllib, urllib2
url = "some url"
param_data = {'businesses': xml}
params = urllib.urlencode(param_data)
passman = urllib2.HTTPPasswordMgrWithDefaultRealm()
passman.add_password(None, url, username, password)
authhandler = urllib2.HTTPBasicAuthHandler(passman)
opener = urllib2.build_opener(authhandler)
opener.addheaders = [('Accept', 'application/xml'), ('Content-Type', 'application/xml'),]
resp = urllib2.urlopen(url, params)

Essentially sending the xml as a param.

This works fine.
The consuming service however does not expect a post param. Instead it expects the xml as data in the body of the post. To be honest this is the first time I had come across this. Weird.

So after a bit of complaining, nagging, googling, interrogating urllib and asking a much more knowledgeable coder. I changed the last bit to this:

req = urllib2.Request(url)
resp = urllib2.urlopen(req)

I used the xml as is without urlencoding it.

So this change will add the xml to the post body.

To add a set of params:

param_data = {'stuff':"things", "more_stuff": "more_things"}
params = urllib.urlencode(param_data)

req = urllib2.Request(url)
resp = urllib2.urlopen(req)

Nice and simple. The complicated things usually are.

For more reading up on urllib:

Official python docs

Doug Hellman stuff

Hope this helped?

Leave a comment

Posted by on July 1, 2011 in Uncategorized


Tags: , , , ,