programming
My first real Drupal project, part I: sending HTTP request for every page
Note: I know that pretty much no one who reads my blog is interested in Drupal stuff. However, when I was working on this project, I got a ton of help from personal blog posts about how someone solved some problem, so I’m going to do the same. Maybe some day this post will be useful to someone else.
The project:
One of my jobs works with Drupal, and recently I’ve been working on implementing my first really big project, which involves writing a custom module and really delving into the theming layer, which I knew almost nothing about when I got started on the project.
Here’s the idea: some folks out at the University of Pittsburgh have implemented a system in which a participating website can send them information about which pages are being viewed on their site. The Pittsburgh system files this information away in a database. Now that website can use this information to display social navigation information: say we want to know how often links A, B and C have each been visited in comparison with other URLS on the site. We can send an HTTP request with the list of the URLs we’re interested in, and Pittsburgh will send back the text for colored HTML span tags for each of those URLs. Those colors will indicate how often the link has been visited – a dark green square means that it’s been visited a lot compared to other URLs on the site; light green/white indicates that it’s rarely been visited.
My job was to figure out how to get this working in a Drupal system. As of now, I’ve got the prototype working; I’m going to explain how I did it in two steps: 1) how to send requests to the external server every time any page (not just any node!) is loaded, even if Drupal caches the page, and 2) using the theming of views to collect lists of links that we want to get images for, sending them out to the external server, receiving the response, and displaying them next to the link in the view(s).
Part I: Sending HTTP request for every page load
This was the easiest part of the project, but it took some delving into the Drupal API to find out exactly what functions were available for me to use.
My first thought was to use triggers and actions in the Rules module – a page load would be the trigger, and a custom piece of php using the drupal_http_request() function. However, there isn’t by default a trigger that runs whenever a page is loaded. There is one that runs whenever a node is loaded, but a) we want to send information for non-nodes, like view page displays, and 2) this trigger only runs when a node is loaded that Drupal hasn’t cached – and obviously, we want to do this even for cached pages.
So I decided to write the world’s simplest module instead. (It’s also the first module I’ve written that’s actually been used on a real, working site.) After poking around for a while, I found hook_boot(), a hook which runs exactly when I need it to: every time any page is loaded, whether or not it’s a node, and whether or not it’s cached (unlike hook_init()). I named the module http_request_per_page. (Not exactly creative, but hopefully it’s easy to tell what it does!)
The final version of the module is more complex because I’ve added a form for users to be able to change what exactly gets sent and where it gets sent to, but the meat of the module is simply the following function:
http_request_per_page_boot() {
//encode the current URL
$my_url = utf8_encode(urlencode($_SERVER[‘SERVER_NAME’] .
$_SERVER[‘REQUEST_URI’]));
//The URL of the external server we’re sending the info to
$your_url = “http://example.com”;
//Here I also encode up some other variables to send, but I’ll ignore that.
drupal_http_request($your_url,
array(‘Content-Type’ => ‘application/x-www-form-urlencoded’),
‘POST’, “url=$my_url”);
}
There you go! The world’s simplest module, at least before I expanded it. I also eventually renamed it, when I decided that the second step should also be rolled into the module (not just be implemented in the theming layer), but this was its original format.
The next step, getting and displaying the resulting images, was a lot harder – it took a lot more research and playing around before I even got started on the right track. I’ll save that for another day.
Continue reading My first real Drupal project, part I: sending HTTP request for every page
Learning in action!
It’s always fun when you get to apply something you’re learning in a class to real life, right when you’re learning it.
I’m taking a Database Internals class, and we’ve spent a lot of time learning about indexes – what kinds there are, how each kind works, the pros and cons, etc. This week we learned about implementing different query operators, different kinds of joins, etc.
Now, for those of you who don’t know, indexes help to make looking up certain information in a database faster. I’ve never see the actual effect myself – all of the databases that I’ve worked with have been little toy databases of my own (with barely any data in them), or pre-established databases that already had the appropriate indexes.
Last night, C needed some help working with some data for his work. He has an ENORMOUS spreadsheet that someone gave him, but he needed to make some changes to it. It either couldn’t be done in Excel or would take a ridiculous amount of time, so he asked if I could do it with an SQL query. I couldn’t actually think of a way to do it in a single query, but I did know a simple way to do it with a PHP script running a lot of SQL queries. So we loaded the data up into a MySQL database I created for it* (and at first the file was actually too large to import into the database!) and I ran my script. It took about a minute to run, maybe a little longer. I tweaked it a bit to fix a couple of errors, and then it worked perfectly.
When finished, I decided, just for fun, to add an index to the table on one of the fields that was used a lot in the query. Then I ran the script again, and it only took ~5 seconds! This was really exciting for me – yes, I’m a nerd. I really enjoyed getting to see the results of indexing in action, just like I’d learned about – and I especially love the fact that I know exactly how and why it worked!
Also, on a side note, computers are awesome. What would take hours and hours of tedious manual labor took only a few seconds with a script and a properly-indexed database. It still boggles my mind sometimes just how fast computers really are!
*Actually, getting the data exported from Excel and into the database was the hardest part, and took a lot longer than either of us expected – partially because the Excel file was so huge that it took several minutes every time we needed to do any operation on the data to get it ready for export.
A cookbook
Last summer I started leaning PHP and SQL (MySQL specifically) with my mom. Then, for my databases class last fall, we had to create a database-driven website that does a certain number of database operations. I took that project further, turning it into something that my family will hopefully use: a family cookbook.
My mom has always kept a Word document with our favorite family recipes, and she gave each of my siblings and my cousins printed copies. The problem is, if she adds or edits something, we don’t see the edits, and we can’t add things to our own (since we just have paper copies). Rather than ask her to send us each electronic copies, I decided to make a cookbook website for us. Now if I find a brownie recipe that I really like, I can add it – and anyone else using the site can see it. If she finds a type-o and fixes it, we all see the change.
Originally it was just intended for my family, but if any of you are interested in browsing or even adding your own recipes, feel free. If you create an account, you can comment on recipes and make a list of favorites. It’s a very basic site, not flashy or fancy, but it was all (with one exception) coded by hand, in order for me to really get a feel for PHP, CSS, etc., and I learned a lot while making it. I’ve got a list about a mile long of features I would like to add (not to mention just generally making it look nice) but those changes will be rolled out slowly, if at all.
Anyway, just thought I’d share.
Websites are fun.
I’m making a wedding website for my cousin, using Drupal. One of the awesome things about making websites is that lay-people tend to be really impressed by how much you can do relatively quickly, especially if you’re making a static website. Or if you’re using Drupal and there’s a module for everything you could possibly want to do (for a simple webpage, anyway) and all you have to do is download, activate and configure it. It makes me feel like I’m really smart – much smarter than I actually am.
That’s what I enjoy most about CS – making things with tangible results that make life easier in some small way. Of course, a mostly static Drupal website is a very very simple example of this, but I’m still enjoying it a lot. What I really need to do now is learn how to write some modules for open source software that I use. I should write a Firefox add-on that I can actually use, or Notepad++, or Drupal, or whatever. I’ll get right on that, in all of my spare time. :) But for now, time to get back to homework!
