More servlet testing with Mockito and JUnit

Hello everyone,

I had some time to implement some more functionality for The Project today, so I thought I'd share more with you about testing our servlets.

Last time we were testing a server's JSON response to make sure it called the right validators and handled bad inputs.

Today we're testing the response of a POST request (also via AJAX). Our target: A simple controller that handles some very basic security (sufficient for our purposes). We want to send a request to the server and set a session variable if the user is authenticated. We are using JUnit integrated into Eclipse.

So first, in our TestAdminController*:

import static org.junit.Assert.*;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.verify;
import static org.mockito.Mockito.when;
import org.junit.Before;
import org.junit.Test;
import org.mockito.Mockito;

 Naturally we'll have more than one test case, so we will put some of the common code into a setup() method that is run before each test case is executed:

private AdminController controller; 
private HttpServletRequest stubRequest;
private HttpServletResponse stubResponse; 
private HttpSession stubSession;  

@Before
public void setup()
{
    controller = new AdminController();     
    stubRequest = mock(HttpServletRequest.class);
    stubResponse = mock(HttpServletResponse.class); 
    stubSession = mock(HttpSession.class);  
}

As always we've stubbed the request and response. This time we're also stubbing a session since we'll be setting a session attribute.

Now we set up one of our test cases: simply testing that, given a valid user, the session variable is set by the server. To do this we first hook up our HttpServletRequest so that when the server attempts to retrieve the current session, it returns a stub instead. We also setup a stub user manager (which handles authentication) so that we don't have to worry about usernames and passwords and all those things:

when(stubRequest.getSession()).thenReturn(stubSession);
when(stubRequest.getParameter("password")).thenReturn(UserManager.SUPER_SECRET_ADMIN_PASSWORD); 
UserManager stubUserManager  = new UserManager(); 
when(stubUserManager.userIsAuthenticated(Mockito.anyString(), Mockito.anyString())).thenReturn(true)

We use Mockito.anyString() because we don't care what credentials have been retrieved, but we always declare the user as valid. This decouples the test from the UserManager class.

Finally, we are ready to initialize the controller:

AdminController controller = new AdminController();
controller.setUserManager(stubUserManager);
controller.doPost(stubRequest, stubResponse);         

To ensure that the server set the session variable, we use verify() to determine that the HttpSession.setAttribute() method was called:

verify(stubSession).setAttribute("authenticated", true);

Now we can run the test and if everything is working, it should pass. In this way, we have ensured that at least some parts of the controller are working so any problems will be on the client.

Well, that's all for now!

*I put the imports here because I hate it when I find a solution to something, but the author doesn't list the imports and the library isn't easily searchable!

It's finals week! (almost)

Hey everyone,

Expect fewer updates in the next two weeks -- finals are coming up and this week is the last week of class. Physical and quantum chemistry are going to require some Dedication. Sadly no computer science classes this semester, ah well.

Stay tuned for project updates.


-- Chris

Unit testing java servlets with Mockito and JUnit

Hey everyone,

SO last time I shared with you about experimenting with code coverage tools. I've been considering code coverage now when writing new functionality and adding tests. One thing I've noticed is that our controllers (servlets) have no test coverage. Now I know you might say "but Chris, you shouldn't be having eg. domain logic in your controllers that need testing anyway!"

That's true guys, I agree. And we really try and maintain strict separation between the domain and controllers. Consider our testing goals for our controllers as the step before functional testing (via HtmlUnit for example). We just want to make sure that when the controller sends off the response, nothing is broken before it gets to the view.

To accomplish this, we can use a mocking framework (such as Mockito) to stub out the HttpServletRequest and HttpServletResponse interfaces that the servlet get/post methods contain.

Our target: a controller used to handle AJAX comment posting requests. When the user posts a comment, the server needs to:

1. parse the parameters and make sure they're all valid 
2. make sure the user hasn't tried to post too many comments in a short amount of time 
3. validate and sanitize the fields in case they contain malicious code (eg. javascript) 
4. create a comment and persist it to the data layer
5. send the new comment object back as a JSON object for display 

The controller doesn't actually do most of the work: it delegates it to the domain/data layers for validation and persistence. But we can still test to see if the servlet actually generates a correct response.

Let's say we want to check to see that the controller is doing its job sending a potential comment to the CommentValidator before it tries to post it.  We do this by sending a request with invalid parameters (such as a blank user name field) to the servlet.

First for our imports:

import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.when;

Then we create our stub HttpServletRequest object:

HttpServletRequest stubRequest = mock(HttpServletRequest.class);

Now we add the expected method calls/return values to the stub request. One parameter that the server should use is "type" specifying what kind of comment is being posted. So we can indicate that when the server retrieves the "type" parameter, it can return one of our choosing:

when(stubRequest.getParameter("type")).thenReturn("recording");

We do this for the rest of our comment's fields. Now we need a way to determine if the server validated the comment or not. In our case, the controller will send a JSON representation of a CommentValidationResult (which contains a message and an error type). For a blank text field, the error code will be BLANK_FIELD.

So let's test for the presence of BLANK_FIELD in our response (which will be a JSON string). To do this we create a StringWriter and a PrintWriter and then inject them into a stub response:

StringWriter sw = new StringWriter();
PrintWriter pw = new PrintWriter(sw);        
HttpServletResponse stubResponse = mock(HttpServletResponse.class);       
when(stubResponse.getWriter()).thenReturn(pw); 

Here we create a stub HttpServletResponse and inject our new PrintWriter so that when the servlet writes to the response's output stream, it will write to our StringWriter instead.

Now we call the method on the controller and make sure it contains the BLANK_FIELD:

controller.doGet(stubRequest, stubResponse);
String result = sw.getBuffer().toString();
assertTrue(result.contains("BLANK_FIELD")); 

We can run the test now and it will tell us if the controller sent back an error due to our comment having blank fields (as it should have done).

We can use this approach to test how the server responds to weird URL strings that are sent by mean people to disrupt our website in addition to testing for normal functionality. Maybe next time we'll try and test the view layer...


*Oh, and I finally integrated syntax highlighting courtesy of SyntaxHighlighter

Code coverage, Hibernate integration, and other fun things.

Hello everyone,

Lately our project has been progressing very nicely. My colleague had written a small program to parse raw data into an SQL-friendly form, so we are not restricted to test data anymore. This has the nice side-effect of boosting motivation because we are dealing with the Real Thing now.

My focus lately has been on adding a comments section to the web app. In our schema, we have a few different comment tables for different types objects. Luckily in the Java code we can just make all of them derive from a BasicCommentBean so we can be lazy. I've made the comments section as generic as possible and self-contained.

With this setup, you can reference comments.jsp and set the required attributes (a comment data source namely) and everything else is handled for Free by CommentController. We also perform server-side validation on the user input and sanitize it via the apache.commons.lang library to prevent malicious code execution on the client. Oh and we have some nice ajax and animation going on courtesy of jQuery...

Now that we're using real data, we've integrated Hibernate. Oddly enough, the process was surprisingly painless, but I wouldn't be care-free just yet... I don't have a ton of experience with Hibernate, but so far it's pretty straightforward once the mappings are working correctly.

One last think: I've been experimenting with code coverage via the EclEmma plugin for eclipse. It's nicely integrated and I've used it to see how much code coverage our unit tests are providing (63% ... looks like room to improve). Over the next few days/weeks/etc. I hope to use it more extensively to improve our tests and coverage of the important parts of the project.

Enabling the jqPlot highlighter to use tick labels instead of tick indices.

Background

So I'm a big fan of jqPlot, which we've been using in our project for our graphing page. Right now it's coming along pretty, well, we even have Real Data now.

Real Data.

Now, one "gotcha" you have to watch out for in jqPlot is how non-numeric data points are rendered. For example, let's say we have a plot of cities versus year. Our x-axis is simple, it's just numeric year values. On the

y-axis however, we have city names. Now I'd expect a decent graphing library to be handle non-numeric axes just as well as numerical ones. It would be "good enough" to just display the data points in no particular order, but even better if you could specify a custom comparator. Right now we handle that server-side and just send a pre-sorted list to the client for jqPlot to render.

The Problem

The way jqPlot handles these text data points is a little awkward (unless I'm missing something). Instead of specifying a point such as (1986, "Boston"), you specify (1986, 0) and then have a list of tick labels, where ticks[0] = "Boston."

So a little weird, but not deal-breaking. However, if you try and use one of jqPlot's plugins called the Highlighter, things don't work so well. Highlighter enables displaying of the x/y values when you hover the mouse over a data point on the graph. Unfortunately, if your data point is (1986, 0), it will display "1986, 0" instead of "1986, Boston." Not good.

As far as I can tell, it's not possible to use highlighter to display string values instead of the tick index. So, being bored and not wanting to study for a chemistry class, I dived into the jqPlot highlighter source to change that.

The Source Code

After spending some time in the source code and experimenting through Firebug, the relevant function responsible for displaying the data point is called showTooltip. This function first sets up a list of y-values (since a series can have more than one y-value), and then adds the x-values depending on the options specified.

For example, the relevant line of code to display the y values of the data point is:

ystrs.push(yf(yfstr, neighbor.data[i]));

 Where neighbor.data[i] is the actual value of the data point, yf is a function that formats the string to be displayed, and yfstr is the "format string" which follows the conventions similar to sprintf.

How do we change this so that it will display the tick mark value instead of just the y tick index? Well, it turns out that showTooltip is passed a series object, which contains the ticks array already. The exact name of the array is series._yaxis._ticks. There is also one for the x axis.

How do we know the tick index? Our y data point will indicate it, so that the line above can become:

ystrs.push(yf(yfstr, series._yaxis._ticks[neighbor.data[1]].label));

 Where neighbor.data[1] is the first y-value. Since we don't want this to happen all the time (just for non-numeric data points), we can add a few parameters to the highlighter plugin to specify whether to enable displaying of tick labels or not. In our situation, we know when to enable it because numeric data points have no associated ticks array that's sent by the server. Some similar modifications for the x-axis give us the results we want:

jqPlot modified Highlighter plugin in action

Success!

Now you ask, why go through all this hassle? Well, our particular reason was that we can have too many data labels (such as 50 cities) so the client will cut off displaying them. This way, the user can still hover over interesting data points and see what the values are.

For anyone interested, the modified highlighter plugin is hosted on my website. The relevant Highlighter parameters to set are useXTickMarks and useYTickMarks.

Multiple heads of the same branch using Mercurial

(No progress on the chemical structure tool -- I've been working on my Other Project)

Anyway, today I'd like to tell you about one of our recent problems that we had to overcome using Mercurial. Mostly for reference purposes so I don't forget about the solution, but if you find it useful that's great.

So, we've been using Mercurial for a few months now, and I personally like it a lot. However, that doesn't mean we've been without our share of "learning experiences." Recently we discovered how important it was to pull before pushing changes on a commonly worked on branch (in this case, it's our Test branch).

Since this branch gets merges from our development branches quite often, there's no guarantee that you have the most up-to-date version when you're merging with it. So one of our philosophies is to just always pull before merging anything.

Now what happens when you don't do this? Well, let's look at our scenario:

• Changes are merged into Test and pushed to the remote repository from machine A. Everything's OK 
• Machine B also merges changes into Test but does not see machine A's changes on Test.
• Machine B attempts to push, Mercurial warns that there will be two heads on the Test branch. Otherwise it would have to decide whose changes are the "official" ones, and that's not it's problem. 
• Machine B then pulls and observes multiple heads on the Test branch. 

Oops. Now, back when we were working on a group project for school, we had this problem but I didn't understand it enough to fix it. This time however, we fixed it and now it's just an inconvenience. We pull the changes on machine B so we see the multiple heads, and just merge one head into the other. Problem solved!

And everyone's happy!

Now, I'm sure some of you are thinking "duh, Chris, what's so hard about that?" Sorry guys, I'm still working on a complete understanding of Mercurial. But that's one less step we have to deal with now!

Programming to interfaces and abstract classes

So while working on the (relatively) small chemical structure drawing project I've been trying to incorporate maintainability into the design. After all, you never know when you'll be reusing the code you've written in the future (I never thought I'd be using the rendering code from nearly a year ago, and it looks terrible!)

One of the ways I'm trying to achieve this is by maintaining a strict separation of graphics-based code in the CompoundViewer from the more behind-the-scenes stuff. This is slightly problematic because by nature, the CompoundViewer serves as the graphical front-end to the graph representing the molecule. In my chemical database project, the CompoundViewer is tightly coupled to using GDI as its graphical front-end, resulting in Graphics objects sprinkled throughout the code.

Well, as much as I like GDI, it's not infeasible to want to use a different rendering library (for example, Direct X for a 3D version)

To keep the CompoundViewer reusable, I've created an abstract CompoundViewer class which implements the generic code that doesn't depend on the graphics library (which is mostly all of it).

Then, for a specific library there is a concrete class (such as GDICompoundViewer) which has any specific code in it. The CompoundViewer then provides a constructor which takes in an IRenderer object (an interface so we can stub it easily in testing).

Some Renderer implementations might need special initialization through their constructor, such as the GDIRenderer which requires a Graphics object. For this case, there is an abstract RendererFactory class which provides an IRenderer object in its createRenderer() method. The GDIRendererFactory takes in a Graphics object in the constructor, and creates a new GDIRenderer object with that Graphics object and returns it in the createRenderer() method.

With this architecture, most of the CompoundViewer code is in the abstract class since it is generic, and there is very little in the concrete graphics-based classes. The benefit of using interfaces allows us to stub out dependency on a graphical front-end while testing. Imagine setting up Direct X in your unit tests. What a mess!

More on chemical structure drawing with C#, TDD, and a dash of algorithms

Progress 

So recently I've been working on implementing a chemical structure drawing tool in C# using WinForms. Along the way I've been adhering to TDD (test-driven development), and it's been a pretty positive experience. I've managed to avoid using the debugger and have a very clear separation between UI and logic code.

The "industry standard" for representing chemical structures is to represent atoms as nodes and bonds as edges of an undirected graph. Although atoms and bonds can have an extensive set of details, I've been sticking to a set of simple properties in the meantime. So far the back-end of the program is fairly developed, and on the GUI front-end I have basic functionality such as creating and deletion of atoms/bonds.

Quirks 

There's always a random quirk that you end up having to implement, in my case it was allowing the user to click on a bond. Now, a bond is just a connection between two (x,y) points, so to test for a mouse click on a bond we need to test the distance of the mouse position from the line segment representing the bond.

Oddly enough, I had a working C# solution to that problem awhile ago (it was a pain to find one too, adapted from this stackoverflow question!)

But the other 'quirk' is a lot more interesting than geometry, and it involves generating a string of text that represents a chemical compound. One of the important formats to do this is called SMILES, and I would eventually like the program to be able to generate and/or parse it.

The problem? The format is kind of complicated and hairy, and has to account for a lot of different chemical properties. So for the moment writing a parser is a task in itself.

Generation using depth-first searching

Luckily, generating the SMILES code is easier. The most simplistic way is to simply do a depth-first search through the graph, while identifying nodes that appear twice (these will be part of rings). This algorithm will generate "correct" but not canonical or normalized SMILES strings, although the programs I've used will all parse them.  To get more involved requires identifying rings, taking into account other bond/atom properties, and all kinds of hairy stuff.

In fact, in the big chemistry libraries (in Java for example), the code to do this is very long and hard to understand. At least, I can't understand it. So for the moment I'm stuck trying to reinvent the wheel, but starting a whole lot simpler.

And that's all for today. Check back next time for more progress and other fun topics.