Unknown Entity in Hibernate and other things

Hey everyone,

Let me get this out of the way first: I made a really careless mistake while writing the hibernate mapping files the other day (probably because I should have gone to bed) so I'm going to share it with you for my reference and so you can laugh at me.

What happened was that I had added a mapping for a class (RecordingCommentUserActivity.hbm.xml) and forgot to add a new entry to the hibernate.cfg.xml file. Oops. One of the side-effects was that Hibernate complained about requiring the fully qualified class name, which should have clued me in because you shouldn't ever need it.

The other thing that tripped me up was that a simple select query (using the full path) did not throw any errors in the log (that I saw). So I thought there was something wrong in another aspect of the project.

Of course when I tried to do something real like persist an object to the database, then it complained, giving one of those fun "Unknown Entity" exceptions.

As with a lot of problems, the hardest part was figuring out what was wrong. The fix took a few seconds. And everyone was happy.

In even more happy news, our updated security logic is nearly complete. We now track user activity and prevent duplicate voting of comments/songs/recordings/etc. I've been doing this off of another head on my branch so I can keep my main branch head free of the changes until they are more scrutinized. In the next few days I'll be switching away from security and finishing up an unimplemented voting feature for songs. That's good, because we allocated 1 week for the task on our estimates.

See you next time with more updates.

Unit testing java servlets with Mockito and JUnit

Hey everyone,

SO last time I shared with you about experimenting with code coverage tools. I've been considering code coverage now when writing new functionality and adding tests. One thing I've noticed is that our controllers (servlets) have no test coverage. Now I know you might say "but Chris, you shouldn't be having eg. domain logic in your controllers that need testing anyway!"

That's true guys, I agree. And we really try and maintain strict separation between the domain and controllers. Consider our testing goals for our controllers as the step before functional testing (via HtmlUnit for example). We just want to make sure that when the controller sends off the response, nothing is broken before it gets to the view.

To accomplish this, we can use a mocking framework (such as Mockito) to stub out the HttpServletRequest and HttpServletResponse interfaces that the servlet get/post methods contain.

Our target: a controller used to handle AJAX comment posting requests. When the user posts a comment, the server needs to:

1. parse the parameters and make sure they're all valid 
2. make sure the user hasn't tried to post too many comments in a short amount of time 
3. validate and sanitize the fields in case they contain malicious code (eg. javascript) 
4. create a comment and persist it to the data layer
5. send the new comment object back as a JSON object for display 

The controller doesn't actually do most of the work: it delegates it to the domain/data layers for validation and persistence. But we can still test to see if the servlet actually generates a correct response.

Let's say we want to check to see that the controller is doing its job sending a potential comment to the CommentValidator before it tries to post it.  We do this by sending a request with invalid parameters (such as a blank user name field) to the servlet.

First for our imports:

import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.when;

Then we create our stub HttpServletRequest object:

HttpServletRequest stubRequest = mock(HttpServletRequest.class);

Now we add the expected method calls/return values to the stub request. One parameter that the server should use is "type" specifying what kind of comment is being posted. So we can indicate that when the server retrieves the "type" parameter, it can return one of our choosing:

when(stubRequest.getParameter("type")).thenReturn("recording");

We do this for the rest of our comment's fields. Now we need a way to determine if the server validated the comment or not. In our case, the controller will send a JSON representation of a CommentValidationResult (which contains a message and an error type). For a blank text field, the error code will be BLANK_FIELD.

So let's test for the presence of BLANK_FIELD in our response (which will be a JSON string). To do this we create a StringWriter and a PrintWriter and then inject them into a stub response:

StringWriter sw = new StringWriter();
PrintWriter pw = new PrintWriter(sw);        
HttpServletResponse stubResponse = mock(HttpServletResponse.class);       
when(stubResponse.getWriter()).thenReturn(pw); 

Here we create a stub HttpServletResponse and inject our new PrintWriter so that when the servlet writes to the response's output stream, it will write to our StringWriter instead.

Now we call the method on the controller and make sure it contains the BLANK_FIELD:

controller.doGet(stubRequest, stubResponse);
String result = sw.getBuffer().toString();
assertTrue(result.contains("BLANK_FIELD")); 

We can run the test now and it will tell us if the controller sent back an error due to our comment having blank fields (as it should have done).

We can use this approach to test how the server responds to weird URL strings that are sent by mean people to disrupt our website in addition to testing for normal functionality. Maybe next time we'll try and test the view layer...


*Oh, and I finally integrated syntax highlighting courtesy of SyntaxHighlighter

Code coverage, Hibernate integration, and other fun things.

Hello everyone,

Lately our project has been progressing very nicely. My colleague had written a small program to parse raw data into an SQL-friendly form, so we are not restricted to test data anymore. This has the nice side-effect of boosting motivation because we are dealing with the Real Thing now.

My focus lately has been on adding a comments section to the web app. In our schema, we have a few different comment tables for different types objects. Luckily in the Java code we can just make all of them derive from a BasicCommentBean so we can be lazy. I've made the comments section as generic as possible and self-contained.

With this setup, you can reference comments.jsp and set the required attributes (a comment data source namely) and everything else is handled for Free by CommentController. We also perform server-side validation on the user input and sanitize it via the apache.commons.lang library to prevent malicious code execution on the client. Oh and we have some nice ajax and animation going on courtesy of jQuery...

Now that we're using real data, we've integrated Hibernate. Oddly enough, the process was surprisingly painless, but I wouldn't be care-free just yet... I don't have a ton of experience with Hibernate, but so far it's pretty straightforward once the mappings are working correctly.

One last think: I've been experimenting with code coverage via the EclEmma plugin for eclipse. It's nicely integrated and I've used it to see how much code coverage our unit tests are providing (63% ... looks like room to improve). Over the next few days/weeks/etc. I hope to use it more extensively to improve our tests and coverage of the important parts of the project.

Implementing interaction between jQplot and java servlets

Today I'm going to share my experiences developing a way to let users plot graphs of certain interesting data.


Way back when, we originally were going with the Google chart API to accomplish this. It's simple and it looks good. However, we decided instead to take a jQuery approach (for learning and to avoid making a call to a second server) with jqPlot. jqPlot is a free graphing library which generates the graph on the client side and accepts data in a convenient format.

After playing around with it, we were able to develop a nice implementation:

Now aside from a few things that need to be worked out (such as fractional years), it looks good and was very simple to implement. Our approach to this using java servlets was:

• Client sends an AJAX graph query to the GraphsController class with a parameter ("getgraph") set to true to signify a graph request
• Servlet gets request, builds a GraphRequestBean and validates it. If validation fails the server sends back an error object using the JSON library for Java called google-gson. 
• Servlet sends request to a GraphsManager, which generates the Graph by interacting with the data layer. 
• Graph is sent as a JSON object and is read by the javascript, where a call to the plotting library is made. 
• User gets nice looking graph

 We also added some error handling and the ability for users to regenerate the graph through a link. I'm pretty excited about this implementation (and jQuery in general...) because it worked very well. And also I have even more respect for unit tests because they saved a lot of time debugging, since I was constantly changing code around while developing.

Design issues: Keeping it simple and practical

On the project I'm collaborating on with a former classmate, we are really trying to make some good design decisions. Stuff like:

• Code reuse
• Decoupled design (ie. data separate from domain) 
• Testability (programming to interfaces) 

At the same time, we want to keep the design simple and easy to follow. One of the things we were trying to solve was handling very dynamic queries in an elegant way. And we came up with a solution. And it was elegant. It had everything based on some common interfaces, used generics to handle them, and even reflection sprinkled in.

And then we got rid of it in favor of a simple, more practical solution. 

Actually, he proposed the simpler solution. His solution was based on his practical experience developing applications. Although our elegant method would have been pretty cool, it would be less maintainable and take longer. And remember that in the end, your users don't care about your elegant solutions. They just want your stuff to work.

Using the Google Chart API (cont'd) and caching images for retrieval

Yesterday I talked about using the Google Chart API on the server to send chart images to a client in response to a query. Today after meeting with my colleague, he revealed a more straightforward approach than trying to cache the generated images and retrieve them from the image tag.

Well, we're still doing that, but instead of generating the image data and sending that, we generate just the url string to send to the google chart site. Then we send that to the client, where it is embedded into an image tag. Now we don't have to cache any image data. However, we still might end up caching it for a simple reason. It would be nice for users to get a quasi-permanent link to a generated graph, so that they could type "address/graph?id=..." instead of having to put up with a Google Chart url string (which might be long depending on the dataset).

There are still some more things to add to the code before it is ready to integrate with the main project, but overall it will make a pretty nice addition.

Using the Google Chart API to send images in response to a GET request

While we are finalizing the design for our project, we've each been working on a different small task. The task I've been working on is implementing the ability for the user to make a chart based on the data set they are looking at. They should be able to say "hey, I want to see the number of concerts each year from 1990-2010," and it should work. To do this, I've been using the Google Chart API to generate the chart on the servlet and then send it to the client.
To accomplish this, I've been following this answer to a question on stackoverflow.com The process works like this:

1. Client sends a POST request to the servlet with parameters specifying the data set and ranges to use
2. Servlet builds a GraphBean object from the client's request and sends it to a GraphBuilder
3. GraphBuilder constructs a url and sends a GET request to the Google Chart API url
4. The image generated is given a unique id and stored in the servlet class
5. The servlet writes a response that references the unique image id in an image tag
6. The image tag sends a new request to the servlet with that image id, and the servlet sends the image back
7. The image is removed from the servlet

Actually, the last 4 steps aren't implemented yet, but I expect to have them done soon. I've been developing the first steps in a regular java application instead of using servlets. However, the integration with the servlet will be (hopefully) painless. Once this is done, we should get some nice pictures out of it when our database is set up.