Design Document

This week I've made a simple XACML template that contains the information needed by an RAdAC policy. I have yet to find a paper with concrete details on how anything dealing with RAdAC should be done; I've only found high-level descriptions of it. I think as long as it implements some mechanism to allow "operational need" to trump "security risk" it falls under RAdAC. Most of the papers I've read discuss multiple policies converging to determine risk, need, and ultimately a final decision. I am using a simple model that describes the basic functionality of RAdAC using a single policy. I have started working on a design document and will finish it soon.

Pex and XNA

(July 12th - July 18th)

Had another meeting with Dr. Xie about the unit testing using Pex with my game. I didn't have a good test (I need to write more unit tests with Pex to get tightly controlled test). Over the week I kept looking at the code and it seems that the way that the parameter is used and the way Pex uses parameters seems to have some conflict.

The class/method is:
public class Entity
{
...
public void handle_collision(MapEngine e)
{ ...}
...
}

The method basically handles the collision of the Map and the entity using it. Inside the method, if the position of the entity's collision box intersects with the a "wall" on the map, the entity's collision state changes to being hit and the entity's position is moved.

Now the problem is that the MapEngine object is only used as a reference for the character to see where it is on the map. It doesn't "change" per se, it only changes which map is the current map (and it has references to the previous and next map if any). The map itself doesn't change, the thing that does, is the entity's position. Depending on the position of the entity on the current map, it will collide or not, not depending on the current map.

The problem with Pex is that it uses different inputs to get high code coverage. In this case, the MapEngine is pretty much static in that the map is not changing everytime the method is called but the position of the entity is (and it is automatically changed in another method). If the input was the position value, then Pex would be able to test it. In order for collision detection in my game to work and for Pex to analyze it, I would have to change the implementation significantly. I am going to write up a thorough explanation of my problem to Dr. Xie next week and also test to see if there is anything I could do without having to test just in case I missed something.

Unit Testing with XNA continued.

(June 28th - July 11th)

I met with Dr. Xie and we reviewed on what I've been working on in the previous weeks. Instead of looking at the overall picture of Game Testing Automation, I'll focus more on certain things that game application use with xna and try to find anything that is an area of we interest like why can't we use certain things (Pex) for xna games in certain situations and write up why we can't. I did create a unit test that didn't use any content loading in my game that passed just as any other application. Earlier I had a problem with doing unit testing anything that used the Content Pipeline to load content for my game Tears (independently made but just used for researching purposes), I found a framework called Scurvy that alleviates that problem. I will retry previous tests that included content loading to test it out. Dr. Xie also recommended to look at GUI testing automation techniques to see if it's applicable to game applications.

Novaleaf Software & Unit Testing with XNA

(June 14th - June 27th. Late)

I interviewed Jason Swearingen, founder and president of Novaleaf Software (http://games.novaleaf.com/). His company tried model-based testing on his games but felt that it was wasting time and that he would rather just design the game and then go from there. Not really helping me at all. I tried to do some unit testing with XNA using Visual Studio Team System 2008's integrated Unit Testing software. I couldn't get anything that used loading content to work properly. Did some more debugging on my PongGame using NModel, still a little confused on the use of "features" but still working on it. Tried Pex a little bit but didn't spend expensive time with it. I will in the next two weeks. Going over and fixing some grammatical errors to my paper for Dr. Xie.

NModelling and Dishwasher

(May 31st - June 13th, late)

I continued to search for articles on anything related to game testing automation or any automation techniques that are used in the game industry to see why they used that technique or why they couldn't use another. Had to do some debugging on the GameModel program I made with the NModel Framework. I used that program to start implementation of a Pong game so that it will be easy to fill in. NModel has an aspect of it's framework called "Features" where you can add or take out parts of the program in a modular fashion. Just have to reread the chapters on it to get a good grasp on how to implement it since their code didn't seem to exactly explain on how to use it.

I also interviewed the creator of The Dishwasher: Dead Samurai, which was made with XNA. It's a Microsoft published game. James Silva, the creator, didn't use any automated test techniques, just the regular manual testing. I'm having a hard time finding any information on automated game testing techniques anywhere online but will continue searching. I started my paper for Dr. Xie on what I have been looking for.

First Post

(May 17- May 30)

So this is my first post concerning testing in video game application using Microsoft's XNA Framework. Dr. Xie recommended me to look at and read Model-Based Software Testing and Analysis with C#. It basically tests the model of your application and see if it can finds an errors like dead states and live states . They developed their own framework called NModel so that you can test out your application. I implemented a very simple application called LightSwitch, with a simple on and off state to get familiar with the framework. I then implemented a control flow of the game loop within a XNA developed application. I also contacted Shawn Hargreaves of the XNA Development Team at Microsoft for some information on any (if any) game testing automation techniques.

XACML and RAdAC

This week I have been familiarizing myself with XACML. It seems to be a very flexible policy language; albeit a bit complex. I've found the programmer's guide for Sun's XACML implementation to be a good introduction, but I'll be skimming the OASIS documentation to get a more thorough grasp of the standard. I am to use XACML to implement an RAdAC policy. To do so, two metrics will have to be calculated: a 'Security Risk' and an 'Operational Need'. I'm sure they will be calculated by some sort of function.

Paper Completion and OOPSLA

I've finished writing my paper. Last weekend was spent converting it to the ACM Proceedings format and reducing it to two pages. After reading Dr. Xie's PowerPoint, “Common Technical Writing Issues,” I found many grammatical mistakes in my own writing. It was very helpful in getting the review process started. After I addressed Dr. Xie's comments and fixed the errors discussed in his presentation, I submitted it to the OOPSLA Student Research Competition; I also submitted a poster describing the approach discussed in the paper.